​recording silhouette

[Eva]

Hi there,

I would like to find a way to record/play only the silhouette of some dancers who are moving in vertical dance, that means with climbing equipment. I know I can do something with an infrared camera (I have now a Kinect - but how can I get a better and more stabile resolution? is there other ways and Ideas how I can make this happen? All I want essentially is to get a silhouette of a body moving. Essentially I want to color it or put videos etc. into it:) But that I have found solutions for. Are there any ideas out there that might help? Here is an image of some of what I want to create:)

All the best

Eva

[Woland]

@eva

This might be helpful: https://community.troikatronix.com/topic/3316/video-infrared-tracking-workshop-from-werkstatt-2016?_=1724149070798

[Woland]

@eva

Also my answer from this thread about using Isadora to projection map onto a moving person, particularly this part, might be useful info:

Kinect is good for small area tracking. Larger areas where you only need to track and project from one direction you can flood the wall behind the performer with infrared light and use an IR-sensitive camera in front of the performer so that the performer shows up as a dead spot in the IR camera feed, which gives you the silhouette you need to start warping in order to get it mapped onto the person

[CitizenJoe]

Would the suf=ggestions from this thread work?
Interesting Ipad OSC apps | TroikaTronix Forum

Data OSC on the App Store (apple.com)

iKeleton OSC on the App Store (apple.com)

Cheers,

Hugh

[Fred]

Re: [recording silhouette](/topic/8947/recording-silhouette)

It sounds like you want to make recordings not do this in real time. If it’s real time then IR flooding of the background and using and IR camera feed would be the best bet. 

You can also use some kind of green screen to do this. If you need really professional results you can also go for a hardware green screen tool like this https://www.blackmagicdesign.com/au/products/ultimatte which can do some amazing stuff and likely be rented. This unit can also help with recording videos for playback. 

If it is all pre recorded the process you want is called rotoscoping. Most good video post production tools can do this in some way. The full version of Davinci resolve does this well. Other tools like Mocha are very good at this - Mocha was the industry stand for a long time. 

There are also rotoscoping services where you can send a video and have it extracted (usually in China). This will be done with a combination of computer tools and human correction. 

But of course AI is great at this and this tool is a good example https://www.vidio.ai/tools/online-rotoscoping

Once you have the human form isolated from the background you can extract the silhouette easily. 

Fred

[jtsteph]

@eva I have done many project using greescreen, rotoscoping and AI based rotoscoping. It can be quite labour intensive, but if you are able to prep the staging for your capture, you can make quick work of it. Here are a few routes to accomplishing what you would like to do.

- green/blue screen: your wall would be painted chromakey blue or green and then the performers can be keyed out reasonably easily.
- ai roto: having strong contrast between your performs and their background helps ai figure out what is what. Light coloured wall, performers in dark solid colours.

- mocap: I am working on a project that also uses silhouettes. We captured the performance with motion capture technology. It is so utterly flexible to work with, it really inspires new ways of looking at movement! This can be done with inertia suits like Rococo suits or there are many AI based mocap tools that involve shooting the performance with a few cameras and uploading these materials to a service that will compile the video and return motion capture armatures for you to work with in blender or whatever cg package you would like to use.

- J

[dbini]

Many years ago I worked with a circus performer who specialises in Chinese Pole. We filmed her in a green screen studio. it was pretty difficult because the vertical movement range was much bigger than with someone working on the floor, so the greenscreen backdrop had to be huge.
More recently we put a pole dancer in a MoCap suit and the results were surprisingly good. If you want really clean silhouettes, I think the best way would be to MoCap the movement and generate the silhouettes in Blender or Unity or Unreal - you then have full control over the body shapes and camera angles.

[Eva]

@fred said:

rotoscoping

 Hi fred, it sounds actually very easy. So, it can also work if you have wires that are hanging in front of the dancers? They are connected on each side of the hip so that they can spin.Thanks for helping:)
Eva

[Fred]

@eva

It’s not that it is easy but the tools to do this kind of work are mature and advanced. You will likely, with any approach need to evaluate the workflow in two stages. The first, how well the initial background removal works and second how difficult it is to refine the separation- you will likely have to do some manual removal of the wires and general cleaning of the matte. 

Personally I would suggest using Davinci resolve as its newest magic mask iteration is very advanced and uses Resolves own AI engine to do the extraction. This is only available in the paid studio version not the free version of the software. 

Here is a tutorial that can get you started:

https://youtu.be/HAlfApCFQwI?si=a6Sv7E8QbwvzTqGO

here is another video that shows a few more advanced features including using fusion (which is inside Resolve), as well as removing things from being tracked (which you may have to do with the wires). Overall though this works very well and is simple to use with lots of options to refine the mask.

https://youtu.be/5PXu2l5YS1Q?si=UzZ9GfFNzhYTICAo

Definitely make things as easy as possible for the system so use steady camera shots, great lighting (and try to get good contrast between the foreground and background. Use a good quality camera, something that has a global shutter and at least 422 colour encoding. 

Getting good quality video with high contrast foreground and background will make any of the workflows you use much easier. 

With good quality video this should not be too difficult. There is also a lot you can do to edit and refine your video once you have a basic mask prepared even if it is not perfect, using other tracking tools inside resolve. You can also export the separated footage with an alpha channel that can be used inside isadora or any other editing software if you want to work with the content further in another application. 

Note that this requires some good compute resources, so a high end GPU on windows or a late model mac with Apple Silicon (m1, m2, m3) and good amount of memory. 

Fred

[Eva]

@fredThank you, that's really helpful. Im in the process of testing things and finding out the right solutions, and your details are very helpful:)

[Eva]

@fred 

I will have access to a blackbox where we will record the video, but I only have some hours with them one day. I imagine that asking them to wear something in a color could help. and then I will have to see what happens with the wires, that seems to be the main challenge I remember. I dont have the paid version of DaVinci, but it seems I might have to invest...