Tracking two Dancers

PatrickPagano

Hi

I have a request to track two dancers and project on them at the same time. Is there a best practice for this in Isadora? I am considering IR might be nice to track their movements and get there silhouettes and then map the video into the silhouetted area? I would love to here any success stories with this request i am seeing more and more. There was talk that the dancers would be wearing white and i am wondering if that will make things easier or more difficult. I have done some blob tracking but that was just for movement and firing off videos, so i would love to hear some suggestions!

Thanks in Advance everyone

Patrick

mark

Dear Patrick,

The Eyes++ actor can track multiple blobs, but it's always going to be a problem when they touch or pass in front of one another because they become one blob; furthermore, it's really impossible to know which one was which when they separate again. That means dancer 1 and dancer 2 may get reversed when they separate.

Now, I've heard tell that the Kinect can track multiple people. But I have no links or other information to share on this. Maybe someone else will chime in on this.

The only sure fire way I know (barring a more sophisticated technology than Isadora offers, e.g., the kinect) is to use color tracking -- i.e., use the Chroma Key actor to identify the performers based on the color of their costumes. But this introduces a different problem in that you'll end up with people in very bright, solid colors. Not really the best dramaturgical option.

As far as wearing white, if the IR light on the back wall behind the dancers is strong enough, and if there isn't too much incandescent light on them from the front, I think it will work. In my piece 16 [R]evolutions, you can see that the dancers are wearing white -- take a look at the [video](http://vimeo.com/2113242).

I hope that leads you in some useful direction.

Best Wishes,

Mark

fifou

Hi Patrick,
It seems (If I understood well) you need to have a key signal only (white shadows of dancers on black bakground video signal). You don't need to track dancers movements for that.
I would do it this way:
Kinect -> NIMate -> Syphon -> Isadora
Kinect device directly produce a key signal. NIMate software (demo work for 2 hours, then rerun) drives kinect, gets the key signal and route it direcly to syphon.
IsadoraCore receives syphon videostream via syphon QC plug-in and makes all the rest .

I've done it with one guy, but with two, it should work too. But I didn't tried on large stage..

Best wishes
Philippe

Skulpture

This might help a little;

http://vjskulpture.wordpress.com/2011/02/13/the-problems-with-motion-tracking/

stj

Hi Patrick,
it seems you are looking at two distinct things here:
1. motion tracking techniques
2. video projection mapping solutions

Regarding motion tracking techniques you already have great feedback here. Maybe two details: if you don't have a Sony with a nightshot for the IR solution, you can always "hack" a webcam (= use an IR filter). Regarding the Chroma Key method, you can also use white clothes on your dancers/performers and light them the color you are tracking. Like Mark said, it all depends on your dramaturgical choices.
Personally, I prefer the IR technique with background substraction, and/or the Kinect solution Philippe mentioned above.

As far as video projection mapping is concerned, you can check out Jamie's work with Wlad: http://www.jamiegriffiths.com/kinect-with-isadora/

You can also look into combining Isadora and MadMapper softwares to get more control over the dynamic mapping. Here is a tutorial by Isadora user Matthew Haber:
http://www.madmapper.com/connecting-issadora-to-madmapper/

Best Stephan

PatrickPagano

SO do you all think it's possible to get the key form the NI-mate and then map video into that mask?

Would anyone be willing to share a patch so i can test this?

Currently i am having problems with NI-nate recognizing the kinect.

PatrickPagano

ok, got a new kinect device to at least show up with the NI-mate software.

Fifou, how did you route NI-Mate into Syphon?

PatrickPagano

Okay.

Got the Kinect working with NI-Mate

Ran NI-Mate into Syphon

Syphon into Isadora with Green Silhouette mode and keyed out the green,

Movie into the keyer

Voila. thanks everyone.

Is there not a CI/QC Chromakeyer?

cheers~

Skulpture

I see you have worked it out, great news. I made a little post about it here (for people reading this thread in the future)

http://www.ni-mate.com/forum/#/discussion/comment/98

You can convert the image and just a standard chromakeyer if needs be. Handy sometimes to add a bit of blur to smooth things off a little.

PatrickPagano

Hey Skulpture, i was wondering, Have you been able to track two people in the same kinect. I find that it NI-Mate turns the second person blue is there a way to set both people in the space to be key green? I know this is OT for Isadora but it's relevant to this thread at least.

Patrick

Skulpture

No I dont think there is way to make each person a different colour but that could be a very good feature request... I will ask the guys at Delicode if that can be added to the next release.

However you do get an OSC message for how many users are currently being tracked.

PatrickPagano

Yes, i can set up two chromas i guess that will work i assume. making everyone green would be ideal or just the ability to switch skeleton colors to key-able values

Skulpture

I've mentioned it on the NI Mate forum and had a reply;

It would supper cool if the ghost images for each user could be changed; rather than all people being green.

A small preferences window/pop-up would be ideal.

Person 1 = green

Person 2 = red

Person 3 = blue

Person 4 = yellow

Be great for use with isadora and colour tracking/keying.

Janne October 11

_skulpture: That's actually a pretty cool idea.. we'll have to see what we come up with for the next version :)_

deliriodelux

Hi

It is possible to send both ghost through Syphon (up to 6) , go to ni mate preferences and activate "all users" in ghost tab. We are using the torso value to track user position in space. There is also a problem with how ni mate assigns user ID to osc. If somebody disconnect and depending in how long reactivates, ni mate will assign a new ID. We are making an izzi patch to being able to assign user id depending in where they are in space and not in the sequence they are seen by ni mate. . If somebody need it, i can posted here,.

As far as know there is no CI or QC chromakey actor.

cheers

a

vrsck

hey DELIRIODELUX: post IZZY file patch, please!

:)

deliriodelux

Hey vrsck,

sorry for the delay, very busy week.

But, here it is.

We have divide the space (you can upgrade it up to 6) in four parts, and the patch will report and publish activity and values (not depending in user id) for 4 users. Attached is the patch and the OSC stream set up.

Hope it helps, any questions feel free to ask.

best

a

9f8714-archive.zip

zanetti

Hi

I am new to here and trying to do a similar installation as mentioned in the first place here.

Is there any tutorial for the IR alternative to get a mask?

My idea is very simple I just have basically two moving image layers and I want the people in front of the wall where this is projected to provoke a cutout to one of the layers so you can see the one below with their shapes.

So I think I wouldn't need the kinect solution don't I? Isn't the problem with the kinect that it really tracks people and cannot do more than 2? Or is the kinect also able to just send a plain mask with its IR feed?

best wishes and looking forward to your answers.

Leo

Skulpture

With a software called Ni-Mate you can track up to four people.

There are no specific tutorials as its such as specific area with many factors to consider.

Start by tracking one person and then build it up.

zanetti

Hi Skulpture,

I don't really need to track people. I just want to have their "shapes" as an alpha mask with which I would be able to process my effect then.

So just wanted to ask if that is possible in that way:

having a infrared camera recording the room of the installation from behind the viewers. Capturing an Livefeed with the shapes of the people in front of the projection and then using that as a mask in Isadora.

Thanks a lot

Skulpture

Hi,

I understand. NI-Mate still could be your friend here. It's track people, grabs their shape and you can then send it via Syphon (on Mac) into isadora - this is then your mask.

This is what i did here:

http://vjskulpture.wordpress.com/2013/04/02/isadora-live-feed-video-mask-how-to/

You can get similar results using the luminance key actor in Isadora but the problem is that once a person(s) move towards and away from the camera the luminance changes and it won't work - the person(s) can only move side to side really.

Keep at it - you will find the best solution I'm sure.