Help - motion tracking for moving surfaces

Bodo

@Fred - thank you for all those detailed informations! - to let you know: rotations are not realy nessesary there are just a 2d triangle, circle and rectangle - which will be moved by an actor mainly from one side of the stage to the other - maybe a little in the hight and depth - do you think this make it easyer and make itpossible to do it just with izzy???

Fred

It is up to you and your clients- how clean they want the tracking. In a theoretical scenario when everything else on the stage is very well lit so if the tracking is off it is burned out by lights and the actor never rotates the object at all - in any vector (keeps parallel to the projector at all times and does not rotate the shapes). Also depends on what you are projecting on them and how much you care if they are off. Marco Tempest stuff is super slick and does not allow for this kind of inaccuracy(well it does but a much lower margin of error than izzy can do.

If you can calibrate your projector and camera (a difficult task but that means you know where each pixel in the camera relates to each pixel in the projector this means compensating for lens distortion if you do it properly) then I would cover the boards in a material that is highly IR reflective, I would make sure that nothing else in the view of the camera is IR reflective and then you have a mask and a blob to track . It will be slow, there will be a lag when you move the item the video will follow several frames later. If that is ok then you are on the way and you can use Izzy kind of. Getting your board to be the only IR reflective source on the stage will be a difficult task and negotiation with costume and set. You would be surprised what reflects IR light and you need a laptop, IR light and small IR camera to test all the materials that will be on stage in that scene.

In the end it depends if you are willing to settle for a kind of OK solution, but the illusion works best by taking care of all the little details, this requires other hardware and software.

Bodo

thank you so much Fred for taking your time to give this detailed information - i will do a test with izzy and maybe we have to hire an expert...

clmuller

Hi Bobo,

I've a similar project, how is yours ?

Thanks for the feedback

Claudio

vanakaru

To keep different shapes IR pointers not mixed up you may want to use different IR wavelength LEDs - like 850nm; 750nm - with appropriate filters and separate cameras. The other possible help could be a GPS or even Wii attached to the shape. This would give you additional clue about specific objects location in space.

I have not done any of this - so it is just a thought.

Marci

OpenCV can handle 2D primitives tracking from any mjpg (jpg) src... there are Python examples out there: http://stackoverflow.com/questions/11424002/how-to-detect-simple-geometric-shapes-using-opencv Might be some sketches on the processing forums covering it using the Java OpenCV libs that could cast a mask into Izzy via syphon... will have a ratch n' see what I can find. As per @Fred: calibration of camera to Kinect to projector will be crucial for quality results, and will need recalibration for every rig. RGBD Toolkit / DepthKit provides this for Kinect & KinectONE respectively... see http://www.depthkit.tv/rgbdtoolkit/

Bodo

Hi clmuller,

because of budget we decided to map the stuff manually (i realised that there is a bunch off fast and expensive hardware nessasary and of course a ot of skills) to realised a good working trackingsystem - so no motiontracking right know - but would be happy to get some news if you find a good and inexpensiv way - we are going to start the project next week...

ciao Bodo

crystalhorizon

Mark mentioned in the Werkstatt Berlin at his IR tracking workshop that he would love something like eyesweb for Mac and that he probably will write it on his own. That would be really great.

fubbi

I did this quite a lot a few years ago and I used IR leds in the middle of the surface for the version where things stayed on their side of the stage (no risk for accidental swapping). And geometry was not important (no clean mapping, just roughing it in)

Having a good camera and well prepared LED was key. I had to upgrade from using the hacked PS3 camera from peau productions to using a CCD camera as recommended on frieder weiss homepage: http://frieder-weiss.de/eyecon/infrared.html, to get enough resolution for a smooth track and the width of a whole stage.

I also prepared the LED with some tubing to concentrate the emitted light, which stabilized the track a lot. I posted images of how that was made in [this post](http://forum.troikatronix.com/cgi-bin/forum/gforum.cgi?post=8697;search_string=fubbi%20IR%20led;guest=22417115&t=search_engine#8697) but they don't load for me, I hope they load for you.

I also tried a show where we had a processing application looking for corner leds and trapezing the image between the points, Now you can probably do the same by publishing inputs for your isadora map and tracking individual LED.

My two cents on keeping multiple LED identified would be to blink them in different frequencies and count the blinks. Either in isadora or processing

F

Skulpture

@fubbi RE: "My two cents on keeping multiple LED identified would be to blink them in different frequencies and count the blinks."

This is how the [blacktrax](http://blacktrax.cast-soft.com/) system work in conjunction with the d3 media servers.