Blob Tracking, Projection on Balloons
-
Hi ,
Juddy from Melbourne here.I am looking at projecting onto slowly moving objects such as balloons. I would like to be able to place video on a balloon in darkness and run the audio as well. It would be a bonus if I could project separate images onto each slowly moving balloon.Can anyone suggest a workflow for this? I have a Kinect 360 (do I need a standalone Kinect?) or should I use a IR camera and light setup? Mac, projector, many midi controllersA decent field ie 4 metres or so from the beamer would be good.I would also like to be able to run audio reactive plugs from QC or the like.Thanks for your thoughts and timeJuddy -
I personally feel that adding a IR led to each balloon and using an IR camera would likely be best.
You might be able to put the Lights in the balloons, but you would need to test if the camera still sees them.
You might be able to do this with a connect, but blob tracking a number of items will likely become a problem. -
Before you start with hardware it is best to have a look at your physical setup. For any tracking you need to have good contrast between what you are tracking and the background. You can use lighting even a strong backlight (like a giant lightbox) to do this. If you use a kinect you can use depth as the contrast but you will have a limited range and pretty low resolution.
Once you have a setup with sufficient contrast you should be able to get a clear image of the balloons for tracking. You will have to deal with blob identification next. There are some controls to ensure that when 2 blobs cross paths they keep their identity in the system but several twisting balloons will make this difficult as the system does not see them as truly unique blobs and can swap their id numbers easily.Next you will have to deal with the lense extrinsics and intrinsics from the camera and projector. There is distortion and different depths of field on the lense of the camera and projector. You can try and get around this by using a quadwarp on your output and trying to match the image on the balloons but there is really more going on than this. Because the lenses are different, as the balloons move back and forward in depth in the space the corresponding size changes from the camera vs projector will not match up. It will be possible to get ok registration on up down and left right movements but toward and away from the camera and projector will be difficult to do without calibrating for the lenses of the camera and projector (Isadora cannot do this at present).Depending on the lense and distance of your projector you may also have a small depth of field on the projector itself, meaning that it may be in focus at one distance but as the balloons move back and forth the video on them may be out of focus.Projecting on moving objects is just tricky, you have to be very fast to get it right and follow the movements, as Isadora executes each actor in a linear fashion you have to wait for the tracking to be done, then process the videos for projection after. If these tasks can be performed in parallel you get a lot closer to following motion.If you have some freedom in designing the project start to try and remove any situation that makes the tracking harder (use a backlight - this can even be IR so you don't see the light), limit the amount of movement of the balloons in every direction etc. Try get good tracking and then if you have to, start adding these difficult conditions and see if it still works. It is not that easy to project on moving objects but best of luck.Fred -
If each ballon was a different colour you could maybe colour track them?
[http://vjskulpture.wordpress.com/2012/10/22/isadora-and-colour-tracking/](http://vjskulpture.wordpress.com/2012/10/22/isadora-and-colour-tracking/)
