Interactive projections through kinect?

MatthewH

I don't have a video to show you but I have done this with big moving scenery "tagged" with IR LEDs on the corners and tracked with an PS3 Camera with the IR mod and an aftermarket lens. Use the 3D Quad Distort actor and some clever math to make it work

shoebeedmantle

Awesome! I'll find some of these IR LEDs online and pin them up. I'm sure I'll be back asking advice when it doesn't go quite according to plan. Thanks for the leads though, all help is greatly appreciated.

Skulpture

Yeah you don't really need a kinect to be honest. Even a basic camera with colour tracking of 4 points could work BUT for the 'illusion' IR is used; as quite rightly described by Dbini and the Marco Tempest examples.

A standard CCTV camera or a hacked PS3 with IR mod trick will work for sure.

Michel

@shoebeedmantle

The actor that will help you turn the image would be the "calc rectangle" actor. For each point eyes++ is seeing you forward the position into the x and y inputs.

Best
Michel

shoebeedmantle

cool, just ordered the ps3 eye yesterday so that should come in, can't wait to hack it and try it out. I'm interested to see the speed at which the tracking can handle and those limitations, but it should still give an interesting effect. I'm going to start researching the LED hook up now, I'm sure there's plenty on izzy forum to read on.

mark

Also, just to chime in here, the Wii Controller has an IR camera that will track four infra-red light sources. OSCulator will pass the data into Isadora via OSC. Michel attempted to use this system for a show, maybe he can give a short report about the Wii and how it worked for him in this situation.

The actor Michel mentions (Calc Rectangle) was created for him when he ran into trouble with that show. The Calc Rectangle is important because it keeps the "sort order" of the four points consistent, so that you can feed them into the 3D Quad Distort actor and it will also affect the right corner.

Best Wishes, Mark

Michel

Yes, as mark said we where using a Wiimote for this, at the end it didn't make it into the show because next to all other things we where doing the "real" actors never had time to play around with it to get comfortable on how they had to handle the projection surface. As you can see in Marco Tempest's video, fast movements let the projection always be a bit behind. The distance the wiimote worked ok was up to 8 meters, but depends extremly on the brightness of the LED's. For expanding the up - down and left - right tilt range we used 3 IR LED's on every corner tilted a bit away from each other, but it depends on what angle the LED's will emit, the larger the angle the less bright they are.

Best
Michel

fubbi

Ir leds is definitely a way to go

Instead of tilt range you might modify the leds by wrapping them

check out [this thread](http://forum.troikatronix.com/cgi-bin/forum/gforum.cgi?post=8669;search_string=smoother%20tracking;guest=21876349&t=search_engine#8669)

If the frame can stay in a certain range with little or nothing else on the same distance line line you could try trapping (tracking and mapping) it with a kinect.

I wrote a processing app for this purpose but it involves using syphon and processing so let me know if you are into it.

yours

fubbi

quarterinchjack

This thread has really helped me out - thanks all! Just one thing - I now have four LEDs ona board being tracked through the wiimote camera, in turn feeding the Calc Rectangle actor and the quad distort actor. The rectangle moves and rotates with the movement of the LEDs, but because the position of the LEDs are constantly recalculated, the video image mapped onto the rectangle constantly maintains its aspect. It rotates so far and then snaps back to its original vertical state.

Am I missing something simple here?

Best,

Andy

Fred

If you are on OSX you can also try AR toolkit, there is a version for quartz. You have to mark your target with a fiducial (can be done with IR reflective material (you can train your own fiducials). The advantage of this is that is gives a homography matrix back that allows you to easily get a 3D rotation of the target on which to re-project. I have used this a few times and it works well.