Dining Experience using Isadora - Advice Please

[JamesRea]

Hi! I am new to projection mapping and trying to teach myself the basics.  I am an event designer and ultimately would like to create a dinner for a number of guests where projection mapping plays a part. I was inspired by the Teamlab installation at the Maison & Objet show in paris where you could move a cup around and when it remained still a flower bloomed inside it.  If you moved the cup the flower fell apart and then bloomed again where the cup was repositioned to. 

I am looking to invest in software and further equipment and wondered if anyone could please point me in the right direction?! I think Isadora is the product I need but will I need additional hardware etc. to achieve the kind of work I have been inspired by?  

For my scenario I would like to place plates down onto a table and then the projections would appear on them.

Thank you in advance for any help and advice. I really appreciate it. 

[DusX]

@jamesrea

Isadora is a great tool for live interactive projections.
Included in Isadora is a powerful Video mapper (izzy map) which allows to to control mapping interactively.
As well as motion tracking tools like 'Eyes' which provides computer vision blob tracking.

To accomplish what you want, you need to have some way of having the Computer 'see' the dishes.
Imagine a scenario where the dishes are white and the table cloth is black (in a normally illuminated room).
If you place a digital camera directly above the table looking down, so that only the table is in frame, it would be pretty easy to have the computer provide coordinates for the dishes.
In fact that is very much what the built in 'Eyes' actors do.

Now this does get more complicated in a dark (poorly illuminated) room. In this case you need to find ways to enhance the image seen by the digital camera.
One common way to do this is to use IR light.
Perhaps you flood the area with IR light (invisible to the guests), and ensure your plates reflect IR, and your table cloth absorbs it.
Now as long as your Digital camera is IR sensitive (usually Black and White cameras work well) you should get a high contrast image much the same as in the illuminated room that can be used for tracking in the exact same manner within Isadora.
You will have easy access to the coordinates of the plate (blob), and can track things like the distance between then, velocity when moved and much more.

That's sort of the basics of getting something like this up and running.

[Woland]

Idea #1) that comes to mind is a projector pointed straight down, each plate having a little IR LED, then having a IR camera in the ceiling also pointed down. Do the math to rectify the difference in position of the camera lens and the projector lens and then wherever the LED moves, you can tell the Isadora to send an image to that location.

Idea #2) Similar to #1, but make the projector go through the non-mirrored side of a 2-way mirror at a 45 degree angle and place the camera so that it sees the ground reflected from the 45 degree angled mirrored side. This has the effect of simplifying the math involved in rectifying the difference between the projector lens and the camera lens since it's the closest you can come to giving the two lenses the same exact position in space.

Idea #3) No special plates, but special tables covered in pressure sensors, then use math to align the projector with the section of the pressure sensor array being triggered.

Idea #4) Also special tables, but embedding magnets in the bottoms of special plates plates and using those to track the plate position.

Idea #5) Color tracking the plates with an overhead camera (but that's light-level dependent, is tricky because you might get false positives from people wearing clothing the same color, and the plates might not register when they're covered in food.

I'll play around in my free time tonight or tomorrow and make an example file for you of how to handle the logic and basic tracking, though you'll need to rework it with your own media, your own scaling, etc, it should help demonstrate the relevant concepts for you.

Best wishes,

Woland

[fubbi]

Maybe investigate using depth cameras from above. There are plenty of options these days and isadora is a good place to interface with them. It could be possible to filter out shapes like plates or glasses and also catch points of interaction. IR is very powerful but you have to control the environment 100% as almost all light, especially the sun, has tons of IR in it.

[dbini]

something like this? https://www.youtube.com/watch?v=yBJEP4lsRFY although, in this project, the mapping is all predetermined rather than tracking.

[Woland]

@jamesrea said:

For my scenario I would like to place plates down onto a table and then the projections would appear on them.

 I finally found the time to do that mock-up

2019 - Plates.zip