Starting a recording on movement

Michel

Keep in mind, if you use the buffer method the buffer fills quite quick and the computer can get very slow and unresponsive. And as @dbini mentioned if the background is not pure black it won't take long until you have a white output.

Best Michel

Woland

@Axanias

Stayed up all night playing with a greenscreen solution, then scrapped most of it and threw this together in less than an hour.

Try this as a starting point: https://www.dropbox.com/s/3ugm...

(Sorry for the link rather than the upload. I think my landlord's wireless router may *actually* be a toaster oven.)

Explanation:

It uses a Video in Watcher to get a live feed of person "X"
The Patch has an Eyes++ actor, and when it can see someone in its view, it will register as tracking them, which:
- Turns on the Movie Player, so that person "X" can see and interact with the "ghosts" "trapped" in the installation.
- Starts a Capture Stage to Disk actor so that the composited live feed of person "X" and the played-back recording of persons "X-1 through X-n" is also being captured to disk for use when the next person enters the tracking area.
This live feed+prerecorded media combo is passed through a couple Effects Mixer actors to hopefully reduce background "noise" so to speak.
- Play around with the modes and the order in which the Effect Mixer actors and their video sources are hooked up, and also try adding in other video effect actors; different combinations will yield some fairly interesting results.
When person "X" leaves the tracking area, the new file replaces the old one in the Media List.
When person "X+1" enters the tracking area, it will trigger turn Movie Player and project the new composited video of persons "X through X-n" which will be combined with the live feed from person "X+1".
Note: you'll need to play with the detection/method of tracking a bit to get it to do exactly what you want.
Note: a Kinect would also be an effective method of eliminating background video "noise".

I have an example video uploading to Dropbox, and I'll link that once it's finished (if it ever does).

Best Wishes,

Woland

Woland

Oh, and it has an annotated Control Panel and the Patch itself is full of Comment actors explaining what's going on. There are definitely some tweaks you'll need to make in order to get it working for your purposes, but the Patch itself is just a proof of concept for what you were trying to do. Feel free to ask me anything if you have any questions. I could even screenshare with you via Skype if that'd be helpful for you.

If you haven't used Control Panels before:
Cmd+Shift+C shows you either the Control Panel or the Workspace
Cmd+Shift+S shows you a Control Panel/Workspace split.

Welcome to Isadora,

Woland

Woland

Here's that example video: https://www.dropbox.com/s/epoe...

Also when tracking starts, it fades in the most recently captured video over one second, then fades it out over one second when tracking stops.

In addition to this, the Movie Player actor is set to the "palindrome" mode so that it doesn't matter how long any particular person records for, as the video will just play forwards, then backwards, then forwards, continuing to fill the time until the participant leaves the tracking area. This gets around the problem @dbini mentioned.

Best wishes,

Woland

Woland

Also, if this is an installation you're going to leave up, you should take a look at this thread which has a script that auto-reopens Isadora if it crashes: https://community.troikatronix...

Thanks to @Michel we now have shiny new version of the script to play with.

bonemap

@Woland said:

Try this as a starting point

Wow! And you do this stuff for free?

I was going to look for a patch I created for a student many years ago that did this exact brief (I remember it was developed on a Mac G4). That production used the idea of a 'photo booth'. The booth provided a way the background behind the user was defined by using a chroma key curtain. The user enters, a camera tracking system detects the user and triggers the capture system, a series of prerecorded audio files instructed the user how to act - specifically, part of a narrative statement to distract the user and then a statement to suggest the interaction had ended and it was time for the next user to enter - I can't remember the exact script. When the next user entered, the previous users video file with the narrative fragment was played back and the capture triggered again when the narrative script incremented.

best wishes

bonemap

Woland

@bonemap said:

@Woland said:
Try this as a starting point
Wow! And you do this stuff for free?

I mean, in the real world I generally ask for monetary compensation for my labor, but when I run across ideas on the forum that pique my interest, I build them, then once I have a prototype, I feel that there's no point in not sharing it with folks. Any of you can feel free to hire me if you like though ;)

Best Wishes,

Woland

bonemap

@Woland said:

Any of you can feel free to hire me if you like though

Yes! I think we can start a conversation around that idea

cheers

bonemap

Woland

@bonemap

Send me a DM anytime my friend. It would be a privilege and a honor to work with or for you.

Also that photo booth production sounds very interesting. I’ve been itching to do more installation-type work literally since I started using Isadora.

Best wishes,

Woland

Woland

@Axanias

Please let me know if this works out for you or if you'd like some help refining it further. It's always nice to know whether or not what I've given to folks is a viable solution for their specific usage case.

Best wishes,

Woland

Axanias

@Woland

Wow thanks man!! Gonna try it today and see if it works, have been sick all week with a nasty flu. I'll send you an update later today!

Axanias

@Woland

Wow this is amazing! Thanks a lot for helping me!! It works great in your video, but i can't seem to make it work on my macbook. It keeps giving me a 2048 error. It also seems that the recording isn't continues but starts and stops the moment i move to a certain point, even if i'm still in frame. This is probably done through the crop node? But i don't really get how i controle this :D I've tried getting a clearer image with less discolouration, but once i do that it won't start the recording either.

Is it possible to send the file without all the non-necessary effects on it? Sorry if i'm abusing the fact that you are doing this but i can't save my files and it keeps crashing on me...

Woland

@Axanias

I’m off to an opera at the Met this morning, but I’ll get right on that once I get back to my house, so I’ll get you something by this afternoon.

Best wishes,

Woland

Woland

@Axanias

I'm not sure what the 2048 error you mentioned means. Perhaps make a support ticket to get that sorted out.

I removed the Motion Blur, but the Threshold actor and the two Effect Mixer actors were pretty much what was making my Patch work. They're necessary in order to remove the background (and the Gaussian Blur actor just makes it look *so* much smoother).

Patch: Axanias Looping Live Capture Installation 03.izz

If you want to use the cropping to limit the tracking area to a specific bit of what the camera is actually seeing,make the Control Panel visible. That's where the controls for the cropping I build in are.
Cmd+Shift+C shows you either the Control Panel or the Workspace
Cmd+Shift+S shows you a Control Panel/Workspace split.

The reason that recording is starting and stopping on you is because the Threshold actor was calibrated to work for me, in a darkish room with my laptop as the only light source, at whatever time of day it was when I recorded that video. I used the Threshold actor to have Isadora ignore everything that wasn't brighter than "x", thereby removing the background and only seeing/tracking my hand when it got close enough to my computer, and therefore bright enough to be seen. I'd advise playing with the threshold value, or trying a different tracking method all together. That was just the easiest way to show you the proof of concept with a setup in which it was easy for me to "enter" and "exit" the tracking area while not actually having to leave my laptop camera's field of view. It also had the added bonus of solving the problem of the background becoming too bright after multiple recordings.

Ultimately, the method of tracking is up to you. You could use the Freeze actor + Effects Mixer actor set to "difference" method (thanks to @Skuplture for the tutorial), or a Kinect and infrared tracking setup, both of which should make it easy for you to remove the background. Another tracking method would be Chromakey and color tracking (again, thanks to @Skulpture for this tutorial), but then you'd have to have folks using the installation wear or hold something a specific color in order to get tracking to register, which might turn out to be rather impractical. Each presents its individual challenges, but what you have in the Patch I sent you is the skeleton of what you'll need for your installation; starting live capture and recording when tracking starts, and automatically compositing the live video with the previously recorded video once tracking is finished.

Best Wishes,

Woland

Axanias Looping Live Capture Installation 03.izz

Axanias

@Woland

I found the 2048 error it has something to do with not being able to read the .mov file. I played around with the threshold settings and it seems i can make it work now. The program has a tendency to crash but i think the link you gave me should help with that!

Thanks a lot for helping me! I don't think i would have ever found this solution by myself!!

Woland

@Axanias

Again, my pleasure. Don't forget, as a result of doing this for you, I have a working skeleton to make an installation like this myself now too. Moreover, now anyone who searches the forum can do an installation like this as well. Thanks for letting me build an interesting idea.

Not sure why it's crashing on you though. Hasn't been doing that for me.

Best wishes,

Woland (Lucas Wilson-Spiro)

Woland

@axanias

How goes it? Would love it if you'd keep all of us on the forum up to date on your thesis project as it develops.

Best wishes,

Woland