Motion Tracking: so far and beyond

[marcusbastos]

Hi,

I have been using Isadora for my Multimedia / Intermedia Studio class, for the Visual Arts Undergrad at the University of São Paulo. I consider Isadora the most suitable tool to introducing students for interactive and projection related projects, given its optimum learning ramp: while powerful, the fact that students do not have to learn lines of code, etc, makes it less intimidating _ specially because a good number of the attendants are more focused on broader aspects of visual arts, than programming, etc. That said to state that both me and the students are aware that the projects being developed are meant to help then to learn and think through processes that probably have alternative solutions that might be more adequate at this stage of technological development. For example, we are aware that probably using a Kinetic instead of the Webcam, might improve the capacities of Motion Tracking. Nevertheless, with existing resources, and for the sake of introducing the process to a group that did not have previous experience, this semester one of the projects was an installation where a camera positioned on the top of a room "sees" a person walking on a specific area, and moves a short animation, to always place it above its body. We are finishing a prototype, but reached a point where it has been hard to fine tune the tracking. If anyone has suggestions, and ideas on how to make it more precise, they will be welcome.

1e50e1-motiontracking.izz

[Michel]

@marcusbastos

You have to tell us where exactly the problem is. If I don't have your live video input from your setup I do not know what to fine tune. One thing sure is important, the camera has to be set to manual mode, otherwise every time when someone enters the area the camera adjusts its settings. And then the tracking can't really be precise.

Maybe putting the difference actor just before the Eye's actor can help, depending what look of the moving picture you want to achieve.

Best Michel

[marcusbastos]

Michel,

actually, with few adjustments by the end of the morning things worked reasonably well (with the updated file posted on this second message). 


 But the main problems we are still facing is:

_ tracked video does not move as fast as wished

_ tracked video's movement oscillates (when supposed to be fixed on a position, it never really stops completely)

As for the difference actor, actually he already was before the Eyes. It is embedded in the FREEZE + DIFF User Actor, below there is a screen shot of its content.


best

Marcus

f319fc-motiontracking.izz

[jhoepffner]

Hello,

The first step would be to use gpu instead of cpu, eliminating classic projectors and so on…

Second is to prepare the image before eyes, chroma key, image freeze + difference etc.

[Michel]

@marcusbastos

As @jhoepffner said, if you have a Isadora 2.x license change everything you can to gpu actors, the cpu will slow down everything way to much.

To minimize the oscillating of the tracked video's movement you can place a smoother actor on the output side of the eyes actor.

I am talking about the actor called "difference" you are not using this actor, maybe using it you can replace your user actor called FREEZE + DIFF.

Best Michel

[marcusbastos]

Hi @jhoepffner and Michel,

thanks for the comments and suggestions. I have just Upgrade for version 2.0 and will update the patch eliminating CPU video as much as possible.

Michel,

ref the user actor x the regular difference actor, I tried a bunch of approaches on the process (including using the Chroma Key to smooth things out) and this seemed to be the more stable. Nevertheless, I figure with version 2.0 and less CPU video onboard your suggestions will certainly help.

best

Marcus

[Michel]

@marcusbastos

if you put a contrast adjust actor after the difference actor and set its in max to 25 you get a more clear output.

Best Michel

[Skulpture]

You should find the upgrade to v2 a lot better performance wise.

I also have lots of motion tracking tutorials on my blog:

https://vjskulpture.wordpress.com/category/motion-tracking/ 

[dbini]

@marcusbastos have you looked at TSPS? (http://www.tsps.cc/)

It could be a useful teaching tool on its own, or for separating out the person and syphoning it or sending OSC into Izzy.

Also Eyes and Eyes++ both have smoothing built in. This will help eliminate any jitter.

[marcusbastos]

Hi @Skulpture, thanks for the link. I have gone through some of those to develop my patch, but will look all of then with due care. Thanks @dbini for the link, this seems very nice indeed ;)

[marcusbastos]

Here is a new patch, with upgraded actors.0665ad-motiontracking.izz