How to keep blobs apart in Eyes++ ?

LTony

Hi,

I'm new here. Been working with Isadora (2) for a couple of weeks.

I'm using Isadora to track 2 persons on a stage, but with an overhead camera setup. As it is impossible to use the 'subtraction' method (live-picture=blob) in our environment, I decided to use InfraRed LED's on one hand of each person on stage. 2 persons, so 2 blobs. I setup the Eyes++ actor accordingly. We get 2 nice blobs that track perfectly, except:...

I can understand that when 2 blobs 'intersect' that Isadora doesn't know anymore which was blob1 or blob2 and might switch them. So we made sure in the performance that the 2 persons never let their hands intersect. What I didn't know is that intersect means either vertically, horizontally or both! When the 2 persons (blobs) are apart, even more than a few meters distance between the persons on stage, the blobs will still change identity (either nr.1 or nr. 2). This happens all the time when the blobs show up in Isadora's Eyes++ actor at either the same X-coordinate (and different Y coordinates) or at the same Y-coordinate (but different X coordinates). The Eyes++ actor will switch vigorously between their ID's when either situation occurs and of course also when the blobs are actually brought to the same location (X and Y).

Is their anything I can do about this? I've tried 'same max dist' and 'same area chg' and 'lifespan', but none of them solve the problem.

Thanks for any suggestions on how to fix this.

Woland

@Skulpture ?

Juriaan

So this is a common error with Motion Tracking, basically what you have to do is have more data to keep the dancers apart. You can do this on multiple ways :

- Give them a proper ID, in your situation you can do this by using Blinking IR leds that are blinking in a different frequency from each other. That way you can find out what the ID of the performer is.

With the more expensive motion tracking systems (Blacktracks, Optitrack) you use a system where each single tracker is unique with a multi camera system that tells you exactly where your performer is in the 3D space.

Michel

@ltony

Can you send a screenshot of your eyes++ settings?

Best Michel

bonemap

@ltony

Hi,

I share your pain with this issue! I have a blob tracking project that I am working on that is giving me the same sort of grief... following is a screen capture of the settings I am currently trying in Eyes++ module tracking two blob objects and another tracking 16. It would be good to get a response from Isadora about the 'lifespan' property and if the module is actually functioning correctly.

http://recordit.co/YjSpc9a4lR

http://recordit.co/y9jocqiXRr

best wishes

Russell

DusX

@bonemap

1 solution for your setup, since your blobs have unique colors it to use Chromo Key to separate each color into a separate video stream (or if you are generating these blobs, don't mix them).

Then use a Blobs actor for each of the video streams. This will track each completely independently, however; it will also increase your CPU usage, so I recommend using the lowest reasonable video resolution for the blob tracking.

dbini

its possible to get IR LEDs that transmit different frequencies, and IR filters that match these frequencies, so with 2 cameras, each of your performers could have a dedicated video feed and no mixups. (needs more equipment and twice as much work to calibrate the cameras though....)

bonemap

@dbini said:

IR LEDs that transmit different frequencies, and IR filters that match these frequencies, so with 2 cameras,

Hi John,

Do you have any idea of the cost of a single performer Black Trax system? If Black Trax is proposed to be integrated into Isadora it would good to know what the costs are for one of their simple systems.

bonemap

@dusx said:

since you blobs have unique colors it to use Chromo Key

Hi Ryan,

I like where you are going with that, but I will be using a thermal camera so that there is no conflict with stage lighting. The FLIR camera I have doesn't see color at all only heat (hopefully only body heat).

The Eyes++ calibration is very tricky... one thing that is a bit baffling is that you have a property called 'lifespan' but when it is incremented higher towards 100% the tracking appears to be unpredictable or stops functioning all together, where you would think it would become more persistent. It appears to require very nuanced parameter settings and that is hard to maintain in changing performance environments.

best wishes

Russell

dbini

@bonemap

i don't know about Black Trax, I would probably try to build my own system out of bits and pieces salvaged from TV remotes and off of Ebay. If Black Trax gets integrated into Izzy then that's another level up, it would be worth investigating. I think they make custom systems depending how many points you want to track.

bonemap

@dbini said:

I would probably try to build my own system out of bits and pieces salvaged from TV remotes and off of Ebay

If only I could be so clever, I think I am too dyslexic to do anything like that.

LTony

@bonemap

You are right about the Eyes++ calibration being tricky. The lifespan parameter seems to have a downside to it that it takes longer to detect the blob in the first place and then it looses the blob faster when it moves. So for the moment I have this set very low (0-5). 'Same max dist' and 'same area chg' seem to do the trick in my case, at least they improve the response, but setting them to high will worsen the switchovers between blob ID's and it is almost impossible to find the sweet spot where response is the best.

Still experimenting at this, but it's improving...

@Juriaan

I have thought about blinking led's, but I wouldn't know how to go about getting Isadora to see the difference. So I guess that would take a separate system all together to make that work. As far as Blacktracks, optitrack is concerned: those are very expensive setups, even to rent, and even then would be difficult to implement for what I'm trying to do here...

Maybe I should mention my human blobs are about 6 m off the ground and constantly moving both in height and in horizontal/vertical axis seen by the camera :-)

Skulpture

@bonemap said:

It would be good to get a response from Isadora about the 'lifespan' property and if the module is actually functioning correctly.

Lifespan as I understand it works when an objects disappears from view but Isadora tries to semi-predict the movement so it can pick it back up a few moments later. SO if a person moves behind a pillar or post for example.

Skulpture

Ah, Yes this old chestnut.

Motion tracking more than one person is very tricky. As amazing as eyes and eyes ++ is to get very good results I have often had to change the physical movement (dancer, actor, etc) which is not always ideal. Isadora does a very good job for its price point and indeed Blacktrax is another level; price and functionality!

bonemap

@skulpture said:

Blacktrax is another level; price and functionality

There is no mention of price points on their website, do you have any idea of a ball-park cost for any of their systems?

Skulpture

To buy outright you are looking at around $60,000 I'd say. Luckily I have a student (soon to be graduate) who owns a system and their office is onsite.

bonemap

@skulpture said:

around $60,000

Well the system tracks in 3D with six degrees of freedom... That's a lot more sophisticated than 2d XY tracking of Eyes & Blobs. I wonder if there is a less expensive entry point after all it is not full motion skeleton tracking.

bonemap

@skulpture said:

outright you are looking at around $60,000

I put an enquiry through to Black Trax about a system to track a dancer in one of my scrim projection rigs (4 x 4 x 6m) the estimate came in at US$72,000. That would take some serious fund raising to achieve. Some alternatives were offered: a preloved system or a system from a rental partner, but I would have to dig a bit deeper to get a price on those options.

@dbini John's suggestion of making his own system sounds more feasible at that price tag.

best wishes