Webcam Motion Capture

bonemap

Hi,

I came across this implementation of webcam motion tracking that uses OSC output and is therefore compatible with Isadora:

It uses a software implementation of MediaPipe, a machine-learning tool that tracks the body, hands and faces. In the video, I am tracking just one hand point (wrist) via OSC using this software download.

It is a virtual youtuber avatar controller but outputs accessible OSC motion tracking points.

This may be of interest to some who want an accessible and free option to try performer motion tracking using just a webcam.

Best Wishes

Russell

RIL

@bonemap said:

software download

I'm super interested!
I could open the Receiver but not the Capture...And I don't know how to continue

Best,

RIL

bonemap

@ril

Hi,

It possibly requires an ARM-based mac. You could try email the developer to ask about compatibility.

Best Wishes,

Russell

Fred

@bonemap you can also check this out:

https://github.com/LingDong-/PoseOSC

It uses the posenet model to do skeleton tracking from a webcam stream in real time, and sends it via osc. Posenet is pretty light weight, not much worse than openNI.

Fred

bonemap

@fred

Thanks, @fred. I have an implementation of MediaPipe constructed using the Isadora Pythoner beta module. However, as the version of Isadora with Pythoner has yet to be released, these options provide an alternative. I will have a look at PoseOSC, too.

Best Wishes

Russell

joe_pavey123

Hi,

This is a super helpful implementation.

I am wishing to track some of the main body points (head, arms, legs) but am struggling given the osc data sent has a couple thousand unique channels. You say you are just tracking one hand point, but how have you found which of the points is the target? Simple trial and error?

Thanks,

Joe

Woland

@joe_pavey123 said:

how have you found which of the points is the target?

I haven't looked at it, but I'd expect the OSC addresses themselves to be somewhat human-readable for this exact reason, something like this:

/hand/left/finger/ring/x	/foot/left/x	/torso/x
/hand/left/finger/ring/y	/foot/left/y	/torso/y
/hand/left/finger/ring/z	/foot/left/z	/torso/z

If their OSC addresses are not human-readable, then the documentation for the project must have some sort of key, legend, or other explanation for the format of their addresses so that you can tell which is which.

mark_m

@joe_pavey123

Are you referring to PoseOSC? If so all the info you need is on the Github page, including a link to the keypoint diagram.

There seem to be only 38 (max) tracking points.
Each seems to have a x,y, and 'score' value, so I think this is only 3*38 OSC values isn't it? Not thousands

HTH

Mark (not that Mark)