Body tracking with Yolo

jfg

Re: [Pythoner Plugin Beta](/topic/8082/pythoner-plugin-beta)

"Anyway, I want to refine this patch before I give a list of instructions to let you all try it, but it was promising in terms of future solutions with body tracking."

Dear Mark,

Have you achieved anything in this matter? Can we still hope for instructions or an actor? I am afraid I can't do it on my own

thanks a lot

Jean-François

DusX

@jfg

The short answer is yes, there is something coming that will be released. It has been held up due to testing. Trying to make the installation manageable for novice users isn't a trivial endeavor due to the GPU specific features used.

I will see if I can get an idea of what timeline we are looking at, and report back.

n.jones

@dusx I I've been using this for a while , using TD to send osc to izzy, probably a really unproductive and process intensive but it seems to work ? https://www.youtube.com/watch?v=Cx4Ellaj6kk

if there was some way of doing this in izzy that would be amazing !

DusX

@n-jones said:

if there was some way of doing this in izzy that would be amazing !

I is pretty easy to use MediaPipe with Pythoner. Which feature of MediaPipe do you use most... face, pose etc?
I can make an example file for this pretty quick. (YOLO runs faster [when GPU enhanced] and has better Pose tracking, that is why YOLO tracking has been worked on :)

n.jones

@dusx wow , that would awesome 🙌.
For me the things I find most impressive is the fact it’s runs locally without a need to install python and the ability to select or switch the models on and off 🤯 .
I’d love to keep my workflow in Izzy so any help in this would be amazing 🙏.

jfg

@dusx

Please :-)

Jean-François

DusX

@n-jones said:

without a need to install python

Well, this code will require some configuration of Pythoner.

By default Pythoner offers the full standard library included with Python, however, this doesn't include everything we need for this script.
So, a new Virtual Environment will need to be created, and then MediaPipe, OpenCV, and Numpy will need to be installed.
The module names for install are:

opencv-python
mediapipe
numpy

This is worth knowing, because the names for installation don't always match the names used for import into the python code. eg: opencv-python uses, import cv2, in the python code.
You can find available modules at: https://pypi.org/

If you haven't I recommend watching: Getting Started With Pythoner to get an understanding of how the configuration is handled. It really should only take 3-4 minutes to create the environment and install the modules needed, once you know how it works.

I will create a quick and dirty video tutorial to walk thru this for this specific script/project. ASAP

jfg

@dusx

1000 thanks

Best regards,

Jean-François

n.jones

@dusx thank you