[ANSWERED] Alternative to Ni-Mate?

jfg

https://blog.tensorflow.org/20...

best

Jean-François

Juriaan

TensorFlow is a really great way to extend Isadora, we should write a KB article describing how to do that properly. Personally I'm more a a fan of the TensorFlow model that can be found here:

https://github.com/tensorflow/...

The package has 3 different kind of flavors;

MoveNet

17 keypoints, up to 50 FPS on modern machines.

BlazePose

MediaPipe BlazePose can detect 33 keypoints, in addition to the 17 COCO keypoints,
it provides additional keypoints for face, hands and feet.

PoseNet

PoseNet can detect multiple poses, each pose contains 17 keypoints.

Let me see or I can spend some time tinkering with the package and publish some documentation about this on the forum. OSC should also be do-able!

jfg

@juriaan said:

Let me see or I can spend some time tinkering with the package and publish some documentation about this on the forum. OSC should also be do-able!

It would be wonderful. Thanks a lot in advance

best

Jean-François

jandraka

@jfg

Hi,

Based on PoseNet is this project: PoseOSC

Best

DusX

Take a look at our latest tutorial: https://support.troikatronix.c...

Where you can train a Tensorflow model, and interface it in Isadora via OSC.

I found a Pose model (Keras format) at: https://www.dropbox.com/s/llpx... from: https://github.com/michalfaber...
which I have not yet tested, but it should be possible to use.

jfg

@dusx

Thanks for your tipps but it doesn't help me because the explanation are only for Windows. If I download NDI-Classify for mac from the link in this tutorial I get an exec file and nothing else. I don't know what I have to do with this file.

At least my Idea with this topic was to ask if there is a possibility to get poseNet as an actor inside Isadora.

Thanks

best

Jean-François

liminal_andy

@jfg To run the program on macOS, right click on the Exec and hit Open, then, when prompted, again hit Open.

jfg

@liminal_andy When I open the program with right click, a terminal window opens and after some time the following appears but I can't find anything where I can click again to open.

terminal.rtf.zip

best

Jean-François

liminal_andy

@jfg and you are running this on the Mojave computer? Do you have access to a machine with Big Sur or later? Some of these machine learning frameworks are bleeding edge and may not exist in older macOS versions.

jfg

@liminal_andy

unfortunately not. I have only Mojave or earlier on my computers.

I will try to get a newer machine.

Thanks for your help.

best

Jean-François

mark

@jfg @DusX @Woland @liminal_andy

So, I wanted to try the PoseOSC to see what it could do. This required manually configuring the camera because I have several camera drivers installed and it was picking the wrong one. Here's how to fix that on macOS:

Choose "About this Mac" and click "System Report"
Go to the sections labeled "Cameras". Find your camera and look for the item labeled "Unique ID". On my computer, for the internal camera, it was something like 0x8033000455aca514.
Right click the PoseOSC app and go into Contents > Resources > app. There you will find the file settings.json
At the bottom you'll find the "cameraConfig" item. You will want to change it as shown in the code snippet below. You need to replace device_id_found_in_system_report with the ID you found in step 2 above.
Reboot PoseOSC and you should now get the camera you want.

CODE SNIPPET

  "cameraConfig":{
    "deviceId":"device_id_found_in_system_report",
    "width": 640,
    "height":480
  }

That said, my quick trial showed that the response was quite jittery. But also I was not in an optimum environment. If others give this a try, I'd be curious to hear how it worked for you.

Best Wishes,
Mark