Open source software for multiple bodies and streams OSC

[Armando]

Hello all, I just found a tool I think you'll love that was given me by the author. Colin Clark, an incredible person, a composer and software developer (and much more !) based in Toronto. He created a software called Movement OSC, (mac PC and linux) that does pose estimation and sends osc. Movement OSC detects body keypoints using Tensorflow.js' MoveNet models and sends data over OSC. It can also compare Movenet and MediaPipe so you can compare them ! The software formats the osc messages 4 ways Bundeled messages per axis, messages per axis, and bundled xyz array. I use just the latter then parse it in the Json parser 

Here are the keypoints and the keypoints diagram 

Enjoy !

MovementOSC provides a normalized keypoint format that is consistent across all its supported models:

• 33 keypoints in total
• the first 16 keypoints are the COCO set
• the remaining 17 keypoints are those unique to BlazePose, in order from head to toe
• regardless of which model you use, all 33 keypoints will be provided; keypoints that aren't supported by the model, or which don't meet the specified minimum score, will be set to NaN

Keypoints

0: nose
1: left_eye
2: right_eye
3: left_ear
4: right_ear
5: left_shoulder
6: right_shoulder
7: left_elbow
8: right_elbow
9: left_wrist
10: right_wrist
11: left_hip
12: right_hip
13: left_knee
14: right_knee
15: left_ankle
16: right_ankle
17: left_eye_inner
18: left_eye_outer
19: right_eye_inner
20: right_eye_outer
21: mouth_left
22: mouth_right
23: left_pinky
24: right_pinky
25: left_index
26: right_index
27: left_thumb
28: right_thumb
29: left_heel
30: right_heel
31: left_foot_index
32: right_foot_index

[mark_m]

@armando

Hey Armando,

Thanks for sharing this.Seems to work very well, but am having a little issue getting data into Isadora.Do you have an example patch using Movement OSC that you could share, please?
Thanks
Mark (not that Mark...)

[dbini]

@mark_m

Hi Mark, I downloaded Movement OSC and it runs smoothly on my Intel Mac. OSC out to Isadora works for me in bundled xyz mode, with a change of port number. I'm teaching a session with some robotics students today, so looking forward to testing it out with them. Thanks @Armando - this is a great resource.

[jfg]

@armando

A thousand thanks. It works fantastic. Do you have an idea how to change the standard port and the format ? By every start it go back to 7500 and bundled Message per Axis and I cannot found how to change it.

Again a thousand thanks for this found.

Best regards, 

Jean-François 

[Armando]

@mark_m 

Here is a test patch. It works In fact the software is even more interesting than what I see, apparently. It is a framework for testing different models that can live parallel lifestyles and be alternated and studied. Please enable from the menu the developer panel Astonishing. 

movement-osc-test.izz

[Armando]

@jfg aven't found the way but look around the developer more you can control pretty much everything. I just shared as soon as I have found out through the developer. I'll invite him on the forum

[colinbdclark]

Hi all, I'm the creator of MovementOSC. I'm super excited to see that it's useful to you all.

First, @Armando, thank you so much for posting about it and sharing how you've set up Isadora with it. 

I actually designed the "Bundled Message Per Axis" format with @princeCarlosthe5 specifically for Isadora, and we used it extensively in a residency at Jacob's Pillow a couple weeks ago to track dancers (both from archival footage and live), visualize the data, and transform it into a format suitable for a cable robot. I'm not an Isadora expert (yet!), but as far as I know you should be able to use that format directly without having to use the JSON Parser actor or do any other kind of string parsing that might slow things down. Hopefully Carlos might be able to share what his configuration looked like.

I responded to @jfg in the other thread about how settings aren't yet saved when you quit the application, but it is a feature I'm planning to add when I have some time. 

@dbini, I'm glad to hear it was interesting to you, and I'd love to hear how your robotics students used it!