[ANSWERED] Skeleton Tracking/Decoding/3D Puppet from GURU 19 vid

[jmellin]

Hello fine Isadora users.  I'm looking to use the d435 depth camera with the Open NI tracking for a interactive dance performance later this year in NYC.  I was excited to see the demos from the GURU #19 video and thought I'd try my hand at it switching out the rokoko with the Open NI tracker.

When I do that, the puppet that is created doesn't appear to follow what's in the video or what I expected to see- what I do see is a few models mapped to the skeleton, but in strange ways. I'm adding a video here if it helps describe the issue.

Has anyone encountered this?  Is the skeleton model different from rokoko to OpenNI tracker?

https://drive.google.com/file/...

[jmellin]

@jmellin - Actually - has there been any further testing of all of this with the new orbbec cameras like the Astra 2?  It boasts some better specs.  I'd buy it and try it out if it were supported.

[dbini]

@jmellin - have a search for Orbbec on this forum - there have been some discussions about the various depth cameras available since Kinect was discontinued, and I know that Bonemap has tested a few recently.

[Woland]

@jmellin said:

Is the skeleton model different from rokoko to OpenNI tracker?

Yes, Rokoko provides xyz data for 17 joints and OpenNI provides xyz data for 15 joints. There is some overlap, but they are not immediately interchangeable so things built for Rokoko Studio Live Watcher data will not be 1-to-1 transferrable to a new use case where OpenNI Tracker data is used; you will need to go into the patch and figure out where the changes need to be made (likely wherever it is that specific joints are assigned to specific 3D Player actors).

You can see the list of skeleton joints by right-clicking the OpenNI Tracker or Rokoko Studio Live Watcher and choosing the top option from the contextual menu "Show Actor Help In Browser". If you scroll down there will be a list of the joints for those specific actors (each actor has its own specific joint list in its own additional HTML actor help).

Rokoko

Joint Names
This is a list of the joints sent by this actor. The joints under BODY are always present for the Smartsuit Pro. The joints under LEFT HAND and RIGHT HAND are present if the Rokoko Smart Gloves are attached to the Smartsuit Pro.

BODY
Except for the joints in the center of the body (e.g., head), each joint will have a left and a right component.

hip
chest
stomach
neck

head

left-shoulder
left-upper-arm
left-lower-arm
left-hand

right-shoulder
right-upper-arm
right-lower-arm
right-hand

left-upper-leg
left-lower-leg
left-foot
left-toe

right-upper-leg
right-lower-leg
right-foot
right-toe

---

OpenNI

The joints sent by the skeleton outputs of this actor are as follows:

head

neck

left shoulder

right shoulder

left elbow

right elbow

left hand

right hand

torso

left hip

right hip

left knee

right knee

left foot

right foot

[Woland]

@jmellin these are two (old) patches of mine that are built around OpenNI that may make the process of data-wrangling easier or clearer:

kinect-example-alpha-mask.izz

openni-tracking-and-gesture-demo-v2.izz

There's also the official TroikaTronix OpenNI Tutorial file: https://troikatronix.com/add-ons/openni-tracker/