Guru Session #13: Body Tracking with Depth Cameras (Fri May 8th, 6pm CEST/5pm GMT/12pm EDT/9am PDT)

DillTheKraut

Status report

Kinect 1414

MacBook Pro 2019 16"
i9 8C 2,4Ghz
64GB
Radeon HD 5500M 8GB
MacOS 10.15.4 Catalina

closest-point-viewer.command and run-simple-viewer.command running immediately without issues.

Isadora Plugins (0.9.2 and 0.9.6) running smooth and stable.

Zotac ZBOXNANO-AQ01
AMD A4-5000 APU with Radeon HD 8330
4GB RAM
Windows Kinect SDI v2.0

openni-tracker-tutorial works fluently.
But 3D Puppet is not working as three.min.js is missing (JS Quaternion Calc Angle actor).
The js file is located next to the openni-tracker-tutorial.izz patch.

Best
Dill

mark

@demetri79 and @jfg

So if I understand things correctly, the the only two people having problems are both on an older Mac Pro (aka "cheese grater" model).

@Demetri79 :

First, I don't think you have to try things ten times. Three is enough. If it doesn't work after three attempts, then it's not working.

That said: you have a Mac Pro 2012 5,1 running High Sierra 10.13.6, yes?

What processors/how many cores? (Can you please put your hardware configs in your signature line like @jfg? That save me having to go through this whole thread trying to find this information. Thanks. ;-) )

When you run the same plugins on your laptop, everything is fine, right?

Also, there was a recent update to High Sierra that broke, among other 32-bit apps, Abelton Live. Now, Isadora and OpenNI Tracker are all 64 bit, but I wonder if a software update could have caused your Mac Pro to stop recognizing the Kinect reliably. (Because -- based on your previous posts -- it worked well on the Mac Pro previously, right?)

@jfg :

You are also using a Mac Pro 2012 (3,33 GHz 6-Core Intel Xeon) running Mojave 10.14.6, yes?

When you run the same plugins on your laptop, everything is fine, right?

@Demetri79 & @jfg

My fear here is that the library "libusb" which handles communication to the Kinect is failing, especially since you both mention the PCI USB 3 cards you have installed. There is an NI Mate thread that details a user having problems with macOS Sierra on a Mac Pro that sounds really similar. (When I wrote this, the original NI Mate post wouldn't open; but you can read Google's cached version.) In that thread they instruct the user to try a different libusb included with NI Mate, but I don't think it ever solve the problem.

In the end, I may have you to install the homebrew package manager and compile libusb on your machine. When you do this, the library is compiled with specifically for the hardware and operating system on your computer. I don't know if this will solve the problem or not... but right now, I don't know what else to tell you two.

If you are willing to take the time to try compiling your own libusb with homebrew, let me know and I'll post the instructions.

Best Wishes,
Mark

mark

@juriaan (and all -- please read I want an opinion)

When I use the 'capture mode' input of the plugin and set it to Color I also receive the depth image. What I expected to happen is that I only get the color image and that the depth image was disabled / not active (So a red line instead of a green line to the projector)

I realize now why this is happening; it was actually by design. If you set the 'body tracking' to off, you will only get the color image. If 'body tracking' is on, then the plugin always enables the depth stream because you are telling the plugin you want to track bodies.

Do people find this confusing? Should I change it so that, if the mode is set to 'color', body/skeleton tracking is simply disabled?

Best Wishes,
Mark

mark

@juriaan said:

Nope, I didn't save the file. Will test that out later tonight when I have some spare time.

I have verified that if you have not saved the file, the current version of the plugin will not save the recording. I have fixed this for the next release.

Best Wishes,
Mark

jfg

@mark

cannot say nothing because for me the body tracking doesn't work

best

Jean-François

mark

@jfg said:

cannot say nothing because for me the body tracking doesn't work

OK. But it helps if you answer all my questions, so I'll ask them again:

1) From what you say, I assume that the answer to "When you run the same plugins on your laptop, everything is fine, right?" is NO. Is that correct?

2) But what about "You are also using a Mac Pro 2012 (3,33 GHz 6-Core Intel Xeon) running Mojave 10.14.6, yes?" Is that true or not?

3) Did you try the test application in the kinect-test-macos.zip file I posted above? Can you follow the instructions there and give me the output of the Terminal window please?

Best Wishes,
Mark

jfg

@mark

First I use the Xtion and not a kinect.

All is running correct with openNI 0.9.2 on both computer Mac Pro 2012(3,33 GHz 6-Core Intel Xeon) and MacBookPro 2015 (2,8 GHz Intel Core i7) both with Mojave.

On both computer I get the depth picture but no body tracking with 0.9.5 and 0.9.6

best

Jean-François

jfg

@mark said:

If you are willing to take the time to try compiling your own libusb with homebrew, let me know and I'll post the instructions.

unfortunately it exceeds my ability

best

Jean-François

mark

@jfg said:

First I use the Xtion and not a kinect.All is running correct with openNI 0.9.2 on both computer Mac Pro 2012(3,33 GHz 6-Core Intel Xeon) and MacBookPro 2015 (2,8 GHz Intel Core i7) both with Mojave.On both computer I get the depth picture but no body tracking with 0.9.5 and 0.9.6

Thank you. OK, I knew you were using the Xtion. My partner may have one here... but at the moment we cannot find it. If we find it, then it will of course help me to figure out why you cannot track.

It's good to know you get the the depth image. That means your problem is different from @Demetri79. If you see the depth image, then it is not libusb and you do not need to try homebrew anyway. You also don't need to try the test application if you see a depth image.

But wait: did we make a custom masquerade.txt file for you that is different than the default? What does it say? Can you .zip it and post it here?

I will continue to seek answers for your issue.

Best Wishes,
Mark

jfg

@mark said:

But wait: did we make a custom masquerade.txt file for you that is different than the default? What does it say? Can you .zip it and post it here?

you don't. I used the normal masquerade.txt and it works with 0.9.2

masquerade.txt(0.9.2).zip

masquerade.txt(0.9.6).zip

I have also tried to use the masquerade.txt from 0.9.2 in 0.9.6 but without success. Same result.

thanks a lot

best

Jean-François

mark

@jfg

Try using this masquerade file (Note: for macOS only). It's a little bit of a guess... but maybe it helps.

masquerade.txt.zip

Best Wishes,
Mark

jfg

@mark

Dear Mark,

you are sooooo good. It works on both computer. On the MacBook Pro (USB 3.0) and on the MacPro (USB 2.0).

With USB 3.0 on the MacPro I have the same problem as Demetri79 but it is not important because it works with 2.0.

Another strange difference between MacBookPro and MacPro (or between USB 2 and 3) is the color video output:

- On the MacPro I get the video with the same little delay as the depth video.

- On the MacBookPro (USB 3.0) the video color picture is frozen on the first picture also if I switch the capture mode on "color"

On both computer the capture mode set on "color" show both picture (depth and color)

I find it a little bit confusing because there is also "both".

thanks a lot

best

Jean-François

JJHP3

More fodder for the mill... MBP 2012, 10.13.6 = simple viewer, as before, not working. You have the script error above. Thought I'd try the same process with iMac (Retina 5K, 27-inch, Late 2014) w

(AMD Radeon R9 M295X 4096 MB) 10.13.6 and simple viewer works, but with lots of faults in the terminal. - John (tried adding terminal info, but don't see the file I attached-sorry)

DillTheKraut

@dillthekraut said:

But 3D Puppet is not working as three.min.js is missing (JS Quaternion Calc Angle actor compile error).The js file is located next to the openni-tracker-tutorial.izz patch.

Someone any advise about this?

jfg

@mark

works still well. On the MacPro (USB 2.0) if I go out of the picture it takes 5-10 s to track the body and get the skeleton and sometime the skeleton get lost and I haven't find why it is coming back (could be 30 s) or not.

I think not so important because all the new computer have USB 3.0. I haven't these problems on the MacBookPro (USB 3.0)

record and play work well.

thanks

best

Jean-François

Juriaan

@mark

Regarding your question about the color mode.

It seems that we have 2 inputs

Capture mode
output depth

Our help file of the capture mode tell us that we can use the capture mode to tell or we wish to capture the depth / color and send it to the desired output.

Output depth overrides this behavoir and basically makes it send out a depth image while you said against the Capture mode that you only want a Color mode for instance.

I personally wouldn't disable tracking when you set it to 'Color', that is where the Body tracking parameters are for. I would be fan if we can just disable the depth-video output if we only tell Izzy that we wish to capture color. (Internally you can still fetch the depth map for the inner workings of the plugin, just dont send it out to the output of the plugin so that the video signal is cut)

jfg

@mark said:

Do people find this confusing? Should I change it so that, if the mode is set to 'color', body/skeleton tracking is simply disabled?

There is something confusing but for me it is that the capture mode "color" or "both" give the same result at output. I think it will be more clear to have only 2 possibilities: "only depth" and "color+depth".

I think it is the purpose of this actor to always track the body even if you only use the color picture as output.

best

Jean-François

JJHP3

@jjhp3 Here's my terminal response to simple viewer launch: Desktop-simple-viewer.rtf.zip

John

Demetri79

@mark

sorry didnt realize about the signature thing.. i filled that in. but my main Isadora computer is:

MAC PRO 2012 5,1 12 CORE XEON 3.46GHZ, 64G RAM
RADEON 5770, PCIE USB3, PCIE ESATA, PCIE SSD
MAC OS 10.13.6 , ISADORA 3.0.7

Yes the 2013 MacBook Pro laptop I tested all works fine with the same Kinect, isadora, OS and plugins.

yes it did seem to work fine on the mac pro for 2 days before the guru session happened. i even tested a backup of my OS on a different physical drive from 3 weeks ago..before I did the ITUNES update that broke 32 bit, and the kinect didn't work on that either..But then I restored the mobileframework on my current OS drive in /private frameworks and got 32 bit apps working again. i would be willing to try anything at this point.. so please send me the the homebrew info you are suggesting. thanks

Demetri79

@mark

Today so far first bootup of the day the kinect was working.. only for about 2 min again. and then data and depth image freeze. this time was able to get it reconnected with only restarting isadora not even unplugging the camera. now then got it working for about 10 min.. but the depth image is a bit choppy.. only running 15% load on isaodra. then it froze again. So far In my testing every time i actually get it recognized the image and data freeze up soon after. Also Im NOT having the power light on the kinect stop blinking no matter what USB port I try so seems like the power bug went away.. and I haven't done anything different today.. still currently testing with Open NI 096.. would you consider that the most reliable version or is 092? i dont need to record feature right now.

p.s. also test delicode ni mate v2.14 and the app wont even open, it was the only and i think latest version avilable on their site.