Guru Session #13: Body Tracking with Depth Cameras (Fri May 8th, 6pm CEST/5pm GMT/12pm EDT/9am PDT)

mark

Dear Everyone,

It was a great session yesterday, thank you for joining.

I have responded to each of you individually below. But want to start with my response to @primaldivine because you should all read what I had to say.

Please respond to my questions here, and I'll follow up with you.

Best Wishes,
Mark

P.S. Your help in promoting Isadora during a time where sales are affected by the pandemic would be much appreciated. If you liked the session, it helps us if you make a commment on YouTube. If you liked the session, please make a comment on YouTube -- especially if you can say something a bit more detailed -- for example: "I really liked the part where..."

@primaldivine

Hardware/Camera: n/a

-- A quick note to Mark that the javascript actor in your demo file is working fine for me now

Yes, everyone should know that the Javascript actors I have in both the tutorial and the guru session files require an external Javascript file called "three.js" If that file is not present in the same folder as the Isadora file, those Javascript actors will not work.

@Light-FX-UK

Computer: ???
MacOS Version: ???
Depth Camera: ???
Isadora Version: 3.0.7?

-- "Sorry I couldn't get the Kinect to be recognized, just a black screen."

I need to know which platform (Mac or Win) and what version of the operating system you are using.

Are you saying that the 'status' output never said 'ready'?

Also, on Windows, I forgot a super important piece of information: you need to install the Kinect SDK. If you are in fact on Windows, can you please install this:

https://www.microsoft.com/en-u...

And see if it improves the situation?

@Astrid

Computer: ??? (It's a Mac, right?)
MacOS Version: ???
Depth Camera: ???
Isadora Version: 3.0.7?

-- "Sadly, I still seem to have an incomplete OpenNI Tracker"

Basically you somehow do not have the latest version of the plugin. Could it be that you have put a copy of an older program into Isadora itself maybe?

To debug this, I would have you try the following process:

1) Go to /Applications/Utilities and open the Terminal program
2) Copy the command below and paste it into the Terminal window

sudo find / -name "OpenNI Tracker.izzyplug" -print 2>/dev/null | grep "^/"

3) Upon hitting return you will be asked for your admin password. Enter it and hit return.
4) What follows will be a list of every copy of the OpenNI Tracker plugin on your entire hard drive.
5) Copy and paste everything that shows up after the command into a private message to me. (The paths listed aren't really sensitive data, but it's better that you send it privately to me.)
6) Respond to this post to say "Mark I sent the results, check your messages." to ensure I see the private message. (Easy for me to miss)

In the end, there should be only one path where this plugin exists:

/Library/Application Support/TroikaTronix/IsadoraPlugins_x64/OpenNI Tracker.izzyplug

But let me analyze the list of what you get and I'll tell you specifically which ones to erase.

@Demetri79

Hardware: Mac Pro
MacOS Version: High Sierra 10.13.6
Depth Camera: Kinect v1 (Model 1414)
Isadora Version: 3.0.7

-- My Kinect V1 (1414) and OpenNITracker v 0.9.2 was working fine for days and then booted up today and it doesn't work at all randomly.

Well, this is what I did on High Sierra 10.13.6 (17G13021) just now:

1) Shutdown computer, detach Kinect from computer, remove power supply from Kinect for at least
2) Start up computer.
3) Attach Power to Kinect.
4) Plug Kinect into computer. You should see the green LED on the front of the Kinect start to blink.
5) Start up Isadora
6) Add fresh OpenNI Tracker

This process resulted in me succeeding to get a depth image from the Kinect. So I'm not quite sure what to say.

Have you installed any homebrew formulas? You can find out by doing the following:

1) Go to /Applications/Utilities and open the Terminal program
2) Enter "brew list" without the quotes.
3) If it says "-bash: brew: command not found" then you do not have homebrew installed.
4) Otherwise, copy and paste the results of this command in your reply to me. What we're looking for is libusb, openni2, libfreenect, libfreenect2. It could be that having these on your computer can cause a conflict.

@DanielPineiro

Hardware: MacBookPro
MacOS Version: Catalina 10.15.4.
Depth Camera: Kinect v1 (Model 1414)
Isadora Version: 3.0.7

-- Unfortunately this is not working too well for me. It works fine but just for maximum of 2 or 3 minutes, and then the video freezes. The reset input does not work, neither changing scene. Only If I delete the OpenNI Tracker actor and put it again, it works again but just for minutes.

I ran OpenNI Tracker v0.9.5 on Catalina v10.15.4 (19E287) for over 10 minutes, with a single body (me) moving in and out of the frame and doing all kinds of weird/funky poses to try to confuse/crash the system. It worked perfectly for me.

The patch was simple: OpenNI Tracker -> Projector actor.

What I need from you are the .zipped crash reports. Go to

~/Library/Logs/DiagnosticReports/

And grab the crashes from yesterday that have Isadora in their name. Post them here and I will analyze them.

@nandada

-- Maybe this is a silly question but is there a way to use the colored silhouette as a masked canvas to put a movie into?

Open this patch: Creating a Mask with OpenNI Tracker.izz

There are comments inside to explain every step.

-- My Kinect 1414 has a 4x3 aspect ratio. Is there a way to change that to match the stage?

The only way to do this is to crop away some of the image. You could pass the output through a Chop Pixels actor, with the 'horz res max' set to 640 and the 'vert res max' set to 360. This will give you a 16x9 image, but will chop away the 120 pixels vertically (60 on top, 60 on bottom.)

DanielPineiro

@mark said:

Library/Logs/DiagnosticReports/

Hi Mark!

I send the crash reports.

Best regards!

Crash reports

mark

@danielpineiro

Well, unfortuantely those are not crash reports. Crash reports always end with .crash.

If you have files that end with .crash, then please do send them. These reports just show heavy CPU usage, but not the crash.

What I do see from the reports you sent (I think) is that your computer was running on battery power only. If that's true, maybe you should connect the computer to a powersupply? Perhaps it changes something with the Kinect?

Best Wishes,
Mark

DanielPineiro

@danielpineiro

Hi again! I guess what happen. I installed first the old plugins from the plugins download page, then I deleted them and installed the new ones from the guru session 13 folder without restarting my computer. After the last crash, I had to hard shutdown my computer and now the actor is working fine. I have been tracking for 20 minutes (and continue) without any issues. Great!!!!!

DanielPineiro

@mark

Hi,

This is the last report that I get just five minutes ago. Everything was working (with power supply, thanks!!) for near two hours.

I hope it can help!

Regards

DanielPineiro

@danielpineiro

Not sure if the file is upload or not (It was not), here it is

report

jfg

Asus Xtion Pro liveand Mac Pro 2012

has somebody an Idea where it is coming from:

I use OpenNi on my Macbook without a problem.

on my macPro (2012) I cannot get the OpenNI tracker to see a body. I get the depth picture from the Asus Xtion Live Pro but no body are tracked.

Both Computer run Mojave.

I have also tried on the same computer with an simple installation of High Sierra ( no extensions or programs running by start): same result.

Could it be a hardware problem (USB 2.0 instead of 3.0 or…)?

I have also installed Delicode Ni-Mate on both. It runs on the MacBook but crash by start on the MacPro.

Perhaps the other difference could be (in relation with USB 3.0) that I am sure I made the Hotfix for USB 3.0 on the Macbook but I don't know if I made it too on the MacPro. Unfortunately my Software Backup Harddrive died and the website with this hotfix doesn't exist any more. https://www.reconstructme.net/... But at least I think it was a firmware for the xtion and is independent from the computer.

one more thing: What I described was with a connexion over USB 2.0. I have also in the macPro a Card for USB 3.0 but if I connect the Xtion over USB 3.0 I only get device init error in the Openni actor

Help !!

thanks

Jean-François

mark

@danielpineiro

Very good to hear. Let me know what else you find out.

Best Wishes,
Mark

bonemap

@jfg said:

on my macPro (2012) I cannot get the OpenNI tracker to see a body

Hi,

When testing the Open Ni plugin a few months ago I was unable to initialise any of my Orbbec sensors directly attached to a Mac Pro (2013). They initialised immediately on the Mac Pro using a thunderbolt hub with USB3 ports. I also tried a USB 2 hub but the Orbbecs' didn't initialise through that either. My Kinect sensors worked directly on the MacPro it was my Orbbec Astra’s that required the hub workaround.

I don’t know if that will help your situation.

Kind Regards

Russell

Juriaan

Hi there,

Windows 10 (Isadora 3.0.7, Normal version)

When I try the sensor (Kinect 2 - Xbox One) I get an image and it sees my skeleton. When it does the garbage collection, so after 10 seconds or so when I'm out of the picture again you see the load going up and then the load goes down again but it crashes Isadora.(It also doesnt go down to the load before the load went up) I sent the crash report to your servers :) (Juriaan as License name)

nandada

@mark THANKS MARK!!

Demetri79

@mark thanks for the help so far.

I dont have any homebrew installed. it came back with "-bash: brew: command not found"

i tried the process you outlined over and over and still cant get it working again regardless of which version of the OpenNItracker i use. I tried every usb port on my machine as some are 2.0 and some 3.0 on a pcie card. Sometimes when the OpenNI is placed in the blank scene it immediately reads: "resolving" in status and does nothing.. sometimes it flashes "init device" for a while and then stops flashing that and stays on init device.. sometimes the kinect itself stops flashing green once i place the OpenNI actor.. ive tired everything.. deleted prefs again, disconnected all non essential devices from computer.. deactivated all my qartz plugins, corea audio plugins not loaded, deleted my global user actor folder (since it had a full setup saved with tracker, skeleton, etc).. makes no sense as it worked all week.. the only software I installed since it was working is Newtek NDI tools.

the kinect itself is not the issue ..i tested the kinect on my macbook pro 2013 also running the same Mac os 10.13.6 and isadora 3.0.7.. and IT WORKS.. just can't get it working again on my tower. the laptop is too old and slow to use for my insatiable needs:(

jfg

@demetri79 said:

on my tower

can you tell us which tower? it is the same as mine: Mac Pro 2012 (5.1) USB 2.0 native+ Inatek PCIe USB 3.0 Card.

best

Jean-François

mark

Dear All,

v0.9.6 of OpenNI Tracker is available in the "alpha builds" forum thread.

The record feature is now enabled. It seems to work well on both Mac and Windows now. I'm sure this feature will be a boon for development, since you can now record and play the depth streams and have the tracking work as if there were a person in front of the camera. Let me know how it works for you.

@jfg This version may solve your problem with the Asus Xtion. Try it and let me know.

Best Wishes,
Mark

DanielPineiro

Hi!

The record feature works fine for me! That´s great. Thanks Mark!!!!

mark_m

@mark said:

v0.9.6 of OpenNI Tracker is available in the "alpha builds" forum thread.

The Windows link doesn't seem to be working, but it's available at:
https://troikatronix.com/files...

Cheers

Mark

jfg

@mark

Dear Mark,

I have to apologise because I gave you yesterday wrong information. I tested yet all different configurations new and the result is:

- 0.9.2 works fine on both computers (I have to use the masquerade file)

-0.9.5 and 6 won't recognise bodies on both (I haven't forget to put the masquerade file at the right place)

on the Macbook with USB 3.0 the Xtion is recognised.

on the MacPro (2012) the Xtion is only recognised over USB 2.0. If I connect it on the USB 3.0 Inateck PCIe card i get only device init error

The color video output give me only a frozen picture on the macBook (usb 3.0).(the first picture after switching). Work ± on MacPro (Jitter)

thank you for your help

best

Jean-François

JJHP3

Just sayin' the new openNI plugs are TERRIFIC! They work perfectly with my aging MBP laptop and kinect 1414. I used the kinect for a performance/installation back in 2012 and it was a pain... it then went on the shelf. I used DeliCode program in get the info into Isadora and then it was a bit of a crap shoot whether it would work or not. Congratulations to Mark and the team on getting this wonderful re-addition to Isadora up and running. The tutorial and the guru session were both most useful. Bravo ~

John

DillTheKraut

@mark

Hello and congratulations to this amazing peace of plugin. Same as @JJHP3, for the Guru Session I freed my 1414 from the years old dust and it ran on my MacBook in no Time!
I'm especially impressed by how fast the body recognition reacts and the accuracy of the skeleton tracking.

But I do have an issue on a windows pc. With the OpenNI plugin in the plugin folder, I get the following error:

This is true with Tracker plugin ver. 0.9.6 and 0.9.2

Isadora 3.0.7
Windows 10 Pro 1909 1836.592
AMD A-5000
4GB RAM

Best
Dill

DillTheKraut

@dillthekraut

I could solve the starting error by installing the OpenNI drivers from structur.io. https://structure.io/openni

But I do have a general issue now, as the pc isn't recognizing the kinect right. In the windows device manager list, only the XBOX NUI Monitor is appearing under other devices, without a proper driver associated.
At a Parallel Windows VM, it shows additional XBOX NUI Audio and XBOX NUI Camera. Which is missing on the pc. The actor in here keeps saying 'resolving uri'.
Does anyone have an idea?

Dill