Guru Session #13: Body Tracking with Depth Cameras (Fri May 8th, 6pm CEST/5pm GMT/12pm EDT/9am PDT)

mark

Here are the instructions for homebrew.

1) Open a Apple's Terminal program (/Applications/Utilities)

2) Install the Xcode command line tools. Enter this command and hit return.

xcode-select --install

As indicated in these detailed instructions, a dialog will appear asking you to install the command line tools. Follow the prompts and install them.

3) Install homebrew. Enter this command and hit return.

Detailed instructions can be found here.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

It will take some time for this to complete.

4) Compile libusb on your machine. Enter this command and hit return

brew install libusb

This also may take some time.

5) Finally you need to get the compiled libusb. Type this command and hit return

open /usr/local/Cellar/libusb/1.0.23/lib

This will open a Finder window and you should see libsusb-1.0.0.dylib.

Note: if the version number of libusb changes, then the command above will not work. To find out what that new version number is, enter the two following commands

cd /usr/local/Cellar/libusb
ls -al

You will see the version number of libusb listed in the result -- in this case 1.0.24. You would use this number instead of 1.0.23 in the command above.

drwxr-xr-x   3 xxxxxxxx  staff   102 Mar 29 14:38 .
drwx------  41 xxxxxxxx  staff  1394 May  7 15:51 ..
drwxr-xr-x  12 xxxxxxxx  staff   408 Mar 29 14:34 1.0.24

6) With the finder window of the compiled libusb open, you now need to open a Finder window for the OpenNI Drivers in the OpenNI Tracker plugin. Enter this command and hit return

open /Library/Application\ Support/TroikaTronix/IsadoraPlugins_x64/OpenNI\ Tracker.izzyplug/Contents/Frameworks/OpenNI2/Drivers/

7) Now drag the libsusb-1.0.0.dylib from the lib folder to the Drivers folder.

And choose to replace the existing libusb

8) You now have a libusb custom built for your hardware inside of OpenNI Tracker. You can reboot Isadora and see if it improves the situation for you.

So please follow these instructions and see how it works out for you.

Best Wishes,
Mark

Demetri79

@mark thanks so much for your help i know you are doing a million things.. I will try a different type of camera if this doesn't work solidly for me.

is the orbbec astra the best most solid option?.. curious to hear form other users which camera they find to be the best overall. thanks everyone.

nandada

@demetri79 said:

orbbec astra

I'd also like to know if there is a higher resolution depth camera compatible with MAC OS 10 and the new actors. None of the newer cameras seem insanely expensive but do they work?

Demetri79

yeah I understand the Kinect v2 has higher resolution than the V1 so the question is between the Kinect V2, the Orbbec Astra, and the Intel RealSense. which is the best quality camera and which one is the most stable on mac with the Open NI Tracker.. I may just buy them all and then sell off the all but the best of them. but curious if anyone has compared them in the real world with isadora 3.

mark

@Astrid

Thanks for sending me the results of the search. I can see what's going wrong now.

You seem to have placed a copy of OpenNI Tracker inside of your Isadora USB Key application at /Applications/Isadora 3/. To remove it:

1) Find the Isadora USB Key app in /Applications/Isadora 3/
2) Right click Isadora USB Key and choose "Show Package Contents" from the menu
3) Go into the Contents folder
4) Go into the MacOS folder
5) Go into the Isadora Plugins folder
6) Delete OpenNI Tracker.izzyplug

The plugin should always be installed at /Library/Application Support/TroikaTronix/IsadoraPlugins_x64/ -- not in the Isadora application itself.

Best Wishes,
Mark

bonemap

@mark

I have no success with Mac OSX 10.15.4 installing the OpenNi Tracker Alpha 0.9.6. I tried a few times but I get a dialog telling me it is broken and a request to have it moved to trash. Is there a fix for that? I am very keen to try the capture feature.

Kind Regards

Russell

mark

@bonemap said:

I have no success with Mac OSX 10.15.4 installing the OpenNi Tracker Alpha 0.9.6. I

I am not sure what this could be. I downloaded the latest from this thread and installed it. No issues at all.

Best Wishes,
Mark

bonemap

@mark

Could it be anything to do with the masquerade.txt file? I will try again.

EDIT: FYI The dialogue appears after relaunching Isadora when the masquerade.txt file is added at the same time as installing the plugin.

If I install the Open Ni plugin without the masquerade.txt file the Isadora app launches and the Plugin is available. if I then install the masquerade.txt file i.e. restarting Isadora a second time it appears to function without the dialog appearing or the app crashing.

I have tried this several times now and it appears I am unable to place the masquerade.txt file at the same time as the OpenNI plugin.

Kind Regards

bonemap

@mark

What is the status of beta testing with the Kinect 2? are you wanting feedback on that one?

I have loaded the alpha 9.6 Open Ni on MBP 15" OSX 10.14.6 (second machine in my signature). The depth camera freezes at 2 - 3 minutes and takes down the Isadora app.

Kind Regards

mark

@bonemap said:

What is the status of beta testing with the Kinect 2? are you wanting feedback on that one?

I am working on it as we speak. As mentioned in the release, the Kinect v2 should not be considered reliable in v0.9.6.

Best Wishes,
Mark

mark

@bonemap said:

EDIT: FYI The dialogue appears after relaunching Isadora when the masquerade.txt file is added at the same time as installing the plugin.

OK. That makes sense. You are breaking the code signature by doing this. I will have to make an update so that the Masquerade.txt file lives somewhere else so this won't happen. (This is probably only going to come up on Catalina because of those wonderful security features Apple gave us.)

I will put that in my to do list.

Best Wishes,
Mark

Demetri79

Well I got my Kinect V1 1414 working again on the mac pro by simply deleting the masquerade file which was inside the tracker.. I thought I tried that in the early stages of trouble shooting but saw bonemaps post about that file and it inspired me to try this one more time and boom.. it works every time now.. so .0.9.6 is working like a charm now for 2 days.

nandada

Follow up session??? I'm loving working with my Kinnect and getting a lot of cool things to happen. But for me a follow up session like the happy hour we had a few sessions ago would be golden. In the scene where you use your hands to move memes around, I have successfully combine several videos and a masked out figure but am running up against a wall when I try to substitute a video with an alpha channel for the memes. The alpha channel is transparent in AfterEffects but black in Isadora no matter how I change the settings in the movie player.

mark

@nandada said:

The alpha channel is transparent in AfterEffects but black in Isadora no matter how I change the settings in the movie player.

Can you export one or two seconds of the movie and get it to me so I can examine it? The upload limit here is 4MB so the movie cannot be bigger than that. (You'll have to .zip it anyway before you upload.)

Best Wishes,
Mark

nandada

@mark
SleepSeconds_2.mov.zip

mark

@nandada

You said:

"The alpha channel is transparent in AfterEffects but black in Isadora no matter how I change the settings in the movie player."

But the movie you uploaded is an uncompressed 10 YUV Movie -- which I found out by hovering over the movie in Isadora. My friend opened your clip in After Effects and it also said that there was no alpha channel.

You must export the movie with a codec that actually supports an alpha channel, e.g., Apple Pro Res 4444 or HapA.

Best Wishes,
Mark

nandada

Thanks Mark, I'm puzzled-the Alpha channel showed up on my end at least in AE. I've been using an older version of Premiere which doesn't have those codecs but I will try on my other computer which has Creative Cloud on it. Thank you for taking the time for this.

enibee

@nandada I'm not sure how it is possible that you see an Alpha in After Effects. Does it also show up if you import the file you posted, right click on it and choose "Interpret Footage"?

As far as I know, Uncompresssed 10-bit codec didn't support RBG+, also in the older versions of After Effects/Premiere, which means you can either get a video with only the Alpha or only the RGB channels. If you want to do that, you'll have to add in the Alpha Channel in Izzy with the "Add Alpha Channel" actor. The codec you should have is "Animation", but it will possibly produce a very large file - make sure to check "RGB+Alpha" on export. Same with the codec set to none "None(Uncompressed RGB 8-bit).. However, if you have the possibility to use ProRes or Hap, I would recommend using one of the codecs Mark suggested above.

nandada

@enibee THANKS SO MUCH!!! This answers the question. I had tried animation and picked RGB + alpha but didn't know to use add alpha channel . It's a hassle to go back and forth between my Mac show machines and my PC editing rig but I will see if HAP is a possibility on my new Premiere

mark

@nandada said:

I will see if HAP is a possibility on my new Premiere

If Isadora is installed on your PC (even the demo) it will install HAP codecs. They should appear in Premiere. Thanks @enibee for sharing your expertise. ;-)

Best Wishes,
Mark