Guru Session #13: Body Tracking with Depth Cameras (Fri May 8th, 6pm CEST/5pm GMT/12pm EDT/9am PDT)

n.jones

Demetri79

yes i have all the skeleton tracker outputs feeding limit scales, then into smoothers.. I should have included a screenshot of the actors for people who didn't want to open the patch and have a look.. this doesn't totally eliminate the jitteriness while allowing for a responsive reaction time.. i dont want to completely choke the output otherwise once I move my hand it will take a while before the corresponding object motion happens... just have to keep experimenting with different approaches.

the thing I am looking for is a math equation to progressively scale the hands x and y values based on its z value..

My scenario is not an actual 3D file (.3ds) mine is made from quartz composer (CI) so I can input different videos to each side of the cube.. it looks 3D but isn't actually so that javascript actor provided in the tutorial for the sphere doesn't apply to my situation. I would have tired to use a .3ds cube but you can only utilize 1 texture input in isadora correct? so that means i can only play the same video on all side as apposed to what i want is different videos on each side.. my quartz cube actually has 7 video inputs.. (6 sides + a background behind the cube). that is what I would want in the 3D word but seems unachievable currently unless i am mistaken. My version also has other added benefits like being able to have all of the dimensions manipulated in realtime with an essentially infinate Width, Height, and depth.. so just need to work out some better scaling math.

The .3ds files do have a distort parameter but it is very limited in how much it can stretch and reshape compared to what mine can do.. that is another aspect I would like to explore to see if it is possible do more extreme reshaping of .3ds files.

mark

@demetri79 said:

I would have tired to use a .3ds cube but you can only utilize 1 texture input in isadora correct? so that means i can only play the same video on all side as apposed to what i want is different videos on each side.. my quartz cube actually has 7 video inputs.. (6 sides + a background behind the cube). that is what I would want in the 3D word but seems unachievable currently unless i am mistaken.

Well, if you have a single cube wrapped with one texture, you can most definitely get a single image on each side. It's simply a matter of mapping the images into the right place in the texture you're feeding into the texture map. I would use a of Matte or Matte++ actors (one for each side) to get the individual images in the right place in the texture, and then feed the final output of that to the texture map input of the 3D Player actor.

I'm sorry I can't provide an example the moment. I'm busy preparing for today's session. But maybe that's enough to get you going.

Best Wishes,
Mark

mark

Dear Community,

Here's the links for Guru Session #13: Body Tracking with Depth Cameras

Watch the Live Stream (also good for viewing later)

Download the session materials. (This upload will not be complete until 5:50pm! Please don't try to download before that time!)

See you a few minutes!

Best Wishes,
Mark

primaldivine

@mark

Live streams says private only.Probably not started yet, but usually you had it up ten minutes before.

crystalhorizon

It says private session

nandada

@mark YouTube says the video is private!?

jfg

Video private

mark

@jfg

Sorry everyone for my mistake.

The video is public now.

Best Wishes,
Mark

Light FX UK

Thanks for trying to answer the question in the broadcast.

Sorry I couldn't get the kinect to be recognised, just a black screen. I have attached the Isadora screen for info. Appears to be an error on the status and on the sensor uri. I closed down Izzy and reconnected but unfortunately no joy.

Thank you

Gavin

Astrid

During the session in the chat I mentioned our research on OSC over internet for BH
Keen to see if it works will all this data from the skeleton decoder (bet it will be fine)

Here's our notes in dutch (hope translate works for you guys):

OSC transmit over internet (various locations)

credits to my HKU colleagues Tjerk Stoop, Tony Schuite & Simone van Dordrecht.

Astrid

Sadly, I still seem to have an incomplete OpenNI Tracker
(re-downloaded, got Jurriaans working one & robooted after throwing out previous versions, all to no avail.
My version of the plugins is from today 16.56)
I am connecting to the Kinect, but my numbers are in a quirky range, would love to adjust the m/cm/mm thing....

Below is what I see (checked but there a no hidden properties).
Any ideas or advice?

Demetri79

My Kinect V1 (1414) and OpenNITracker v 0.9.2 was working fine for days and then booted up toady and it doesnt work at all randomly. I dind't change anything from last night when it was working. In the OpenNI tracker BETA status output i see: init device .. when i hover over the stauts it shows a message : inactive device init err ?? Sometimes status says resolving, sometimes, no device. So i the deleted all 3 actor plugins: openni tracker, skeleton decoder, skeleton visualizer, and installed the new ones provided today with guru 13 materials and restarted computer.. still same results.. not working at all.. same error message: inactive, device init err?? i tried all the obvious, restarting, unplug kinect usb, unplug power to kinect, re plug in, remove opennitracker actor, deleted prefs,tried every possible order for trouble shooting…isadora crashes when i even try and delete the opennitracker actor from a scene with no other actors in it. also crashes isadora if i even try to close the patch or quit isadora. so i went back to the previous versions before the ones provided today.. same results.. its all not working and makes no sense.. nothing changed.

mac pro 2012 high sierra 10.13.6, isadora 3.0.7 OpenNI tracker v.0.9.2 and 0.9.5 skeleton decoder 0.8.3 and 0.9.1 skeleton visualizer 0.9.1 and 0.9.0

DanielPineiro

Hi!

Thanks for the todays guru session. OpenNI Tracker actor has a lot of possibilities!!!

Unfortunately this is not working too well for me. It works fine but just for maximum of 2 or 3 minutes, and then the video freezes. The reset input does not work, neither changing scene. Only If I delete the OpenNI Tracker actor and put it again, it works again but just for minutes.

I am using Isadora 3.0.7 and a Kinect 1414 on a MacBook Pro running Catalina 10.15.4.

Thanks for the help!!!!

Best regards!

Daniel

primaldivine

A quick note to @mark that the javascript actor in your demo file is working fine for me now that I realised I had saved my Isadora file to a different location on my drive. It couldn't find the other files needed in your guru 13 package. All working fine for me. I'll try adding my second kinect camera tomorrow.

Thanks for a great session today.

Jamie

nandada

For a while nothing was working for me, so I went for a walk and when I came back it was all humming along. I guess the Goddess smiled upon me. Maybe this is a silly question but is there a way to use the colored silhouette as a masked canvas to put a movie into? Another simple question-my Kinnect 1414 has a 4x3 aspect ratio. Is there a way to change that to match the stage?

nandada

Also and most importantly-Mark and Eni-----THANK YOU SO MUCH!!!! for these awesome sessions. They are a landmark I can center my workweek around and are really helping me get off the couch and feel like there is a future. Isadora's user support and community is not like anything else I have experienced in the art world. Just wonderful and heartwarming generosity in what are some not so wonderful times. Abraccioni da Portland

mark

Dear Everyone,

It was a great session yesterday, thank you for joining.

I have responded to each of you individually below. But want to start with my response to @primaldivine because you should all read what I had to say.

Please respond to my questions here, and I'll follow up with you.

Best Wishes,
Mark

P.S. Your help in promoting Isadora during a time where sales are affected by the pandemic would be much appreciated. If you liked the session, it helps us if you make a commment on YouTube. If you liked the session, please make a comment on YouTube -- especially if you can say something a bit more detailed -- for example: "I really liked the part where..."

@primaldivine

Hardware/Camera: n/a

-- A quick note to Mark that the javascript actor in your demo file is working fine for me now

Yes, everyone should know that the Javascript actors I have in both the tutorial and the guru session files require an external Javascript file called "three.js" If that file is not present in the same folder as the Isadora file, those Javascript actors will not work.

@Light-FX-UK

Computer: ???
MacOS Version: ???
Depth Camera: ???
Isadora Version: 3.0.7?

-- "Sorry I couldn't get the Kinect to be recognized, just a black screen."

I need to know which platform (Mac or Win) and what version of the operating system you are using.

Are you saying that the 'status' output never said 'ready'?

Also, on Windows, I forgot a super important piece of information: you need to install the Kinect SDK. If you are in fact on Windows, can you please install this:

https://www.microsoft.com/en-u...

And see if it improves the situation?

@Astrid

Computer: ??? (It's a Mac, right?)
MacOS Version: ???
Depth Camera: ???
Isadora Version: 3.0.7?

-- "Sadly, I still seem to have an incomplete OpenNI Tracker"

Basically you somehow do not have the latest version of the plugin. Could it be that you have put a copy of an older program into Isadora itself maybe?

To debug this, I would have you try the following process:

1) Go to /Applications/Utilities and open the Terminal program
2) Copy the command below and paste it into the Terminal window

sudo find / -name "OpenNI Tracker.izzyplug" -print 2>/dev/null | grep "^/"

3) Upon hitting return you will be asked for your admin password. Enter it and hit return.
4) What follows will be a list of every copy of the OpenNI Tracker plugin on your entire hard drive.
5) Copy and paste everything that shows up after the command into a private message to me. (The paths listed aren't really sensitive data, but it's better that you send it privately to me.)
6) Respond to this post to say "Mark I sent the results, check your messages." to ensure I see the private message. (Easy for me to miss)

In the end, there should be only one path where this plugin exists:

/Library/Application Support/TroikaTronix/IsadoraPlugins_x64/OpenNI Tracker.izzyplug

But let me analyze the list of what you get and I'll tell you specifically which ones to erase.

@Demetri79

Hardware: Mac Pro
MacOS Version: High Sierra 10.13.6
Depth Camera: Kinect v1 (Model 1414)
Isadora Version: 3.0.7

-- My Kinect V1 (1414) and OpenNITracker v 0.9.2 was working fine for days and then booted up today and it doesn't work at all randomly.

Well, this is what I did on High Sierra 10.13.6 (17G13021) just now:

1) Shutdown computer, detach Kinect from computer, remove power supply from Kinect for at least
2) Start up computer.
3) Attach Power to Kinect.
4) Plug Kinect into computer. You should see the green LED on the front of the Kinect start to blink.
5) Start up Isadora
6) Add fresh OpenNI Tracker

This process resulted in me succeeding to get a depth image from the Kinect. So I'm not quite sure what to say.

Have you installed any homebrew formulas? You can find out by doing the following:

1) Go to /Applications/Utilities and open the Terminal program
2) Enter "brew list" without the quotes.
3) If it says "-bash: brew: command not found" then you do not have homebrew installed.
4) Otherwise, copy and paste the results of this command in your reply to me. What we're looking for is libusb, openni2, libfreenect, libfreenect2. It could be that having these on your computer can cause a conflict.

@DanielPineiro

Hardware: MacBookPro
MacOS Version: Catalina 10.15.4.
Depth Camera: Kinect v1 (Model 1414)
Isadora Version: 3.0.7

-- Unfortunately this is not working too well for me. It works fine but just for maximum of 2 or 3 minutes, and then the video freezes. The reset input does not work, neither changing scene. Only If I delete the OpenNI Tracker actor and put it again, it works again but just for minutes.

I ran OpenNI Tracker v0.9.5 on Catalina v10.15.4 (19E287) for over 10 minutes, with a single body (me) moving in and out of the frame and doing all kinds of weird/funky poses to try to confuse/crash the system. It worked perfectly for me.

The patch was simple: OpenNI Tracker -> Projector actor.

What I need from you are the .zipped crash reports. Go to

~/Library/Logs/DiagnosticReports/

And grab the crashes from yesterday that have Isadora in their name. Post them here and I will analyze them.

@nandada

-- Maybe this is a silly question but is there a way to use the colored silhouette as a masked canvas to put a movie into?

Open this patch: Creating a Mask with OpenNI Tracker.izz

There are comments inside to explain every step.

-- My Kinect 1414 has a 4x3 aspect ratio. Is there a way to change that to match the stage?

The only way to do this is to crop away some of the image. You could pass the output through a Chop Pixels actor, with the 'horz res max' set to 640 and the 'vert res max' set to 360. This will give you a 16x9 image, but will chop away the 120 pixels vertically (60 on top, 60 on bottom.)

DanielPineiro

@mark said:

Library/Logs/DiagnosticReports/

Hi Mark!

I send the crash reports.

Best regards!

Crash reports

mark

@danielpineiro

Well, unfortuantely those are not crash reports. Crash reports always end with .crash.

If you have files that end with .crash, then please do send them. These reports just show heavy CPU usage, but not the crash.

What I do see from the reports you sent (I think) is that your computer was running on battery power only. If that's true, maybe you should connect the computer to a powersupply? Perhaps it changes something with the Kinect?

Best Wishes,
Mark