Kinect + Isadora Tutorials Now Available

Skulpture

Am I right in thinking this will only track one person at a time?

mark

Well, yes and no. I had at least six people in the frame at one point and they all had Skeletons. But I didn't revise the code to _handle_ those multiple skeletons. We should make this change. I can't do it at the moment with my coding responsibilities for the main program, but maybe someone here has the desire to make it happen?

Best,

Mark

leben

Hello,

I am running Izzy version 2.1 on a mbp retina on yosemite. I have followed all the steps for this tutorial, I have run the processing sketch and have an image coming from the kinect in the viewing box, but there is no skeleton image..am I missing something?

thanks

Pascal

Hello,

It works great...thanks a lot for all.

Best,

Pascal

DusX

@ leben

It can take sometime. You need to stand in front of the kinect, allow it to detect your figure (will color your form a solid colour)

After a little while (if your form is clear) it should switch to adding the skeleton. (this process is very cpu intense, and on my PC almost freezes things for a couple seconds) but once in this mode returns to performing well.

gapworks

yes it works great! i was just wondering if there would be a possibility through processing to get kind of a ghost image and/ switch between different possibilities?

DusX

@gapworks

Like @mark mentioned in regards to tracking multiple skeletons, switching the video feed would have to be added to the Processing code.

I imagine that OSC control of the video type could be added.

Perhaps a Processing Pro in the forum might take this one and share the final script.

leben

@ DusX

All is working fine. Thanks for the help, and to those who wrote the tutorial. this all very exiting

mc_monte

Thanks all for the wonderful comments and helpful feedback. Get out there and make some cool art :D

Skulpture

I am guessing from the screenshots and from my own patch that all the red text (errors?) that show up in Processing patch are normal?

Marci

"I am guessing from the screenshots and from my own patch that all the red text (errors?) that show up in Processing patch are normal?"

Copy & paste a sample - all library initialisation output is generally red by default... so __shouldn’t__ be anything to worry about, but I can take a look.

"Like @mark mentioned in regards to tracking multiple skeletons, switching the video feed would have to be added to the Processing code.

_I imagine that OSC control of the video type could be added._

_Perhaps a Processing Pro [user] in the forum might take this one and share the final script."_

Can do... but probs not til the weekend.

DusX

@Marci

It would be a greatly appreciated contribution.

Pascal

Oh yessssssss

Marci

Have hoisted the code over to my GitHub repo, so just keep an eye on that... feel free to report issues via that too as we go along, or fork / contribute etc. Not done anything to it at the moment, just preparing the ground so to speak.

@Mark / @mc_monte / @dusx / @skulpture / @michel et alle - if you want some specific text putting in the README.md acknowledging orig sources etc, just fire it over to me via message or similar and I’ll swap it in. At the minute I’ve left license type as “none”... suggest ‘CC0' seeing as it’s all open source anyways. Thoughts?

See https://github.com/PatchworkBoy/isadora-kinect-tutorial-files

Skulpture

I've also done a video walkthrough (unofficialy)

https://www.youtube.com/watch?v=0HY5U6QSyhM

Marci

@skulpture - Added link to README.md

Skulpture

Cheers @Marci

Marci

OK - 2 things y'all need to be aware of when using these tutorials...

1: natural limitations of SimpleOpenNI

The SimpleOpenNI library used in Processing is one of a few frameworks for interfacing with a Kinect. It combines the old way of doing things where one had to manually install OpenNIv1 and NiTe etc to get skeleton / limb and user tracking, thus simplifying the process. The Kinect itself doesn’t do any skeleton or limb identification or tracking.

If on a USB3 host (Macbook Retina for instance), running anything other than Depth camera at the same time as skeleton tracking may randomly throw an iso_callback() error and/or trigger 'Isochronous transfer error log messages’... this is inherent to SimpleOpenNI and can’t be avoided, and will render everything unstable. It could go at any time, whenever it feels like it. It will either plain bomb the processing sketch (in case of the iso_callback() error), or cause _everything_ to lock up (in case of the Isochronous transfer error) until eventually the sketch bombs (which occasionally you have to force by simply pulling the USB cable).

The only way to get round this is to use a USB2 host (older Macbook Pro, MacMini), or just chance your luck. (I’ve documented this over on the GitHub Repo, yonder: https://github.com/PatchworkBoy/isadora-kinect-tutorial-files/issues/1)

I’ve documented previously on here that skeletons should only be used and implemented within a sketch when absolutely necessary / desired... as these are the only features unique to the SimpleOpenNI libraries and introduce a lot of CPU weight. If not using skeletons, turn to lighterweight OpenKinect libraries etc for simple camera feeds & depth point tracking / blob tracking. From the Isadora end of things, dealing with Skeletons is the easiest way of handling Kinect data in an obvious way... but from the Kinect middleware point of view, it produces the most unstable results. Purely a result of SimpleOpenNI being out of development (hence no Kinect v2 support, and no fix to this particular issue as when it went out of development [aka, the PrimeSense technology & software rights were snapped up by Apple then passed to Occipital and are now part of the Structure.IO SDK] USB3 wasn’t released - OpenNIv1 and v2 are now in a complete code freeze with no further development). Ultimately, that means this all has rather a limited lifespan - sorry!

And

- Mirror mode only affects the RGB / IR output. It has no impact on depth or user output.

2: natural limitations of the Kinect v1 hardware

You can either:

- Start in RGB mode, and switch between that and User & Depth mode.

OR

- Start in IR mode and switch between that and User & Depth mode.

Once the camera has been initialised in either RGB or IR mode, it can’t be switched to the other mode basically!

To change from RGB to IR mode, the sketch must be stopped and restarted.

To change from IR to RGB mode, the sketch must be stopped and restarted.

You can’t:

- Start in RGB mode and switch between that and IR mode

- Start in IR mode and switch between that and RGB mode

Bearing all of that in mind... the GitHub Repo is now updated with an updated Processing sketch and Izzy file (for Mac) with stream switching enabled via OSC and a few other bits.

Please read the Warnings on the GitHub page, the OSC MultiTransmit Actor notes in the Izzy file, & comments in the processing sketch code.

https://github.com/PatchworkBoy/isadora-kinect-tutorial-files

bruper

this is great, thanks

Skulpture

Thanks for the detailed reply @Marci. We are semi-aware that this method has a lifespan. But we get a lot of questions on the forum, inbox messages and emails asking how to get the kinect sensor working with isadora. I must get two emails a week to my personal email. So we thought it was best to come up with an 'official' method. I must admit I was not aware of the 'iso_callback() error' I have had the occasional lock up - but nothing too bad. I often wondered why and this may be it. Like you've said, the lifespan of this method is limited with the take over and apple; prime sense, etc etc. Which is a shame of course. Similarly we get/I get a lot of questions about the Kinect V2 and why won't it work, etc. This is a tough nut to crack really.... But thanks for your help and notes on github, etc. Cheers.