Kinect + Isadora Tutorials Now Available

Skulpture

Thanks for the detailed reply @Marci. We are semi-aware that this method has a lifespan. But we get a lot of questions on the forum, inbox messages and emails asking how to get the kinect sensor working with isadora. I must get two emails a week to my personal email. So we thought it was best to come up with an 'official' method. I must admit I was not aware of the 'iso_callback() error' I have had the occasional lock up - but nothing too bad. I often wondered why and this may be it. Like you've said, the lifespan of this method is limited with the take over and apple; prime sense, etc etc. Which is a shame of course. Similarly we get/I get a lot of questions about the Kinect V2 and why won't it work, etc. This is a tough nut to crack really.... But thanks for your help and notes on github, etc. Cheers.

Marci

It’s same discussion as we had previous. First question should be: skeletons and limbs, or blobs. What tool do i specifically need...?

If blobs, OpenKinect framework which is still maintained, supports KinectONE & USB3 fully iirc. If skeleton, OpenNI framework as no other choice. Skeletons are quicker and easier to get an idea going with, but usually the same can always be achieved with blobs also.

If you must use skeletons and it’s going to be mission critical to a show or installation, use an older MacMini / MBPro / Laptop that only has USB2 support dedicated to handling the Kinect side of things, and fire the OSC over the network to your ‘main’ system. Avoids all the risk.

gapworks

@Marci a ghost Image would be simply great!! I mainly need the shape of the performer/s and i´m not tracking a skeleton nor OSC date. Only a simple Shape trying to get close to an infrared cam quality.

Marci

Ghost support added... https://github.com/PatchworkBoy/isadora-kinect-tutorial-files/blob/master/Isadora_Kinect_Tracking_Mac/Isadora_Kinect_Tracking_Mac.pde

Run the sketch, let it get your skeleton, hit 5 key on keyboard to switch to ghost view, s key to disable skeleton.

Marci

To change ghost color, in the source code, search for...

// set Ghost color here

...and change the color(255,255,255) values: (R,G,B) where values can be between 0 & 255.

Add some blur via Isadora if needed.

DusX

@Marci What do you think would be involved in porting this over to pc?

Marci

Errrr.... Good question. On PC iirc we're reliant on the proper MS Kinect SDK aren't we? Thought that had nuked out the opportunity to use anything but the proper XBox for Windows models which would preclude me from testing...? Will need to read up on it. I'll look into it...!

mark

First, I've been so busy working on Isadora v2.2 that I haven't been looking at the forum much in the last weeks. I was excited to see all the energy here among the community for this Kinect solution. Thanks to everyone for contributing, and especially @Marci for updates and setting up the GitHub.

I wanted to note to @Marci that I tested the Windows version using a Kinect 1473\. So you definitely are not _forced_ to use Xbox for Windows models.

For a recent workshop, I did a quick and dirty implementation of multiple skeletons. Once v2.2 is out, I'll add that to the repro so everyone can have it.

Best Wishes,

Mark

Marci

Ah cool - in which case I’ll look to reproduce for win. Never really bothered hooking it up to a PC as it was always simpler to get things going on OSX.

Marci

Just skimmed through the windows sketch - yeah it'll be no bother to port. Will get it done this week.

bruper

Something I cannot find is the way to change the Processing sketch to start with camera in RGB mode or IR MODE.

I tried with no success to change it in the OSC multitrasmit of the "Isadora Skeleton Test Mac" supplied in
https://github.com/PatchworkBoy/isadora-kinect-tutorial-files

and the Processing sketch of 9 days ago and the today one.

I searched in the Processing sketch if there was something I would be able to change but nope... It starts on kCameraImage_Depth = 3.

I would like to try the others kCameraImage.

Thanks for enlighten me.

mark

Dear @bruper,

Look at this part of the code:

// --------------------------------------------------------------------------------

// CAMERA IMAGE SENT VIA SYPHON

// --------------------------------------------------------------------------------

int kCameraImage_RGB = 1; // rgb camera image

int kCameraImage_IR = 2; // infra red camera image

int kCameraImage_Depth = 3; // depth without colored bodies of tracked bodies

int kCameraImage_User = 4; // depth image with colored bodies of tracked bodies

int kCameraImage_Ghost = 5;

int kCameraImageMode = kCameraImage_IR; // << Set this value to one of the kCamerImage constants above

// for purposes of switching via OSC, we need to launch with

// EITHER kCameraImage_RGB, or kCameraImage_IR

You should be able to set it to kCameraImage_RGB to show the RGB channel.

Best,

Mark

bruper

@mark

thanks, yes how silly of me.. totally overlooked...

gapworks

@Marci thanks for the fast reply ! i will give it a try on Wednesday when i return from venice as i don´t travel with my kinect!

Marci

"tried with no success to change it in the OSC multitrasmit of the "Isadora Skeleton Test Mac" supplied in https://github.com/PatchworkBoy/isadora-kinect-tutorial-filesand the Processing sketch of 9 days ago and the today one.”

Like I said, if you start in RGB you can’t switch to IR. If you start in IR you can’t switch to RGB. Limitation of the Kinect hardware, not Processing. Nothing anyone can do about it regardless of what software you’re working in.

Both can’t be activated and switched between in a Processing sketch. You must choose one to work with. You can have (IR _OR_ RGB) & Depth & User & Ghost. You can’t have IR & RGB & Depth & User & Ghost.

Marci

The confusion everyone is making here is passing the camera images to Isadora full stop.

If you’re creating and affecting visuals in Isadora, pass OSC data to Isadora, and do everything visual in Isadora.

If you’re creating visual effects in processing, just use processing. Then only pass the syphon feed to Isadora to integrate that into an Isadora scene. Pass the minimal OSC data as needed to control anything additional you may want to control in Isadora, if anything at all.

The majority of folks dabbling in Kinect at the moment here seem to be wanting to produce visual effects that are done wholly in processing (i.e.: stuff from openprocessing.org, but substitute mouse for hand/kinect).

Isadora exists so people don’t have to learn code to create visuals. When using Processing, you’re just using Isadora as a camera / media player switcher / sequencer, and hardcoding all your visuals and interaction within Processing.

You have to abstract how you think about it. Work out what you want to achieve and WHERE to achieve it...

Marci

UPDATES

Have brought the Windows Processing Sketch up to date with the Mac Processing Sketch. Both now identical bar the syphon / spout bits (i.e.: Windows uses Spout, Mac uses Syphon).

Also, both sketches now output all detected skeletons via OSC to the address:

/skeleton/[useridnumber]/[limbpart]

I’ve updated the Mac Isadora file to reflect this in the stream setup FOR THE 1st SKELETON ONLY! I can’t do the Windows Isadora file as don’t have a windows machine handy I’m afraid (if someone wants to open up the Windows Izzy file and replicate across the OSC MultiTransmit actor and comments from the Mac file, and prefix all the OSC streams in stream setup with /skeleton/1 and attach it here, I’ll add that to the gitrepo also.)

To receive multiple skeletons in Isadora, users will need to head to stream setup and add the channels for /skeleton/2/ thru to /skeleton/6/, and then duplicate the various Kinect Point actors and set their channels correctly (again, if anyone wants to do that and send me the files I’ll happily add them to the gitrepo).

https://github.com/PatchworkBoy/isadora-kinect-tutorial-files

Lucmarcel

Marci -- when I pull the izzy file (sadora Skeleton Test Mac.izz) from github it tells me that the file can't open because it was written in a later version... I have 2.1 -- what can I do? I have a mac and just got 2.1 in Nov...

Marci

download the file from github again... sounds like an interrupted upload, so have just reuploaded it.

Marci

(For reference, it was written in 2.1)