​Kinect v2 Actor on Mac?

[mark]

@josharoo4 said:

Kinect v2 OpenNi Tracker and Skeleton Actor on Mac? Any chance you are  going to release it? 

I have not focused on this 1) because I've been working hard on some awesome new audio features, and 2) because the coronavirus pandemic threw everything off starting about 10 days ago. I would love to, but I simply cannot say at the moment. The main problem now: I am on lock down, and I don't have a Kinect v2. How can I even get one?

That said, one important question: are your students using macOS, Windows, or both?

Best Wishes,
Mark

[Woland]

At the moment the possibility to use the Kinect v2 also exists on Windows via @DusX 's Kinect2Share with Isadora 2.6.1 (and I think 3.0.7 as well because it uses OSC and Spout)

Theoretically you could have a Windows machine running this and stream the OSC and an NDI video feed to a number of Macs over a wired local area network.

[josharoo4]

@mark TOTALLY UNDERSTANDABLE. AND Thanks so much for ALL that you do. Im sure you were totally swamped BEFORE all this started happening. 

I thought maybe since the V2 was listed as a possibility in the OpenNI tracker Read Me that there might already be some progress. Not sure about getting you a kinect v2,  but  Ha, I'd LOVE to mail you one from Boston!  I have a few spare.   Guess we'll just stick to our previous method via Ni-Mate for now. 

and

@Woland ,  many thanks about @DusX Kinect2Share. I've used it before with Izzy  2. (And many thanks to @DusX for all his amazing work). Havent tried in izzy3 but maybe I'll test tonight! The osC and NDI are good ideas but we're trying to figure out some remote learning options for my Interactive design class. Students with Macs and Windows in different places. In the past, we've been able to work together in one room. 

[mark]

Dear All,

A communi angel reached out to me today and he's sending me his Kinect v2 so I can work on this while in lockdown.

So maybe progress can be made.

Thank you @Fred ;-)

Warm Wishes,
Mark

[josharoo4]

@Fred Saves the Day again (and again and again and again).  

[axelartes]

Kinect v2 OpenNi Tracker and Skeleton Actor on Mac??  YES Please! @mark
and yes, definitely @Fred saving the day everytime

[Juriaan]

Hi @mark, 

I know you are working on 20 things at the same time, but just going to ask the question here :) 

If you are done with the Kinect v2 would you be able to share it along so we can report issues / etc ? We are currently exploring it for a remote piece that we are doing.

[bonemap]

@mark and all,

I am always really intrigued by the comparison of Kinectv1 and Kinectv2 on the Mac platform. As far as I can tell from reports and my own experience with using the v2 on Mac with NiMate there are resolution, speed and range improvements and differences. It appears to me that the extended features of the v2 have been locked out of Mac software development. Features like hand (thumb) and face tracking don’t appear to be possible on Mac. Is this true or are there other features of the v2 that will be significant for a OpenNi Tracker in Isadora?

Best wishes

Russell

[axelartes]

@axel_acs said:

Any update on Kinect v2 OpenNi Tracker and Skeleton Actor please let me know. best

[axelartes]

@bonemap said:

have

 I am intrigued too. And I would like to know too if the extended features are/will be available on Mac. best

[mark]

@bonemap said:

Features like hand (thumb) and face tracking don’t appear to be possible on Mac. Is this true or are there other features of the v2 that will be significant for a OpenNi Tracker in Isadora?

Yes, that is true that the features you mention are not avaiable on macOS. That is because those features are not part of the open source/hacked driver for Kinect v2 on that has been cobbled together for macOS. The features you mention only available on Windows if you use Microsoft's official Kinect v2 SDK.

We have discussed on the team whether or not we should make a Windows only tracker that allows access to those features. But I am always concerned about having features on one platform that are not on the other.

Best Wishes,
Mark

[mark]

To everyone following this thread, a status report:

As I have alluded to elsewhere, I have an internal build of OpenNI Tracker that captures and body tracks using the Kinect v2 capturing on macOS. However, I do not have the same working on Windows.

The open source system on which OpenNI Tracker is based only accepts an image of 640x480 or 320x240. The macOS driver that hackers/programmers have created (libfreenect2) converts the Kinect v2's 512x424 pixel image to 640x480, allowing it to track.However, since the Windows version grabs the image using the Windows SDK, it is not the right size.

It is my intention to port the resolution conversion in libfreenect2 for macOS to Windows to solve this problem. But with all the guru sessions, coding new plugins (Screen Capture, NDI Watcher with audio) I simply haven't gotten this done yet. It is high on my priority list though, and now that we've added some features important to remote performance -- a topic that seems urgent at the moment -- I will turn my attention to getting Kinect v2 working for both platforms.

I guess I could release the Mac version for thsoe who want to use it, but I hate creating situations where there are special versions and users get confused. That always costs us money because we have to pay people to offer techincal support -- either here on the forum or via our ticket system.

Let me see how quickly I can get this stuff done. I want you all to have it, but there are only so many hours in a day.

Best Wishes,
Mark

[guevarajorge]

Hi @mark, thanks for your time,   

where can I find the internal build you mention here?

Mac16 2020, Catalina 10.15.7

isadora 3.1.1 

[DusX]

@guevarajorge said:

where can I find the internal build you mention here?

 The actor has been updated with support for Kinect V2. It is available at: https://troikatronix.com/add-o...

[guevarajorge]

@dusx 

m working with this one  but keeps on crashing when i connect the kinect v2.

m sending the crash reports every time Isadora opens again (i just sent 3 crash reports)

Any idea why is this happening ?

    

[DusX]

@guevarajorge

What model number of Kinect V2 do you have?
I have a 1520, and can test it on Mac tomorrow. I have a Catalina machine so that should be a good test.

[guevarajorge]

@dusx

Mine is also model 1520

The Kinect is working when i use it via windows bootcamp, Touch designer.    

[DusX]

@guevarajorge

I am experiencing the same crash on Mac with the Kinect v2 running on an Intel based Mac running Catalina.
I have submitted crash reports with details, so we will be able to look deeper into this.

The Kinect running via Bootcamp shows that the hardware is good.. so that is helpful.

The OpenNI actor is working as expected on Windows for me as well. So this appears to be a Mac only issue.