(Kinect) - Isadora - OSC - VRChat
-
Hi all,
I've made a baby step and wondering if others are further down the path. I'm working on a project that's partially experienced in virtual public spaces in VRChat. With the new VRChat Beta release, avatar parameters can be controlled through OSC. This means that (amongst many other things), we can ostensibly use OpenNI for full-body tracking without extra hardware. I've managed a straightforward proof-of-concept modulating parameters like outfit color, hair-length, etc., through Isadora, but I'm hoping to get the Kinect in the chain as well. Has anyone had experience with this? I'm sure there are hurdles I'm about to tumble into.
Best,
Rory
MacBook Pro, M1, 2020, Sonoma 14.5
-
Getting the Kinect data with Isadora will be easy with the OpenNI Tracker BETA actor plugin. Once you've downloaded and installed it, you can follow the tutorial for it here: https://troikatronix.com/add-ons/tutorial-openni-tracker/
From there I think the biggest hurdle will be math and scaling because you'll need to be getting the range of the Kinect Data.
It's not for the Kinect, but if you take the patch from here and swap out the Rokoko Studio Live Watcher and replace it with the OpenNI Tracker actor you should be able to get a basic idea of how to use the skeleton data to control things.
You can also take a look at my OpenNI example files here for a bit of information on how to do fuzzy logic/comparing the locations of body parts and using specific conditions to trigger things (e.g. When my right hand is above my head, do thing x.)
