Storing OSC Data

bonemap

a sample file doing exactly this

Hi Ryan,

I would be very interested to see how the OSC data capture has been set-up and would very much appreciate access to the example patch file if it is available.

Best Wishes,

Bonemap

Juriaan

Alright.. I guess this is what you mean ? If not, please let me know and I will try my best to create an other one :) You need Isadora 2.6 (Latest stable release) to open this patch.

ExampleOSCWrite.izz

bonemap

@juriaan said:

I guess this is what you mean

Hi Juriaan,

Thanks for attaching a sample patch, it is really interesting for me to think about how incoming OSC skeleton data might be captured and stored by Isadora. There are a couple of considerations, in my humble opinion, that need to be reconciled by the patch.

Particularly, with an OSC skeleton from a Kinect with multiple XYZ coordinate floats representing each joint and armature of the tracking skeleton, therefore I do not think it is a trivial task. However, creating a well functioning system that captures just one triad of XYZ coordinates could then be duplicated for each point until the whole skeleton is captured.

My thoughts run first to the timing of the OSC data and to understand the accurate way to match or scale the rate of incoming OSC values to the pulse triggers that save the data to the Data Array. If the writing of data to the Data Array text file can keep up with the data flow enough to recreate the skeleton's tracked movement when playback is required is a big question for me?

Second is to plan a system with the Data Array that can capture multiple XYZ points associated with the skeleton armature simultaneously. My first thought, is to consider if the number of items required (for a skeleton it is going to be over 240 continuously changing OSC values) and what a single Data Array can handle to keep the system viable.

I imagine to start with a Data Array to have 3 items (x,y,z) and therefore 3 data inputs and three data outputs. To then build up the number of tracking points, checking the systems viability as each triad XYZ is added.

It is not clear at this point what latency or CPU strain level, capturing the data for a full OSC skeleton will induce in the system.

Just some preliminary thoughts.

Best Wishes,

Bonemap

Juriaan

Hi there @bonemap,

Ofcourse you right with that. It would be nice if we could get a format going to actually make this happen. So I'm all ears :) Personally I'm a bigger fan of combining it inside a single file then have 'X' files with proper naming ofcourse..

Timing is crucial and that is something that we can actually 'calculate' by checking the amount of packets that we receive in a certain time period. Let's say one second. If we have that information we can actually create a calculation and refresh rate for the data array write.

Personally I'm a big fan of a system where it is really clear what kind of data is coming in. So I'm more a fan of using 'x=20.05' then '20.05' and work with 'line' numbers instead.

bonemap

@juriaan said:

I'm a bigger fan of combining it inside a single file

Hi,

If you mean keeping the capture to one instance of the Data Array, I agree. Apologies, I was thinking through and editing my previous post then had to take my daughter to school so it reads quite differently now :). I have used the Data Array to capture large arrays (over 220 items), but not at the read/write speed that would be required for capturing the animation of an OSC skeleton.

best wishes

bonemap

bonemap

@juriaan said:

'x=20.05' then '20.05' and work with 'line' numbers instead

Hi,

You might be right about encoding the data written to the Data Array in the way you have described. The Kinect OSC skeleton data has many xyz nodes such as /head, /right_elbow which identify each multi value stream (the new OSC multi listener actor allows those labels to be displayed).

Conceivably, each line captured to the Data Array would include the 200+ values representing the joints of the skeleton armature. The line numbers would then represent the progression of movement of each value.

Is this how you are thinking?

Best Wishes,

bonemap

mark_m

@bonemap @Juriaan
I'm not so up on all this OSC Data stuff, but I did a session at the last Werkstatt with @DusX showing us how he gets JSON data from the Kinect via his wee tool. Sorry, that didn't come out right... By using his Kinect2share software, which is here:
https://github.com/rwebber/kin...
Might be worth checking that out as part of this exploration.
Or maybe this isn't what you're looking at at all, and I'm completely showing my ignorance! In which case, ignore me.... NEXT!

bonemap

@mark_m said:

Kinect2share software

Thanks for that advice.

The challenge we are discussing is how to record the data using Isadora. So it is about what can be done with the OSC data stream once it is received into Isadora. I am using Ni-Mate on Mac OSX to get the OSC skeleton tracking streams into Isadora. It would be great to have an efficient way to record the tracking data for later playback. I can currently get two unique kinect skeletons into Isadora with my MBP, but I am uncertain if it will be possible to effectively record the OSC data using Isadora.

The paid license version of NiMate allows recording of kinect skeleton tracking sessions and these recordings can then be played back through NiMate and converted to OSC for Isadora. If there is an effective way to record the tracking session in Isadora then the need for a NiMate license can be removed .

Best Wishes,

bonemap

rjravitz

Thank you for all of the advice! The example .izz file was really helpful and I will try and I will apply this to the motion data I am getting from the Kinect. I will make sure to keep everyone posted.

mark

@bonemap said:

My first thought, is to consider if the number of items required (for a skeleton it is going to be over 240 continuously changing OSC values) and what a single Data Array can handle to keep the system viable.

I would say that the Data Array actor's internal working are relatively lightweight. While it's true that pushing out 240 outputs is going to be somewhat costly in terms of CPU, it's really no different (and more CPU intensive) than reading the OSC data in the first place. (In fact, it might be lighter.) So I would say this is a totally viable solution. The heaviest part of the patch will be connecting all those wires. ;-)

Best Wishes,
Mark