[ANSWERED] Recognizing footsteps with Open NI Tracker

[inst]

Hi,

I tried the examples from the great Isadora Guru13 session "tracking with openni tracker".
Now I want to track the foot movements of a person who is always standing on the same place. I am interested if the person makes a step forward with one foot.

With the OpenNi Tracker and the Skeleton Decoder I can read out the left and right foot with x,y,z. I thought I only needed the Z position - the back and forth movement of the foot. But that is not accurate enough.

Can someone give me a hint how to find out the movement of each foot more exactly. So when is the beginning and the end of the step? I would only need a message if one step is made.

Thanks a lot!

[DusX]

@inst

You may need to setup some more details tests to determine if a step is made. You will need to look at your available data to see if you can spot any consistencies between 'steps' data. using some form of Fuzzy Logic might help you determine if the data points match closely enough to your goal to call the motion a 'step'.
Another option would be to use Machine learning as the method of determining what a step looks like. A tool that you can use with Isadora to do this is http://www.wekinator.org. It will allow you to send in the data vai OSC. You can then record a number of sample steps, and have wekinator determine what they have in common. Then when watching your live input data, wekinator can send a OSC value to Isadora when it 'sees' a step.

[Armando]

@dusx @inst

Hello,

I Think there is a simpler way than wekinator using body movenment analysys knowledge. I cannot do a patch now so I'll go with pseudo code.

In order to identify what a step is we need to identify relationships between the chest and the feet because a step is done in order to  to displace the body (we could stick to the torso and/or head) in space but at the same time there is at least one foot that has to stay in place  for a certain time WHILE the torso should have a translation on the x and on z plane.

So, I'd say we need some javascript to analyze and count cycles that could be expressed like that

if right OR left foot x and x positions stays the same (or move very little) for t time (t to be determined between 3/10 of a second and 1 second I'd say) while torso or neck move in x and z plane we could consider that a step.

Does it make sense ?

Cheers

Armando

[inst]

@DusX @Armando 

Hi,
thanks for the feedback!
Well, it's an installation, so there will always be other people with other body data. But yes, there has to be a change between torso and foot.
The gesture of walking is basic for all people . One thing I didn't mention is that the visitors walk on a treadmill - so they are always in about the same place but they walk. To cut a long story short - I would like to synchronise the stored foley sound of footsteps with the real movement of the visitors footsteps.

If I understand you correctly, you would take a start sample of the start position and then compare that to a next value t+3/10sec.

Unfortunately I don't have any programming experience with javascript to solve this. Would this also work with the modules from Isadora?

1) Capture foot and torso
2) Save foot position 1
3) Save torso position 1
4) Compare - has foot position 2 changed for t+3/10sec?
5) Compare - has torso position 2 changed to position 1?
6) Foot step is detected
7) Send OSC signal (integer)

best,
fränk

[jfg]

@inst

sorry my remark is perhaps stupid because it is difficult to exactly visualise your installation. But could you have the Kinect not in front but on the side of the walking person?

another idea is to use IR proximity sensor or laser sensor to capture th position of the leg.

best

Jean-François

[Armando]

@inst @DusX

hi, ok I understand better. The treadmill thing is more tricky because this means that the torso z plane won't move. But there might be another way.

One more body movement analysis thing that might help is that while a step is done there is a so called transference o weight. This means that that the center of the two hips moves on the x plane roughly above the foot that is pushing WHILE the other foot is on the air.

Other way is to se that the foot on the air is moving on the z plane in a negative value (goes closer to the camera) while the foot that pushes move in a positive value (in a direction far from the camera. In this case, in pseudo code this would mean

Check for the foot Z AND Y positions : look for the direction inversion of each foot and trigger the sound when a foot z values start to invert. 

Beware that the 30 fps of the camera will introduce a little delay between the calculation of the inversion and the sound triggering.  

I hope I am clear.

All the best