<![CDATA[Kinect for background subtraction]]>Hi there,

I am trying to understand what the best possible way is to use Kinect instead of a taking live feed in with green screen. I've looked into processing and quartz composer with syphon, but I'm new to this and can't seem to make it work.
Any advice?
Thanks
]]>https://community.troikatronix.com/topic/2865/kinect-for-background-subtractionRSS for NodeMon, 18 May 2026 10:10:49 GMTTue, 20 Sep 2016 06:44:21 GMT60<![CDATA[Reply to Kinect for background subtraction on Thu, 12 Jan 2017 22:59:57 GMT]]>Hi!

thanks a lot for your help, I really appreciate it!
@fubbi, I will check it out, this was exactly what I wanted to do and the thing is that I'm very new to processing and have very little time ahead of me to make this work.
I've had the amazing contribution of DusX with his cool little app, he also played with the contrast internally so that you can get more detail. 
I still intuitively feel with my little experience that minimising the scan distance will help the kinect get the right information.
Worst case scenario, we're even considering having her move on a rotating stool, so that she's a bit detached from the floor.
Will let you know how this is developing. It's been quite a saga. 
But with all you guys helping out, it feels like a solution is close!
Thanks
]]>https://community.troikatronix.com/post/22550https://community.troikatronix.com/post/22550Thu, 12 Jan 2017 22:59:57 GMT<![CDATA[Reply to Kinect for background subtraction on Wed, 11 Jan 2017 15:23:02 GMT]]>I am pretty sure the depth threshold is independent of the openni or skeleton tracking that recognises humans and makes the mask. There are a few threads about this on the microsoft forums. You can get a better chance at tracking if you try to increase the contrast behind the person (like mat black background that also absorbs infra red light). The skeleton tracking that produces the mask uses all images together to find people.

Also.too obvious probably, but make sure that the orientation of the camera means that even lying down you appear to be standing te right way up in the image, the systems have a very hard time if you are sideways or upside down.
Fred
]]>https://community.troikatronix.com/post/22527https://community.troikatronix.com/post/22527Wed, 11 Jan 2017 15:23:02 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 04:15:47 GMT]]>Hi

This was a processing app I made a few years back. It created a black and white mask from the kinect depth mask and sent it out via syphon. I have not opened it for a while, but it worked fine and maybe there is something useful in there.
@[eratatat](http://troikatronix.com/troikatronixforum/profile/6259/eratatat) there is a threshold value for the mask in the code attached so you can define the z depth start and end of the mask, I cant say if it is sensitive enough for somebody lying on floor. I remember mapping the threshold to osc, so I could tweak it from isadora, i.e move it back and forth.
hope it helps you
yours
fubbi

7630f9-trapping_v12.pde.zip

]]>https://community.troikatronix.com/post/22526https://community.troikatronix.com/post/22526Sun, 18 Jan 1970 04:15:47 GMT<![CDATA[Reply to Kinect for background subtraction on Wed, 11 Jan 2017 13:14:54 GMT]]>The app can be easily changed/added to, to supply the depth image. Perhaps using levels and then lumakey could provide something useful...

]]>https://community.troikatronix.com/post/22523https://community.troikatronix.com/post/22523Wed, 11 Jan 2017 13:14:54 GMT<![CDATA[Reply to Kinect for background subtraction on Wed, 11 Jan 2017 11:06:40 GMT]]>import spout.*;
import KinectPV2.*;
PImage img;
PGraphics canvas; 
PGraphics canvas2;
KinectPV2 kinect;
Spout spout;
Spout spout2;
int thresholdH = 1200; // max distance (in mm)
int thresholdL = 0; // min distance (in mm)
void setup() {
  size(640, 360, P3D);
  textureMode(NORMAL);
  kinect = new KinectPV2(this);
  kinect.enableDepthImg(true);
  kinect.enableBodyTrackImg(true);
  kinect.enableColorImg(true);
  kinect.init();
  canvas = createGraphics(1280, 720, P3D);
  canvas2 = createGraphics(1280, 720, P3D);
  img = loadImage("SpoutLogoMarble3.bmp");
    spout = new Spout(this);
      spout2 = new Spout(this);
  spout.createSender("rgb image");
  spout2.createSender("mask");
  
} 
void draw()  { 
background(0, 90, 100);
noStroke();
canvas.beginDraw();
  canvas.image(kinect.getColorImage(), 0, 0, 1920, 1080);
      canvas.endDraw(); 
      
          spout.sendTexture(canvas);
          
int [] rawData = kinect.getRawDepthData();
 //read kinect depth
  canvas2.beginDraw();
  canvas2.loadPixels();
  // draw the depth image in white between high and low limits
  for (int x = 0; x < kinect.depthWidth; x++) {
    for (int y = 0; y < kinect.depthHeight; y++) {
      int offset = x + y * kinect.depthWidth;
      int rawDepth = rawData[offset];
      int pix = x + y*img.width;
      if (rawDepth > thresholdL && rawDepth < thresholdH) {
        canvas2.pixels[pix] = color(255, 255, 255, 255); //draw white inside limits
      } else {
        canvas2.pixels[pix] = color(0, 0, 0, 0); //draw black outside limits
      }
    }
  }
  canvas2.updatePixels();
  canvas2.endDraw();
    spout2.sendTexture(canvas2);
    
}
]]>https://community.troikatronix.com/post/22521https://community.troikatronix.com/post/22521Wed, 11 Jan 2017 11:06:40 GMT<![CDATA[Reply to Kinect for background subtraction on Wed, 11 Jan 2017 10:24:10 GMT]]>Thanks guys.

First of all @DusX, your app is a great tool for simple background subtraction. 
Unfortunately, I'm doing something very specific, filming the person top down, and they're laying on the floor. 
It seems like the BodyTrackImage cannot work in this setting, as the person is very close to the floor.
Is there a way to set the kinect up so that it scans only a segment of the z axis? 
@[jhoepffner](http://troikatronix.com/troikatronixforum/profile/15/jhoepffner), I tried to implement your example using Lengeling's library and Spout, but can't seem to make it work. depthWidth cannot be resolved. 
Any ideas?
Thanks
ET
]]>https://community.troikatronix.com/post/22520https://community.troikatronix.com/post/22520Wed, 11 Jan 2017 10:24:10 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 09 Jan 2017 22:43:20 GMT]]>This is part of the Microsoft c++ redistributable... 

https://www.microsoft.com/en-ca/download/details.aspx?id=48145
]]>https://community.troikatronix.com/post/22489https://community.troikatronix.com/post/22489Mon, 09 Jan 2017 22:43:20 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 09 Jan 2017 19:38:52 GMT]]>Ran the app, it says it can't run because MSVCP140.dll is not installed

]]>https://community.troikatronix.com/post/22486https://community.troikatronix.com/post/22486Mon, 09 Jan 2017 19:38:52 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 09 Jan 2017 19:36:02 GMT]]>Ok found the release section. Sorry! don't really know how to use github.

]]>https://community.troikatronix.com/post/22485https://community.troikatronix.com/post/22485Mon, 09 Jan 2017 19:36:02 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 09 Jan 2017 19:32:49 GMT]]>@DusX
I'm sorry I can't find the release zip. I download the zip folder from github. Nothing there.
Is there a way to calibrate what bit we take from the depth data? I'd like to do quite a detailed background subtraction.
Many thanks

]]>https://community.troikatronix.com/post/22484https://community.troikatronix.com/post/22484Mon, 09 Jan 2017 19:32:49 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 04:12:58 GMT]]>@Eratatat Download the release (un zip it) And open the exe/application file. All required libraries are included. Note: both syphon and Spout support transfer of video with alpha. This image shows the alpha cutout video inside the openframeworks app and inside Isadora. https://camo.githubusercontent.com/bfeb9fa759cc5f984491d0daecd84cd878211521/68747470733a2f2f73636f6e74656e742d79797a312d312e78782e666263646e2e6e65742f762f74312e302d392f31343532303535355f3632383434313634373332363639395f323030393038363034333338343439333735375f6e2e706e673f6f683d3133626138383034353261333531393363386264666163646161643639363365266f653d3538414333354633

]]>https://community.troikatronix.com/post/22482https://community.troikatronix.com/post/22482Sun, 18 Jan 1970 04:12:58 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 08 Jan 2017 14:42:29 GMT]]>@eratatat

– first question: I am more accustomed to MacOs, but I use windows on Bootcamp and I tried Kinect on W and M, with Processing and Touch Designer. For what I see it's much better implemented in Windows and with Windows Processing Kinect Library, you can obtain directly a mask to make background replacement. But you have to struggle with Spout who is less easy to use that Syphon (DirectX dependent, Graphic card dependent…).
For my actual usage (https://vimeo.com/194632092) I use Processing and Isadora on MacOs with Syphon to pass image in both ways).
– To use my patch with other library, you have to change all what is related to the library and I dont know how to obtain "getRawDepth"
– For the second question, if you use Windows, you dont use Syphon but Spout… Concerning Syphon, if you declare your background in Processing with alpha mask (as color(0, 0, 0, 0), you send alpha with Syphon but I dont know about Resolume and Spout…
In Millumin you have a quite easy way to obtain alpha mask from Kinect, give a try to the beta release.
Jacques
]]>https://community.troikatronix.com/post/22476https://community.troikatronix.com/post/22476Sun, 08 Jan 2017 14:42:29 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 08 Jan 2017 10:53:27 GMT]]>@DusX thanks for sharing. As I'm totally new to windows, I'll ask you something a bit dumb, what do I have to do to run this app? 

]]>https://community.troikatronix.com/post/22475https://community.troikatronix.com/post/22475Sun, 08 Jan 2017 10:53:27 GMT<![CDATA[Reply to Kinect for background subtraction on Sat, 07 Jan 2017 17:17:57 GMT]]>@jhoepffner the problem I got with Shiffman's library does happen, I don't know why and how, this has developed into an unsolvable mystery.

https://github.com/shiffman/OpenKinect-for-Processing/issues/80
A couple of questions below, if you have the time, I would really appreciate your feedback.
I will be using windows instead with the other two libraries, can I ask you, will your processing patch work with the other processing libraries? 
Also, I will be compositing the final video on Resolume, so I was thinking of sending the image from Isadora to Resolume via Syphon. Is it possible to keep the alpha values intact when importing the syphon image to Resolume. 
A huge thanks for your help. 
]]>https://community.troikatronix.com/post/22472https://community.troikatronix.com/post/22472Sat, 07 Jan 2017 17:17:57 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 04:12:58 GMT]]>I have made a few updates to this app:
https://github.com/rwebber/kinect2share
It now outputs the Hand state (is the right or left hand open or closed) as well as a number of other small things.

The black and white mask (silhouette) is available via spout by default, as well, the 'green screen' type feed is there, incase you want live background subtraction.
]]>https://community.troikatronix.com/post/22398https://community.troikatronix.com/post/22398Sun, 18 Jan 1970 04:12:58 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 02 Jan 2017 16:12:58 GMT]]>Hello,

I you follow the procedure of libraries install (as described in Shiffman post, mainly freenect), it will work well.
some things to note
– you need a USB3 connection
– sometime you need to unplug/replug the kinect
– kinect V2 on mac gives you rawDeep info, color image, IR image but no skeleton infos.
I used it on a an installation one month ago, Processing to Isadora to Processing, no problem.
https://vimeo.com/194632092
I do not know of other working kinect Processing library on mac.
Millumin is able to have skeleton infos on mac but no way to output the info to Isadora…
Hope that helps
]]>https://community.troikatronix.com/post/22397https://community.troikatronix.com/post/22397Mon, 02 Jan 2017 16:12:58 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 02 Jan 2017 15:42:56 GMT]]>Hi there,

do you know if there is an alternative for MAC to Daniel Shiffman's library in processing?
While testing the file [@jhoepffner](http://troikatronix.com/troikatronixforum/profile/jhoepffner) I got an error, processing cannot find the device (kinect v.2). 
It seems like mac and kinect v.2 are not working well. 
Any suggestions how to solve this?
Thanks
]]>https://community.troikatronix.com/post/22396https://community.troikatronix.com/post/22396Mon, 02 Jan 2017 15:42:56 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 02 Jan 2017 15:20:48 GMT]]>Hi there,

do you know if there is an alternative for MAC to Daniel Shiffman's library in processing?
While testing the file [@jhoepffner](http://troikatronix.com/troikatronixforum/profile/jhoepffner) I got an error, processing cannot find the device (kinect v.2). 
It seems like mac and kinect v.2 are not working well. 
Any suggestions how to solve this?
Thanks
]]>https://community.troikatronix.com/post/22395https://community.troikatronix.com/post/22395Mon, 02 Jan 2017 15:20:48 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 24 Oct 2016 10:11:08 GMT]]>Thats ok. Just wanted to check.

]]>https://community.troikatronix.com/post/21730https://community.troikatronix.com/post/21730Mon, 24 Oct 2016 10:11:08 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 02:21:54 GMT]]>The 'release' zip has the app 'exe'. This is windows only.

The izzy sample file is in the code files... I need to update that... i forgot to add it to the release.
]]>https://community.troikatronix.com/post/21729https://community.troikatronix.com/post/21729Sun, 18 Jan 1970 02:21:54 GMT<![CDATA[Reply to Kinect for background subtraction on Mon, 24 Oct 2016 06:07:04 GMT]]>Thanks Ryan. I am going to ask the dumb question here, but when I download something from github it comes a zip. When I unzip it its always just lots of files and code like documents. Am I doing something wrong; or should this be an application ".app" ?

]]>https://community.troikatronix.com/post/21726https://community.troikatronix.com/post/21726Mon, 24 Oct 2016 06:07:04 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 02:21:54 GMT]]>If anyone would like to try/test this

https://github.com/rwebber/kinect2share
Please do and let me know how it goes.
]]>https://community.troikatronix.com/post/21725https://community.troikatronix.com/post/21725Sun, 18 Jan 1970 02:21:54 GMT<![CDATA[Reply to Kinect for background subtraction on Sat, 22 Oct 2016 23:26:19 GMT]]>Or perhaps, use a blend more directly in processing, I've tried it but can't seem to make it work. Sorry, really new to processing, so haven't figured out the logic yet.

thanks!
]]>https://community.troikatronix.com/post/21714https://community.troikatronix.com/post/21714Sat, 22 Oct 2016 23:26:19 GMT<![CDATA[Reply to Kinect for background subtraction on Sun, 18 Jan 1970 02:20:12 GMT]]>@jhoepffner p.s. got the kinect V2 now. 

]]>https://community.troikatronix.com/post/21713https://community.troikatronix.com/post/21713Sun, 18 Jan 1970 02:20:12 GMT