• Products
    • Isadora
    • Get It
    • ADD-ONS
    • IzzyCast
    • Get It
  • Forum
  • Help
  • Werkstatt
  • Newsletter
  • Impressum
  • Dsgvo
  • Press
  • Isadora
  • Get It
  • ADD-ONS
  • IzzyCast
  • Get It
  • Press
  • Dsgvo
  • Impressum

Navigation

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Popular
    • Tags

    Guru Session #13: Body Tracking with Depth Cameras (Fri May 8th, 6pm CEST/5pm GMT/12pm EDT/9am PDT)

    Events
    guru-sessions body tracking tutorials kinect tracking
    20
    123
    64004
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jfg
      jfg @Demetri79 last edited by

      @demetri79 said:

      on my tower

       can you tell us which tower? it is the same as mine: Mac Pro 2012 (5.1) USB 2.0 native+ Inatek PCIe USB 3.0 Card.

      best

      Jean-François

      • Izzy 3.2.6
      - MacBook Pro M1 Max 16" 64GB RAM, Mac OS 15.3.2 Sequoia
      - Mac Pro 5.1 middle 2012 (3,33 GHz 6-Core Intel Xeon, 32GB RAM, Radeon RX 580 8 GB ),
      Mac OS 10.14.6 (Mojave)
      - Mac mini Pro M4, Mac OS 15.3.2 Sequoia

      • A range of deployable older Macs
        Located in Bremen, Germany
      1 Reply Last reply Reply Quote 0
      • mark
        mark last edited by

        Dear All,

        v0.9.6 of OpenNI Tracker is available in the "alpha builds" forum thread.

        The record feature is now enabled. It seems to work well on both Mac and Windows now. I'm sure this feature will be a boon for development, since you can now record and play the depth streams and have the tracking work as if there were a person in front of the camera. Let me know how it works for you.

        @jfg This version may solve your problem with the Asus Xtion. Try it and let me know.

        Best Wishes,
        Mark

        Media Artist & Creator of Isadora
        Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

        mark_m jfg 2 Replies Last reply Reply Quote 1
        • DanielPineiro
          DanielPineiro last edited by

          Hi!

          The record feature works fine for me! That´s great. Thanks Mark!!!!

          MacBook Pro Retina (2013) 2,4 GHz Intel Core i5, 8 GB 1600 MHz DDR3, Intel Iris 1536 MB.
          Big Sur 11.7.6
          Located in Pontevedra, Spain.

          1 Reply Last reply Reply Quote 0
          • mark_m
            mark_m @mark last edited by mark_m

            @mark said:

            v0.9.6 of OpenNI Tracker is available in the "alpha builds" forum thread.

            The Windows link doesn't seem to be working, but it's available at:
            https://troikatronix.com/files...

            Cheers

            Mark

            Intel NUC8i7HVK Hades Canyon VR Gaming NUC, i7-8809G w/ Radeon RX Vega M GH 4GB Graphics, 32GB RAM, 2 x NVMe SSD
            Gigabyte Aero 15 OLED XD. Intel Core i7-11800H, NVidia RTX3070, 32GB RAM 2 x NVMe SSD
            PC Specialist Desktop: i9-14900K, RTX4070Ti, 64GB RAM, Win11Pro
            www.natalieinsideout.com

            mark 1 Reply Last reply Reply Quote 0
            • jfg
              jfg @mark last edited by jfg

              @mark

              Dear Mark,

              I have to apologise because I gave you yesterday wrong information. I tested yet all different configurations new and the result is:

              - 0.9.2 works fine on both computers (I have to use the masquerade file)

              -0.9.5 and 6 won't recognise bodies on both (I haven't forget to put the masquerade file at the right place)

              on the Macbook with USB 3.0 the Xtion is recognised.

              on the MacPro (2012) the Xtion is only recognised over USB 2.0. If I connect it on the USB 3.0 Inateck PCIe card i get only device init error

              The color video output give me only a frozen picture on the macBook (usb 3.0).(the first picture after switching). Work ± on MacPro (Jitter)

              thank you for your help

              best

              Jean-François

              • Izzy 3.2.6
              - MacBook Pro M1 Max 16" 64GB RAM, Mac OS 15.3.2 Sequoia
              - Mac Pro 5.1 middle 2012 (3,33 GHz 6-Core Intel Xeon, 32GB RAM, Radeon RX 580 8 GB ),
              Mac OS 10.14.6 (Mojave)
              - Mac mini Pro M4, Mac OS 15.3.2 Sequoia

              • A range of deployable older Macs
                Located in Bremen, Germany
              1 Reply Last reply Reply Quote 0
              • JJHP3
                JJHP3 last edited by

                Just sayin' the new openNI plugs are TERRIFIC! They work perfectly with my aging MBP laptop and kinect 1414. I used the kinect for a performance/installation back in 2012 and it was a pain... it then went on the shelf. I used DeliCode program in get the info into Isadora and then it was a bit of a crap shoot whether it would work or not. Congratulations to Mark and the team on getting this wonderful re-addition to Isadora up and running. The tutorial and the guru session were both most useful. Bravo ~

                John

                MBP'21- M1-Max; Sonoma 14.7; Isadora: 4.0.7 (ARM) ---- artwork: terragizmo.net

                1 Reply Last reply Reply Quote 0
                • D
                  DillTheKraut @mark last edited by

                  @mark

                  Hello and congratulations to this amazing peace of plugin. Same as @JJHP3, for the Guru Session I freed my 1414 from the years old dust and it ran on my MacBook in no Time!
                  I'm especially impressed by how fast the body recognition reacts and the accuracy of the skeleton tracking.

                  But I do have an issue on a windows pc. With the OpenNI plugin in the plugin folder, I get the following error:


                  This is true with Tracker plugin ver. 0.9.6 and 0.9.2

                  Isadora 3.0.7
                  Windows 10 Pro 1909 1836.592
                  AMD A-5000
                  4GB RAM

                  Best
                  Dill

                  D mark 2 Replies Last reply Reply Quote 0
                  • D
                    DillTheKraut @DillTheKraut last edited by DillTheKraut

                    @dillthekraut

                    I could solve the starting error by installing the OpenNI drivers from structur.io. https://structure.io/openni

                    But I do have a general issue now, as the pc isn't recognizing the kinect right. In the windows device manager list, only the XBOX NUI Monitor is appearing under other devices, without a proper driver associated.
                    At a Parallel Windows VM, it shows additional XBOX NUI Audio and XBOX NUI Camera. Which is missing on the pc. The actor in here keeps saying 'resolving uri'.
                    Does anyone have an idea?


                    Dill

                    D 1 Reply Last reply Reply Quote 1
                    • Juriaan
                      Juriaan Tech Staff last edited by

                      Hi there, 

                      A few things that I noticed

                      1) Kinect v2 (Xbox One) is still crashing with the current version that was provided to us today. Sadly Isadora crashes so badly that I can't send the Crash logs to you since it doesnt create one with the normal workflow (So that you can send it from the Error prompt that pop-ups)

                      2) Load becomes massive like 70.6% on a Windows 10, MSI GE73VR 7RF Raider laptop (2000 euro gaming laptop with a Nvidia dedicated graphic card with just the Kinect v2 / OpenNi Tracker and a Projector attached to it's depth image in full screen mode)

                      3) When I say against Isadora that I only wish to receive the color map it still sends out the depth video signal as well.

                      4) I can't get the recording to work properly. What I did was I set the File input to tracking.oni and told the plugin to Record, do a silly dance, and press 'stop'. Then I try to play it back by pressing 'Play' and it doesn't show anything. It stops the active sensor flow and showcases a blank screen. When I'm looking in my file explorer or it even made a file called tracking.oni it doesn't show up.

                      Isadora 3.1.1, Dell XPS 17 9710, Windows 10
                      Interactive Performance Designer, Freelance Artist, Scenographer, Lighting Designer, TroikaTronix Community moderator
                      Always in for chatting about interaction in space / performance design. Drop me an email at hello@juriaan.me

                      mark 1 Reply Last reply Reply Quote 0
                      • D
                        DillTheKraut @DillTheKraut last edited by

                        @dillthekraut

                        Could solve it by removing the OpenNI drivers and installing the official Microsoft Kinect SDK.
                        Works perfectly now on the pc.

                        1 Reply Last reply Reply Quote 0
                        • D
                          Demetri79 last edited by Demetri79

                          @jfg 

                          Hardware: Mac Pro 2012 5,1 PCIe USB 3
                          MacOS Version: High Sierra 10.13.6
                          Depth Camera: Kinect v1 (Model 1414)
                          Isadora Version: 3.0.7

                          @mark 

                          I dont have any homebrew installed. it came back with "-bash: brew: command not found"

                          So after much more testing I have more insight to my problem at least:

                          I tested all 3 versions now (0.9.2,0.9.5,0.9.6) , over 100 times shutting down my computer and following the suggested process. I think the problem for me is something related to the USB ports.. USB communication or USB power. I have tried all 9 (5 usb3 via pcie and 5 USB2 via 3 rear and 2 front). The USB3 ports on my PCIe card had better results working to provide power most of the time. i.e. sometimes the blinking green light on the kinect would stop blinking after a few blinks when connected to a USB2 port. This did happen a couple times on the USB3 ports as well though. 

                           This led me to believe maybe an SMC reset might fix the problem which seems to be either USB communication or USB power. BUT I HAVE NO PROBLEMS WITH ANY OTHER USB DEVICE EVEN THOSE DRAWING POWER.. THEY ALL WORK PERFECTLY??

                          I performed an SMC reset (remove power cable from mac for 15 sec) and booted up and the Kinect was finally recognized for the first time since it stopped working thursday. But soon as I got the skeleton decoder and projector connected the skeleton data froze and the depth image froze. Deleting the Open NI tracker and placing in a new one did not work. The only way to get the kinect to be recognized by the Open NI is doing the whole process over again.. shutdown - SMC reset - boot up - place Open NI- kinect recognized - then freezes again within 5 sec.. even this process does not get the kinect recognized every time.. only about a about 25% of the time. I cannot get the kinect working properly like it was earlier this week.. it was working perfectly every time i connected it and never crashed once even with 4hour+ sessions.

                           All 3 versions of openNI have been tested over 30 times each .. I only got the kinect even recognized about 4 times total.. and the times it did get recognized it froze within seconds.. but the green light did not stop blinking when the data and image froze. I'm so bumed and confused

                          #1-2024 Corsair Vengence a8100 Ryzen 9 7950X3D, Nvidia RTX 4090, Windows 11, Isadora 3.2.6
                          #2 MAC PRO 2012 5,1 12 CORE XEON 3.46GHZ, 64G RAM
                          RADEON SAPPHIRE VEGA 64, pixelas mod
                          PCIE USB3, PCIE ESATA, PCIE NVME M2 SSD
                          MAC OS 10.14.6 (17G10021), ISADORA 3.2.6, ISADORA 2.6.2

                          mark 1 Reply Last reply Reply Quote 0
                          • mark
                            mark @mark_m last edited by

                            @mark_m said:

                            The Windows link doesn't seem to be working.Ooops. Sorry about that. I fixed the link in the alpha builds thread. Thanks for the heads up.

                            Best Wishes,
                            Mark

                            Media Artist & Creator of Isadora
                            Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                            1 Reply Last reply Reply Quote 0
                            • mark
                              mark @DillTheKraut last edited by mark

                              @dillthekraut said:

                              [EDIT: I see you figured this out and it worked. Glad to know that this installing the SDK solves the problem.]

                              But I do have an issue on a windows pc. With the OpenNI plugin in the plugin folder, I get the following error:

                              There was an omission in the original installation instructions. Please try the following:

                              1) Make a backup copy of the OpenNI Tracker.izzyplug folder
                              2) Delete the entire plugin OpenNI Tracker.izzyplug
                              3) Reinstall v0.9.6
                              4) But also also install the Kinect SDK v2.0. This is required to use the Kinect drivers on windows.

                              I'd like to know if that procedure gives you a working plugin. While the old drivers you downloaded may work, they do not support changes I have made in those drivers improve how they work. (Specifically, if you try the new record function with those old drivers, I'm pretty sure Isadora will crash as soon as you try to play the ONI file.)

                              Best Wishes,
                              Mark

                              Media Artist & Creator of Isadora
                              Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                              1 Reply Last reply Reply Quote 0
                              • mark
                                mark @Juriaan last edited by

                                @juriaan said:

                                A few things that I noticed

                                I'll go through your list point by point

                                1) Kinect v2 (Xbox One) is still crashing with the current version that was provided to us today. Sadly Isadora crashes so badly that I can't send the Crash logs to you since it doesnt create one with the normal workflow (So that you can send it from the Error prompt that pop-ups)

                                Yeah, I don't really expect it to work well yet. It was an accident that it went out with the 0.9.5 build, but I figured at this point I'll just put it in and let people try. I think people on MacOS will have a better result since the entire driver was hacked/created from the ground up. They will have to deal with 512x424 image being embedded in a 640x480 frame right now.

                                2) Load becomes massive like 70.6% on a Windows 10, MSI GE73VR 7RF Raider laptop (2000 euro gaming laptop with a Nvidia dedicated graphic card with just the Kinect v2 / OpenNi Tracker and a Projector attached to it's depth image in full screen mode)

                                It's unclear to me if you're talking about a Kinect v1 or Kinect v2 here. I guess it's Kinect v1 since v2 is crashing, but please be specific. That said, loads that high are not my experience. I'll edit this post in a few minutes with my load figures from Windows 10. Keep in mind that the GPU is going have like zero impact on the efficiency of this plugin. The data from the camera comes in as CPU bitmaps, and everything is thus handled on the CPU not the GPU.

                                3) When I say against Isadora that I only wish to receive the color map it still sends out the depth video signal as well.

                                I think that "against Isadora" must be a typo... I guess what you're saying is that if you attempt to get the RGB camera only, you cannot. Yes??

                                4) I can't get the recording to work properly. What I did was I set the File input to tracking.oni and told the plugin to Record, do a silly dance, and press 'stop'. Then I try to play it back by pressing 'Play' and it doesn't show anything. It stops the active sensor flow and showcases a blank screen. When I'm looking in my file explorer or it even made a file called tracking.oni it doesn't show up.

                                Did you save the file? Unless you specify a full path name (e.g., C:\Users\USERFOLDER\Documents) the plugin creates a path name based on the path of the Isadora .izz file. If you haven't saved the file, maybe this fails?

                                Best Wishes,
                                Mark

                                Media Artist & Creator of Isadora
                                Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                                Juriaan 1 Reply Last reply Reply Quote 0
                                • mark
                                  mark @Demetri79 last edited by

                                  @demetri79 said:

                                  I'm so bumed and confused

                                  Well, I am not sure what to say to this. I mean, it seems like you had everything working for some time, and now it doesn't work. If you reboot the computer, there is no way that any data or code from the tracker plugin is lingering inside of your computer's memory. So I really don't think it's the OpenNI Tracker itself. Furthermore, you indicate the previous version (0.9.2), which was working well previouslyl, no longer works.

                                  We need some way to prove the hardware itself is still working, because given the description of the frustrating process you describe, and the fact that the green light stops blinking, it is possible that your Kinect has failed.

                                  Do you have some other software you could use to try to use that Kinect, e..g, the demo or release version of NI Mate perhaps? Or else the free Shapify.me from the App Store? (This successfully worked with my Kinect 1414, including moving it up and down using the internal motors. It never succeeded in scanning me, but I could see the image.)

                                  Best Wishes,
                                  Mark

                                  Media Artist & Creator of Isadora
                                  Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                                  1 Reply Last reply Reply Quote 0
                                  • mark
                                    mark last edited by mark

                                    @Demetri79 and everyone else:

                                    I've put together a simplified Kinect test app for MacOS using the examples from the open-source original OpenNI distribution.Inside this download you'll find two scripts: run-closest-point-viewer.command and run-simple-viewer.command. The first runs the "ClosestPointViewer" test app, the second runs the "SimpleViewer" test app.

                                    Could people running High Sierra try running run-simple-viewer.command and verify it shows a depth map image? I need this to kind of prove that it is a good, universal test to show a working connection to the Kinect without any of my modified drivers in the loop.

                                    These test apps work for me on High Sierra 10.13.6 (17G13021)

                                    Note that these scripts and apps are not code signed. To run them under High Sierra, right click the script and choose "Open". You'll get a dialog like this:



                                     Click the "Open" button to run the script. If everything is working, you should see this:



                                    Note that it might take two attempts before it works. If you don't see an image the first time or the script otherwise fails, try it once more.

                                    Diagnostic information will print in the Terminal window that opens. If this fails, posting the the info in the Terminal window might help me to debug your setup.

                                    kinect-test-macos.zip

                                    Best Wishes,
                                    Mark

                                    Media Artist & Creator of Isadora
                                    Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                                    1 Reply Last reply Reply Quote 0
                                    • mark
                                      mark last edited by mark

                                      Dear All,

                                      As you can see, I'm trying to address the problems people have identified here. But, to give me a better picture of how this actor is working out, it would be good to hear from people for whom it is working too.

                                      Please be specific about what sensor your using, and the operating system with version if it's not in your signature line. I assume you're all working with Isadora 3.0.7 or the current beta if you are a beta tester.

                                      Best Wishes,
                                      Mark

                                      Media Artist & Creator of Isadora
                                      Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                                      D 1 Reply Last reply Reply Quote 0
                                      • Juriaan
                                        Juriaan Tech Staff @mark last edited by

                                        @mark

                                        "Yeah, I don't really expect it to work well yet. It was an accident that it went out with the 0.9.5 build, but I figured at this point I'll just put it in and let people try. I think people on MacOS will have a better result since the entire driver was hacked/created from the ground up. They will have to deal with 512x424 image being embedded in a 640x480 frame right now."

                                        Yep understood :) Still wanted to let you know. I sent you the crash reports earlier that basically have the same behavior. Isadora starts just fine with the OpenNI actor, after a while (5 minutes or so) the OpenNI actor crashes Isadora. After that the time duration that Izzy crashes after a restart is like 1.5 minutes or so 

                                        "It's unclear to me if you're talking about a Kinect v1 or Kinect v2 here. I guess it's Kinect v1 since v2 is crashing, but please be specific. That said, loads that high are not my experience. I'll edit this post in a few minutes with my load figures from Windows 10. Keep in mind that the GPU is going have like zero impact on the efficiency of this plugin. The data from the camera comes in as CPU bitmaps, and everything is thus handled on the CPU not the GPU."

                                        Isadora 3.0.7 (Stable), Windows 10. Kinect Sensor v2. I added a reg key to actually boost my Intel CPU to 30 frames for the Kinect. When I didn't do that I only got 15 frames causing the Kinect Verifier application to yell ;) On the Kinect SDK forums you will find a forum post describing how to disable CPU sleeping.

                                        "I think that "against Isadora" must be a typo... I guess what you're saying is that if you attempt to get the RGB camera only, you cannot. Yes??"

                                        When I use the 'capture mode' input of the plugin and set it to Color I also receive the depth image. What I expected to happen is that I only get the color image and that the depth image was disabled / not active (So a red line instead of a green line to the projector)

                                        Did you save the file? Unless you specify a full path name (e.g., C:\Users\USERFOLDER\Documents) the plugin creates a path name based on the path of the Isadora .izz file. If you haven't saved the file, maybe this fails?

                                        Nope, I didn't save the file. Will test that out later tonight when I have some spare time.

                                        Isadora 3.1.1, Dell XPS 17 9710, Windows 10
                                        Interactive Performance Designer, Freelance Artist, Scenographer, Lighting Designer, TroikaTronix Community moderator
                                        Always in for chatting about interaction in space / performance design. Drop me an email at hello@juriaan.me

                                        mark 3 Replies Last reply Reply Quote 0
                                        • mark
                                          mark @Juriaan last edited by mark

                                          @juriaan said:

                                          Isadora 3.0.7 (Stable), Windows 10. Kinect Sensor v2. I added a reg key to actually boost my Intel CPU to 30 frames for the Kinect. When I didn't do that I only got 15 frames causing the Kinect Verifier application to yell ;) On the Kinect SDK forums you will find a forum post describing how to disable CPU sleeping.

                                           Hmmmm I didn't need to do any of this. My LOAD was 21% with Windows 10 running on Bootcamp on my MacBook Pro. (I know that's not _really_ a PC, but it's pretty much a PC.)

                                          I'm investigating the Kinect v2 crash now. I just ran the Kinect v2 for 10+ minutes, dancing around all the while, and no crash. I'm now trying it again without the debugger running... it's been 7 minutes and all is well.

                                          @Juriaan : Did you need to be in the frame to get it to crash? Or could you just let it sit with no one in the frame? Had you added anything to your patch aside from the OpenNI Tracker and a Projector? (I suppose heavier load/stuff in the patch could change the picture.)

                                          Best Wishes,
                                          Mark

                                          Media Artist & Creator of Isadora
                                          Macintosh SE-30, 32 Mb RAM, MacOS 7.6, Dual Floppy Drives

                                          1 Reply Last reply Reply Quote 0
                                          • JJHP3
                                            JJHP3 last edited by

                                            Hi Mark

                                            I tested the new files you made available. The simple player window did not open. Report included. Note: this is the same laptop (10.13.6) / 1414 that works so well with the new openNI plugins. Hope this helps you. John

                                            MBP'21- M1-Max; Sonoma 14.7; Isadora: 4.0.7 (ARM) ---- artwork: terragizmo.net

                                            JJHP3 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post