Intel RealSense x OpenNI - Flashing Issues

andielloyd

Hello!

Looking for some help again!

I'm working on a large lighting design project using the Intel RealSense to translate tracking/shapes/videos into intensity data for a grid of 25 incandescent lights.

I'm super new to using the RealSense and have only done it with this project, and in all of our research residencies I found it easy to deal with the settings (I'm only messing around with these parameters:)

This new venue is a little different than where we did our research, and I'm experiencing difficulty getting the camera sensitive enough to light my performers when they are in range, but when I get the lights to an appropriate level, there is a lot of super distracting flashing that occurs. [Video Example]

Would love to connect with someone who has experience working with this camera!

Thanks SO much in advance :)

Juriaan

@andielloyd

Hi there!

Sadly I can't open the link that you included to showcase the flickering. Any chance that you could reply with the link so that we can see what is going on?

I'm also going to tag some people here :) I personally don't own such a camera, but I know my colleagues do!

@DusX @Woland

Woland

@juriaan said:

I personally don't own such a camera, but I know my colleagues do!

Unfortunately, I don't have an Intel RealSense either.

andielloyd

@juriaan

I uploaded it! Here's the link: https://peertube.stream/w/mT8c...

DusX

@andielloyd

That looks like noise in the data being picked up by the camera/sensor. Perhaps it's more noticeable in this space due to lighting level changes (fixtures often are more sensitive to changes in certain ranges)
You may want to look at actors like Smoother, Seek Target Value, or Compare Guarded.

Compare Guarded can help you create 'safe' guarded range where you can stop values from fluctuating while your sensor is reporting a value within the range.
So if the sensor reports the subject is within range A - B, then set the desired lighting level, and when the subject moves out of the 'safe' range, allow the lighting value to change again.

How the lighting levels change might be made smoother (when transitioning into and out of the guarded 'safe' range) by using a Smoother actor, to minimize any jump in value.

mark

@andielloyd

Dear Andie,

@Juriaan asked me to take a look at this thread; I'm sorry I'm late to the party, I've just been swamped because of the 3.1 release.

I had a look at your patches, but they are pretty complex. What I see is that your taking in the Real Sense via the OpenNI Tracker, conditioning that image (scaling it, cropping it, etc.) and then broadcasting the result to numerous User Actors that -- I assume -- are each designed to control one light.

I also assume you are not attempting to do body tracking, but instead using the IR features of the camera to "see in the dark" as it were, to allow you to respond to the performer as they move across the stage.

Is the idea that when the performer is under the light, the light turns on?

In any case, without fully understanding your patch, one thing that concerns me is the fact that you seem to be using incandescent lights. All incandescent lights produce infra-red as well, and I wonder if the varying level of the lights is changing the way the image looks in the Real Sense, and thus introducing unwanted noise (as @DusX mentioned) into the whole setup. The thing that's tough is that it would be impossible for me to replicate this setup at home; one really needs those lights changing to see what's happening.

But what does the Real Sense image look like when those lights are changing? Do you see the brightness go up and down in ways that might disturb your algorithm? Thing is, if the lights themselves are messing up the Real Sense image, then it's going to be difficult to get things working.

Anyway... let me know if these ideas trigger anything in you based on your real-life experience in the theater. I'll do my best to help if you answer the questions I've posed above.

Best Wishes,
Mark

andielloyd

@mark @DusX

Sorry for my delay! I had trouble uploading my example video on my hotel internet. I spent today's rehearsal trying out Ryan's ideas (without much luck, I don't really know where place those limiters to get them to stop this specific issue?)

Mark, thanks so much for the detailed reply. Exactly, light is meant to turn on when dancer is underneath it. Our original research with an IR camera failed due to the incandescent lights producing too much bounce, but the RealSense has mostly mitigated this, save for some video noise on the perimeters of the video, which drastically increases when fog is introduced to the space. Yes, those user actors in the main patch all correspond to one of the twenty five lights that make up our grid. They are all hung top down, as if each one was one pixel of a screen.

Anyway. here's an example with my narration, maybe this will illustrate the issue better? To be clear, it'll be okay the way it is, I'm just hoping I might be able to tidy it up further!

https://peertube.stream/w/2j5t...

Thanks again!

Aidan

Hi Andie,

Two comments:

1. Is your Open NI depth max (cm) truly above the floor level? You need to detect only the head and shoulders and avoid detecting the floor.

2. Surely blob-tracking would eliminate all this noise since you can control the range of blob size to track.

Regards,
Aidan

DusX

@andielloyd said:

I don't really know where place those limiters to get them to stop this specific issue?

If you can email me your Isadora file, I can take a quick look and suggest where in the data flow you might need the actors to filter the data.
I will PM you with my email.