IR Camera/Hardware Question

[atrawickb]

Hello all,

While I understand that the Kinect is great for tracking people, I'm not interested in tracking people per se, but rather shadows, be they generated from the human form or otherwise. 

The new installation that I am building is situated in a 2,000 sq.ft. warehouse space with 15' ceilings.  

In the corner through which people will enter, I have a slide projector setup, advancing automatically to project empty slides diagonally across the space such that it generates a pattern of shadows on the opposing wall that is generated by a combination of the (kinetic) installation itself, in addition to whatever shadows visitors to the exhibition also introduce to the system.  (The set-up is not dissimilar to the description in Plato's, "Allegory of the Cave", if you're familiar with the text.)

From the other corner of the room, diagonally facing the same opposing wall, I want to be able to track the shadows as they are produced in real-time, and project pre-generated footage into only the shadows.  

This setup should be relatively easy in terms of writing a sketch in Isadora, so my questions here are only concerning hardware:

(please only respond with hardware advise that YOU HAVE ACTUALLY USED SUCCESSFULLY! -thanks)

1\. Are there any IR CCTV (specific model/make) cameras out there that have an adjustable focal length (to match the projector with which it is supposed to sync)?

2\. Which IR DV Camcorders (specific model/make) have you used successfully to track shadows in real time?  

3\. Are there any IR DSLR cameras (specific model/make) that you have used successfully to track shadows in real time?

3\. Are there any video cameras out there at all, that DO NOT require that the video signal be sent through a Blackmagic box (to switch codecs) prior to entering the Isadora interface?  (excluding the prohibitively expensive Blackmagic Cinema camera line.)

Thanks in advance for any and all advice,

  --TB

[Fred]

I use this

http://www.ebay.com/itm/281010620283?_trksid=p2060778.m2749.l2649&ssPageName=STRK%3AMEBIDX%3AIT

I bought a few different lenses for itm there are a lot available, I get them here

http://www.surveilzone.com/

I also use a midwest optics bandpass filter that blocks all visible light (the camera above has a moving IR blocking filter you can engage and disengage from the menu).

Getting a camera that does not need a capture interface will give you a lot more trouble, you are then looking at USB3 or Ethernet cameras. (the blackmagic camera does not plug directly into the computer the thunderbolt port is of data transfer). If you want to go this way, you will need an expensive machine vision camera that will cost a lot lot lot more and then you will have to write an Isadora plugin or other application to pass the video feed to Isadora as they use custom capture libraries). Gigevision is another potion, again expensive cameras and so far there is little support for capturing gigevision on OSX, and again Isadora cannot capture gigeVision. 

I am a little confused about what you mean by "DO NOT require that the video signal be sent through a Blackmagic box (to switch codecs)" Are you referring to a capture box? Video signals do not have any codecs to switch, they have a format.

The above camera can output SD in NTSC or PAL, 720p 60 720p50 1080p25 and 1080p30\. Using the blackmagic series of capture boxes with this device, the 1080p25/30 modes will not be available as Isadora uses an older capture library that does not expose these formats. You can get around this by using blacksyphon to capture and pass the signal to Isadora, or just use the camera in another mode, SD, or 720p. If you want super clear shadows it is worth getting the 1080p signal in.

I use these cameras a lot for tracking and they are super easy cheap and reliable and interchangeable lenses makes the very useful. The connect VIA SDI, so you have a great robust and pro connection without signal degradation.

WIth the cheapest blackmagic SDI capture box and the camera it will cost you about 300 for the setup with cables and a variable lens.

[ian]

And if you're looking for something on the cheap end that has a built in zoom / IR illumination I've had good luck w/ the lorex line of security cameras. Adding an IR bandpass filter will force the camera to night / IR mode even in daylight.

http://www.bhphotovideo.com/c/product/984977-REG/lorex_lbc7083_960h_weatherproof_night_vision.html

and if you just need a cheap IR sensitive camera for tracking purposes I often inexpensive board cameras, which come in a wide array of lenses. Here's a model you can force set to night (IR) mode. http://www.bhphotovideo.com/c/product/819136-REG/EverFocus_EM700P1_Ultra_720_TV_Indoor.html

With both there are versions w/ composite or SDI outputs as you require. Like all of @Fred's suggestions you'll need a  capture device.

good luck

[atrawickb]

Hello and thank you Ian and Fred for your great recommendations.  

Studio flooding had me tied up over the weekend.  All apologies to be a bit late responding to your comments.  

It seems that the first camera mentioned might be the best fit:

[http://www.ebay.com/itm/281010620283?_trksid=p2060778.m2749.l2649&ssPageName=STRK:MEBIDX:IT](http://www.ebay.com/itm/281010620283?_trksid=p2060778.m2749.l2649&ssPageName=STRK%3AMEBIDX%3AIT)

It captures 1920x1080 at 30fps which is an ideal match for the projector on the one hand, and Isadora on the other.

The second two cameras mentioned produce effective picture elements of 976x494\.  I'm not sure how that would map onto a projection matrix, and buying blind (without being able to actually test the cameras) I feel much more comfortable not having multiply technical hurdles in a new situation.  

As I'm basically new to wired externals (primarily working with EpocCam for live stuff), which BlackMagic SDI capture box would you recommend for this camera?  Would it be this BM UltraStudio SDI?

https://www.blackmagicdesign.com/products/ultrastudiousb3/techspecs/W-DLUS-01

Would you trust a used item on Ebay? http://www.ebay.com/itm/Blackmagic-Design-UltraStudio-SDI/251648787249?_trksid=p2047675.c100005.m1851&_trkparms=aid%3D222007%26algo%3DSIC.MBE%26ao%3D1%26asc%3D25879%26meid%3D1325107866ac4c4dbf4871096c544bcc%26pid%3D100005%26prg%3D10819%26rk%3D2%26rkt%3D6%26sd%3D360420141229&rt=nc

Thanks again for the support.

[Fred]

If you go with USB3 go to the blackmagic forums and check your USB3 chipset is supported. Most of the earlier controllers came in under spec and were not fast enough to capture HD. Are you on mac or PC? I would opt for thunderbolt every time if you have the capability. You can then use the ultrastudio mini recorder.

https://www.blackmagicdesign.com/products/ultrastudiothunderbolt

Fred

[atrawickb]

I'm working from a late-model 2013 MacBook Pro.  I was hoping to get away with not having to give up one of the two native thunderbolt ports (that I need for controlling projectors), given that the USB3 can now pull close to 5G per second, but you're probably right to not risk hamstringing the cameras' capabilities.  Thanks again Fred.  

[crystalhorizon]

I have a MBPro Mid 2012 and a Blackmagic Intensity Shuttle USB 3.0. Works great and very reliable.

[atrawickb]

Thanks for the input Feinsinn,

What camera(s) are you working with?  

All best,

[atrawickb]

@Fred

I got the CCTV camera and Blackmagic Ultrastudio Mini Recorder as you recommended.  Links to hardware here:

http://www.ebay.com.au/itm/HD-SDI-1080P-Full-HD-Box-Camera-2-Megapixel-1-3-CMOS-Image-sensor-CCTV-Security-/281010620283?ssPageName=ADME:X:AAQ:AU:1123

https://www.blackmagicdesign.com/products/ultrastudiothunderbolt

As you can see from the attached image files, the camera feed is coming up in the Blackmagic Media Express Player, but when I go to pull the feed into Isadora, the frame rate and pixel matrix combination that the camera is capturing in (1080p 30f/s) does not come up as an option, which is a bit strange as the system preferences in the Blackmagic Media Express player is set to that, AND Isadora is (as you can see) recognizing that a Blackmagic device is connected and prepared to transmit a signal.  

I don't understand what to try next, and neither did the Blackmagic support staff who I was just on the phone with for almost an hour trying to troubleshoot this.  Please advise.  

My next question will be, what will I have to do to get the camera to only see infrared light?!  

I was under the impression that that's how it would record straight out of the box.  As it is, it's capturing feedback loops from my projector which is EXACTLY what I'm trying to work around through this process. 

Thanks in advance for your time and attention.  All best,

[Fred]

Isadora uses an outdated capture system so it does not see the 1080p 30 setting. Using black syphon you will get the image into Isadora or by switching the camera to 720p will let you get the image into Isadora directly- there are many posts about this on the forum, it is an Isadora problem- later capture modules such as AV foundation or the native grabber on Windows have no trouble with this.

Yes the camera will see IR light, as it is it has an IR blocking filter in it, you can turn this off by going through the menu and finding the colour/B&W or night mode (the menu in the camera).

This will remove the IR blocking filter (you will here the solenoid pull the filter away from the sensor). However if you want IR only you need an IR bandpass filter (I have posted about this a few times on the forum, as with the other issues you are having).

You can either buy a high quality IR pass filter from mid west optics (or a distributor) or you can go for a DIY solution. 

Google DIY IR pass filter and you will see a lot of potions, basically using exposed film or the inside of a floppy disk. Depending on you situation you can get away with it.

IR is good for tracking- I am not sure what you are doing, some video projectos will emit IR light. The other way you can stop the camera seeing images from the projector is to use opposing polarising filters, one on the camera the other on the projector- IR is great for providing high contrast images from tracking without changing the light if your set, but I am not sure what you are trying to achieve.

Let me know what you are trying to achieve, much of the work in IR tracking is setting up the lighting, making sure you have no other IR emitting sources (other lights especially halogen but most incandescents also emit IR- you can use IR blocking filters on these lights). For tracking you are trying to get as much contrast between the background and the subject as possible, this can be either front or back lighting. The next job is to set up the tracker to ignore any artefacts you don't want (blobs that are too small or too big).

You will have to go through all the camera settings and turn off all the auto adjust settings- there are quite a few, then play with the shutter and gamma to get a good high contrast image- sometimes, only sometimes using a slightly out of focus setting will help. These lenses are default f 1.2, so focus with depth is difficult, you are better to set the aperture closed a little and add more light (change the gamma to avoid hot spots) to get focus near and far from the camera.

If you are using shadows, you will need to create the shadows with IR light- most of the details are already in this thread- have another look and take the time with lighting.

Fred

[mark]

Dear @Fred,

I am going to start working to address these Blackmagic problems for Isadora 2.0\. But the question I cannot seem to answer through my initial searches is this: does AVFoudnation actually allow you to adjust the various input settings? I see nothing in the API that supports this, but I am very happy to be proven wrong. Or is the only solution to directly access Blackmagic's proprietary API? 

I've also written Blackmagic seeking answers to these questions, but I figured you might have a quick answer for me.

If you know of any open source software that supports the needed features, all the better, as I can see how they approached the problem.

Thanks in advance for the tips,

Mark

[Fred]

I think gstreamer will do it, otherwise what settings exactly do you want?

Here is the avfoundation example

82db51-avrecorder.zip

[Fred]

Sorry, that is the avfoundation example, as for the Blackmagic API, what settings exactly would you want to change, it lets you select between any of the formats that the hardware is capable of- then you can get pixels-> textures from the input. Is there a need to go outside of the input settings? if you have this hardware and it works on your machine it is fine to get the actual format in right? It is faster than the quicktime capture and lets you change inputs (physical) and formats on the fly after initialisation.

Fred

[Fred]

@Mark Sorry, I thought you meant avfoundation, the blackmagic API lets you change formats (within the limitation of the standards that the capture card accepts) and even hardware inputs on the fly once the object is created. Is there a need to change formats- I take it you are referring to a codec? Is there a necessity to change codecs- if you get pixels they are always uncompressed right- and the upload to a texture is fast and then it is a texture.

Here is my fork of an OF addon to interface with blackmagic hardware. It is basic - I did not add all the functions, but it supports multiple devices- the formats are set to the available capture formats (in a lazy way, I put all possibilities, some are not available in some devices).

https://github.com/fred-dev/ofxBlackmagic

Avfoundation also supports the hardware and so does native windows capturing. Let me know if there is anything I can help you with, I am not sure I understood your question.

Fred

[atrawickb]

@Fred

The situation that I am trying to figure out is as follows:

I have constructed a large mobile sculpture (approx. 52 foot max. wingspan) which is anchored in an I-beam that carries the roof/ceiling for the building.  The mobile has a large white wall surface area at a certain distance behind it on which interesting shadows are easily generated by casting hard light on the mobile as it moves.  

Therefore the idea is to cast halogen (containing IR light) on it from one angle to generate shadows, which from another angle we will seek to capture those shadows live with a video camera that captures only IR light.  That video camera should then be lined up with a projector such that it is capturing the same field that the projector is throwing into.  At that point, employing the 'Add Alpha Channel Actor' in Isadora should make it relatively easy to capture the shadows clearly and proportionately with the camera and project directly into them (and only into the shadows) whatever footage we want.  

It is the hardware end of this situation that is proving difficult.  

1\.  As you noted, Isadora does not currently support bringing in a Blackmagic signal that is 1080p at 30 fps. 

2\.  While the camera you suggested does capture in 1080p, which matches the projector's resolution, said camera captures in such a wide-angle format that it is impossible to match the much more telescopic throw of the projector.  

3\.  I currently don't have (but now thanks to you know where to find) the right filter for the camera and in-camera menu adjustments to make.  

4\.  Are there any out-of-the box video cameras that are well-suited for IR tracking?  Teaching at a university, I have quick access to a number of models.  While I do have time to track down an external lens filter or something along those lines, a full-blown internal hardware hack is not possible with cameras that aren't mine (owned by the university).  

Thanks again for all of your help thus far and in advance for any and all further advice.  

  --TB

[mark]

Dear @Fred and other Video Capture Experts,

I've started a discussion thread about Video Capture for Isadora 2.0

http://troikatronix.com/troikatronixforum/discussion/1609/isadora-2-0-video-capture-expertise-and-opinions-needed

If you have expertise or opinions to offer, please chime in there.

Thanks,

Mark

[Fred]

Projector camera calibration is a tough task, getting camera and projector intrinsics and extrinsics requires a full implementation of open CV and a calibration routine.

To answer some of your questions, no DSLR is ready for IR, then need hardware modification. They are also notoriously bad at live capture and the lenses wont match- no matter what you think you see, you need to run a calibration grid to see where they are off, it is not just getting the same outer rectangle.

What you want to do is possible but I suggest doing your research, many of your questions were even answered in the above thread, take the time to read this and other threads in the forum.

For projector camera calibration you could start here- it is what you need to do to achieve the results you want. No lenses just match.

Here is some code and a link to a tutorial for what you want to do with the calibration

https://github.com/elliotwoods/artandcode.Camera-and-projector-calibration

As yet this is outside the possibilities of Isadora, but you could make some tools that are syphon enabled to let you use Isadora in this process.

Fred