Multiple NDI Streams with Alpha!

[dnichols]

Greetings! I have been a long time user of Isadora (since 2008) and I am currently working on a virtual production during the times of covid.

We are using keyed NDI feeds piped into Isadora and I am saving resources by doing the chromakey and crop before it reaches Isadora. Although I can see the alpha channel premultiplied in the stream out of the vid-gpu port, I am still having to apply the alpha tool to get the correct feed. Although I do nothing with the alpha tool, no setting changes on this actor at all. 

The issue I'm worried about is I am taking 7 (seven) ndi feeds at once, and by passing through the alpha tool there is a considerable load appearing on the machine (in the 85%+ range). I would love for there to be a way for the alpha coming out of the NDI v4 feed to be already be usable without the alpha tool. 

I can build the show and the system seems to be able to handle the load, but this feels like a bug perhaps.

Also, to maintain connection to the feeds, I have placed them in their own (Active at all times) scene, and I am broadcasting them to listener actors in appropriate scenes as necessary. I haven't seen this be a problem performance wise, but I am open to suggestion if this is terrible idea.

If needed I can post a patch, but just wanted to get some insight from this wonderful hivemind. I appreciate any assistance!

Thank you!

[DusX]

@dnichols said:

this feels like a bug perhaps

 It might be. If the thumbnail shows it as correctly and you can't use it without first setting the alpha flag (Aplha Tool) that seems wrong.
What are you using to create the NDI stream with Alpha?

[dnichols]

@dusx Thank you for your reply. Currently the alpha stream is coming from OBS windows 26.0.2 version 64 bit version.

When I hover over the isadora feed lines the alpha looks correct, but the output without the Alpha tool has a subtle gray that appears.

Cheers,

[Woland]

@dnichols said:

Also, to maintain connection to the feeds, I have placed them in their own (Active at all times) scene, and I am broadcasting them to listener actors in appropriate scenes as necessary. I haven't seen this be a problem performance wise, but I am open to suggestion if this is terrible idea.

This is what I would recommend doing actually. It's exactly what I did for a show two weeks ago. Background Scenes are a brilliant way to keep your feeds active at all times. It also means if you need to see your actors even when they're not "onstage" to make sure they're in position before taking a specific cue, you can keep a visual feed of them going at all times using a Control Panel with a Monitor Control for each performer.

[Woland]

@dnichols said:

We are using keyed NDI feeds piped into Isadora and I am saving resources by doing the chromakey and crop before it reaches Isadora.

This sentence has me wondering if some part of this is the cause of the NDI alpha weirdness.

If you use Isadora to send out a Stage via NDI with an alpha channel (say just a Shapes actor spinning by itself output by checking the NDI output box for that Stage in Stage Setup) and then bring it back into Isadora via the NDI Watcher actor, do you have the same problem with needing the alpha tool and the weird greyness? If so, there's definitely a bug in Isadora. If not, that'd point to your keying and cropping pre-Isadora process as a possible reason why the alpha is misbehaving. 

Also try sending that NDI feed to a couple of other pieces of software that can accept NDI and see how it appears and how the alpha behaves there. If it works without issue with a couple of other NDI-capable applications, it's likely an Isadora bug. If it does the same thing in other applications, then again that'd point to your pre-Isadora key & crop process as possibly being the culprit.

Best wishes,

Woland

[Kathmandale]

@woland @dnichols That's the approach I've always found best. A 'set-up' scene for ingesting the feeds, whether from NDI or Screen Capture, and applying universal changes like crops, keys, covering-up-of-logos(ahem) can all be contained there. The only variation I tend to use is sending each 'camera' stream to a virtual stage rather than using broadcast/listener. I don't know if there's a knock on to performance, doesn't seem to be,and it gives you all the benefits of virtual stages and just feels somehow neater to me. 

[DusX]

@dnichols said:

the output without the Alpha tool has a subtle gray that appears

 This sounds to me like it might be the difference between pre-multiplied and straight alpha channels.
What are the 'Alpha Tool' inputs set to when it is working for you? It may simply need to have a flag set, and Alpha Tool is the way to do this.
Your high Load, must be coming from somewhere else. (NDI isn't lightweight on the CPU).
If you can, I would suggest sending video via Spout/Syphon when at all possible, it is by far the lightest on the CPU (although it is limited to one machine).

[dnichols]

@woland Thank you for your response. I will test and see how it operates in other software and report back. Im doing the keying and cropping in OBS to try and simplify the work Isadora has to do.

[dnichols]

@dusxI I have actually done just that now. I did test sending directly from my system over a hardline into a capture card for the streaming server. I then tested setting my stage output to spout and using Spout to NDI to send the feed via network to the same streaming server. The Spout to NDI handled things the best just slightly. Which I found surprising.