Screen Capture - some performance notes

Kathmandale

Hi All,

I've been playing around with the screen capture a bit and thought I'd share some notes on behaviour/performance as I imagine there are plenty of people looking in to similar things at the moment.

My colleagues and I at imitating the dog are starting to put together something where we pull video in to Isadora from a zoom call. We'll probably have 4 or maybe 5 performers on the call. Now my preferred work flow is to have a background scene bringing in the different bits of 'captured' screen and sending them to virtual stages. In each scene I then pipe in the virtual stage as though it was a camera. I just find it easier to work that way, it keeps all the geometry associated with each 'camera' in one place, I can set the size/shape/res by adjusting the virtual stage etc etc. So, with that set up in mind I did some tests. I've written them up here with screen grabs showing the load in each case. (I was talking to myself on Zoom, there's only me there so the images don't make much sense, it was only a test where I was roughly quartering the zoom screen).

First I tried having 4x Screen Capture actors each grabbing a different portion of the screen and sending that to the 4 virtual stages. This worked fine to an extent but we were getting some real performance issues as the scenes using the virtual stages as inputs got more complicated. The first picture here shows the background scene, the second the playback scene. Stage 1 would be the main output, stages 2-5 are the virtual stages. You can see the load up at about 50%.

So we tried another set up with a single Screen Capture actor grabbing the whole screen that image was sent in parallel to 4x crop actors and each crop actor on to a virtual stage. This set up had significantly better performance. Again, background scene first followed by output scene, you can see here in the output scene the load is down at about 12%.

Finally, as what I really want for ease of work flow is a 'full screen' image in each virtual stage, I changed this set up slightly so the cropped images were scaled to fill the virtual stage (rather than the image being in one corner of a screen with alpha over the rest of the screen).

So, I thought I'd settled on the best way to do it. Then I tried connecting an external display. Again, for ease of working, I want the zoom call on a separate display so it wont be obscured by other windows and I won't need to use the CPU feature on the Screen Capture actor. Running the tests again, I got similar but slightly improved performance from the single Screen Capture actor set up with the external display. But I got vastly improved performance from the set up with 4 Screen Capture actors, so good it's about the same as the single set up. Here are three grabs from the output scenes but with the source on an external display, you'll see the load is in single figure %s on all examples.

Now, first I thought this might be because my discrete GPU kicked in with the external display, I'm on a MBP, with a discrete NVIDIA 750. But having checked, the discrete card seems to be engaged wether I've got the external display attached or not. Either way, I'm getting better results from the external display.

So, I'd be interested to hear from people who understand these internal workings better than I do why that might be. And also, to maybe save some people some time figuring things out put my notes above. .izz file attached here as well if it's useful for anyone.

For comparison, I'm on a mid2014 MacbookPro, 2.5 i7, 16GB and discrete GeForce GT 750m 2GB GPU. I plan to repeat the test on my PC when I next get it out (Joys of lock down #83: the PC normally lives in my office that I can't get in to at the moment. I've brought it home but there's not really anywhere to set it up, so to keep our tiny flat a pleasant place to live, it only comes out a couple of times a week when I need it and then goes away again so we can eat tea off the table).

Kathmandale

Oops. I didn't attach the .izz

Here it is...

Also, for ease of viewing here's a breakdown of the results:

Laptop Display:

4x Screen Captures to 4x V Stages - Load: 38-55%

1x Screen Capture to 4x crops to 4x V Stages - Load:12-16%

External Display:

4x Screen Captures to 4x V Stages - Load: 8-10%

1x Screen Capture to 4x crops to 4x V Stages - Load: 8-10%

Screengrab Power.izz

mark

@kathmandale said:

Now, first I thought this might be because my discrete GPU kicked in with the external display, I'm on a MBP, with a discrete NVIDIA 750. But having checked, the discrete card seems to be engaged wether I've got the external display attached or not. Either way, I'm getting better results from the external display.

First what an incredible, awesome and detailed report you've offered all of us. That is really (really!) helpful for everyone: thank you!

Honestly, I wish I could say why you are seeing the performance differences you are. I mean, it makes sense to me that running four Screen Capture actors would give reduced performance: each of them is going to attempt to get a texture from the screen and thus it's going to be 4x slower. And yet, as you said, when you use the external display, that's not the case. I simply don't know why this is, or if it this would be consistently the case for every Mac.

In the end, I probably need to optimize this actor so that if you add multiple Scren Capture actors that are all requesting the same display, they don't pull the image four times. Instead, what should happen is that the image for a given display should be pulled once and then the individual screen capture actors do the cropping for you request without you needing to use a Crop actor.

Regardless, what I would suggest is that you use a Chop Pixels actor instead of a Crop. With the Crop actor, you end up with black borders around the image. With the Chop Pixels actor, you will end up with an image that is just the chopped part you want. Give that a try and see what you think. (It won't change performance really, but it will be more convenient when working with the images.)

I will look into the improvement I described above. It would also (in the end) allow you to simply have a Screen Capture actor in each scene, which probably makes the workflow more convienient.

Best Wishes,
Mark

mark_m

@kathmandale

Apropos of nothing else, am I right in thinking that Pete Brooks is part of your company? Pete directed a flying trapeze show I performed in back in the early nineties, and ran the Scenography MA my partner, Laura Daly did. He's something of a star in our household! Say 'hi' from us!

Kathmandale

@mark

Thanks Mark, great suggestion for the Chop Pixels actor, that's exactly what I want. We'll be exploring this a lot over the next few weeks so any extra insight I'll put on here.

Thinking about it a little further, my laptop screen is notionally 1920x1200 (although I don't really understand how 'retina' displays work that out) and the external was 1920x1080, so there's a bit of difference there but I wouldn't have thought enough have such a dramatic effect.

Kathmandale

@mark_m

Amazing! Pete is indeed one of our directors. I'll pass your greetings on. Small world.