Screen Capture actor (Alpha) - Bug Reports & Updated Versions

[CitizenJoe]

This is similar to my post in the other thread, but updated with observations of v0.9.3 on Windows:

• It only works in CPU mode;
• On my main laptop display, it captures the whole window as expected;
• On the second display, it only captures a portion of the window. Please see the attached image;
• Even though it's not showing anything in GPU mode, the load is around 250%! Switch to CPU mode and  the load immediately drops to 40%;
• It will not show Chrome windows on either display, but has no trouble (on Display 1) showing work, excel, thunderbird, etc.
• the Actor sees all the windows that are open in the window list, including Chrome, regardless of which display they are on;

Cheers,

Hugh

[mark]

@sommerdi said:

I still have to topic with Catalina ....  maybe i have a missunderstanding ...

You are seeing the correct behavior.

With CPU mode off, Isadora can only capture exactly what you see on your screen. That means you will not be able to see the contents of any window that is behind (obscured) by another window. With CPU mode on, Isadora can capture the contents of a window even if it is obscured. However, this leads to much higher CPU usage on MacOS.

To be really clear about this, here's a screenshot of a browswer window displaying the front page of https://giphy.com and then Isadora's window in front of it.

If I capture this with CPU mode off, the area being captured is correct; it is the bounds of the window displaying Giphy in the browser. But the image that is captured is exactly what you seen on the screen: the Giphy window with Isadora's window in front of it. 

Total CPU usage in the Activity Monitor is about 20% in this mode which is pretty good.

Now, if you turn CPU mode on, then you can get see the contents of the giphy in the browser window, even though it is behind the Isadora's window. Here's the captured image in that mode.

>>> But total CPU usage in this mode is 73%!! <<<

By using CPU mode, you can capture an obscured window but you are slowing down your computer – especially when the window is large.

I hope that makes things clearer for you.

Best WIshes,
Mark

[mark]

@citizenjoe

Basically, there are severe limitations when using the Screen Capture actor when you have both a high performance graphics card (GPU) and the Intel Integrated Graphics GPU -- this is not just problematic in Isadora, but in programs like OBS which is highly developed in terms of screen capture.

The problem is that the high performance GPU and Intel Integrated Graphics two separate GPU's that do not share images. You can't capture a display from one when you're running on the other.

To be explicit:

1) Restart Computer - Windows Boots Up. The main display (display 1) of your laptop uses Intel Integrated Graphics to show you the computer's desktop, windows, etc. Note that additional displays (display 2, display 3, etc.) are ususally assigned to the high performance card and should not face the limitation I'm about to describe.
2) Start Up Isadora -- it gets assigned to your high performance GPU because the operating system see it needs high performance features. (This can be adjusted, see below.)
3) Add Screen Capture actor which is running in Isadora on the high performance GPU. It tries to capture display 1, but the image for that display is on the Intel Integrated Graphics GPU and Isadora cannot access it because the two seaprate GPUs do not share images.

As I said, this is a problem on OBS and other softwares too. I have not tried the procedure in this video, but it suggests forcing OBS to use the Intel Integrated Graphics instead of the high performance GPU. Try following the procedure in the video, but replace OBS with Isadora as you follow the instructions. In theory (again I haven't tried it myself yet) this should allow you to use Screen Capture for display 1, but with the cost that Isadora is no longer running on your high performance GPU.

I hope that helps you understand the problem. If you try the procedure in the youtube video above, let me know what happens.

Best Wishes,
Mark

[sommerdi]

Thanks for clarification and your support .... 

Diether

[kikko]

Thank you @mark for the clarification. Exactly same OBS behavior, actually I've already try the procedure you suggest and it work, but the limitation is when you have more than one display, you have to chose if grab only the laptop display or the other display belonging to Nvidia GPU

Francesco

[CitizenJoe]

@mark 

Thanks for the clear explanation. It all makes sense now. Of course, the second display is running on the dedicated GPU (via Thunderbolt),  which is why I can see it. I'll try you're suggestions above.

Cheers,

Hugh

[deflost]

hello all,

it would be great, to have a option to switch the visibility of the mouse pointer on and off?

and sorry for anothertime the question why is while using the screencaptureactor the load in izzy 180% and the framerate goes to 12,

when a youtube video in a browser window is paused, and goes down by play the video to 10 % and the framerate goes back to 60?

thx

r.h.

[mark]

@deflost said:

and sorry for anothertime the question why is while using the screencaptureactor the load in izzy 180% and the framerate goes to 12,

 It's really important that you specify the operating system (Mac or Windows), operating system version (Mojave, Windows 10, etc), and the version of the Screen Capture actor when asking these questions. The covention is the forum is to put that information in your about your hardware and operating system in your signature line so we don't have to ask that every time. (For example, look at @CitizenJoe 's signature above.)

Can you do that please? Also tell me the version of the screen capture actor.

Thank you,
Mark

[deflost]

@mark

sorry, ...

win 10 64 bit, actor version 0.9.3, 

thx,

r.h.

[Videosmith]

hi folks

I really enjoyed the guru session on this actor, and i'm super grateful you released it early. I have just started teaching an interactive video Isadora course for a uni in Vancouver, so I was able to introduce my students to Isadora. I showed them the Screen Capture actor this week, and I'm hoping it will open up some new ideas for them.

I've also been making some short video tutorials for the students. I have made a video on the SCREEN CAPTURE actor - it doesn't go into as much depth as the guru session, but hopefully it works as a nice intro. I thought I'd share it here:

Screen Capture Actor intro pt 1 - VIDEO

Screen Capture Actor intro pt 2 - VIDEO

also - intro to Camtwist - VIDEO

[Fred]

@mark with screen capture 0.94 and 0.93 on OSX 10.15.7 I get flashing windows when trying to capture using CPU from any browser. Is this a know issue, (it seems less frequent with Firefox compared to safari and chrome), it is not really useable. I have verified on a few different machines that the behaviour is the same.

[Woland]

@fred
I've logged your issue. In the meantime is it possible for you to work with the plugin in non-CPU mode as a workaround?

[Fred]

@woland no none of the scenarios will let me do that. I am using another software for this show for now.

[Woland]

@fred

We're glad that you were able to find a solution, even if it didn't involve using Isadora

[Kathmandale]

In a perhaps related but probably separate issue, I noted a behaviour recently where the performance of the screen capture actor drops significantly in CPU mode if the targeted window is on a lower layer for a long time. The frame rate would drop to about 3 or 4 fps. Moving it to the front or possibly just part of it to the front, and then returning it to the back would fix it for a while before, eventually, performance would again drop off a cliff. This was on a mac, OS10.14

I guess the OS decides a window that has been hidden for a while is 'inactive' and stops refreshing it so often until it brought back to the front.

Keeping the window I was targeting at least partly visible seemed to make a huge difference. Eventually I got a second display hooked up to better facilitate GPU mode, which I guess is always the preferred option where possible.

[Fred]

@kathmandale I just switched to syphon screen capture and don't get any issues even when the other window is maximized underneath Isadora. I don't think there are limitations but just some teething issues with the screen capture plugin.

[Woland]

@fred said:

@mark with screen capture 0.94 and 0.93 on OSX 10.15.7 I get flashing windows when trying to capture using CPU from any browser. Is this a know issue, (it seems less frequent with Firefox compared to safari and chrome), it is not really useable. I have verified on a few different machines that the behaviour is the same.

I've just tested this on my Catalina partition using two monitors and my Mac Pro 2013 trashcan (because your syphon report reminded me of this). I opened a Google Chrome tab playing a Youtube video, then:

• Set the Screen Capture Alpha to display 1 and just captured the entire display
  • With 'window cpu mode' set to 'on', the framerate is fine (29-30fps) but I don't see any flashes.
  • With 'window cpu mode' set to 'off', the framerate is fine (29-30fps) but I still don't see any flashes.

• Set the Screen Capture Alpha to display 1, and selected the window from the "window list" input. 
  • With 'window cpu mode' set to 'on', I don't get a very good frame rate, (around 6 fps) but I don't see any flashes.
  • With 'window cpu mode' set to 'off', the framerate is fine (29-30fps) but I still don't see any flashes.

I have noticed that if I screen capture something that's a static website, the Screen Capture ALPHA actor only delivers one frame (and will only deliver a new one when something on the screen being captured changes). Youtube is constantly changing, so the actor is sending new frames at a steady rate, but so I'm not sure what would happen if you're screen capturing a website that only updates sporadically. That being said, I'd like to know what website you are trying to capture from; this may actually be a factor, so I'd like to include it in the my attempts to reproduce your issue.

[Fred]

@woland the website is an Ai bot that has been developed for this show. It is largely static. I will make a screen capture of but this and the syphon error and post them soon.

[Fred]

@woland here is a video of the screen capture flashing, it is quite pronounced. I have posted a video of the syphon version in the other thread.

https://www.dropbox.com/s/toks...

[mark]

@fred said:

I get flashing windows when trying to capture using CPU from any browser. Is this a know issue, (it seems less frequent with Firefox compared to safari and chrome), it is not really useable. I have verified on a few different machines that the behaviour is the same.

It is not a known issue. I am capturing right now from Safari on 10.15.7 (same as you) and I am not experiencing and black frames on my 2018 Mac Book Pro. @Woland tested this on his Mac Pro (round one) and also did not see any problems.

Upon debriefing another user about his recent show, which included weeks of rehearsals and a two week run for a paying audience, reported to me no problems with the Screen Capture actor at all -- though he was using it with Zoom, not a browser.

We need to know the exact website you were trying to capture from so we can try to recreate this issue. Please get that information to @Woland as soon as you can manage.

Best Wishes,
Mark