Isadora movie player performances with respect to VLC

[ikse]

Dear Mark,
Please find below an ultimate series of tests on the projector actor family. I hope it will help solving the issue raised in this thread such that Isadora eventually plays as well as VLC or MPC do.
The bug I should report lies in the projector actors once they are active in the scene whether a movie is played at their input or not.
**Projector actor
**- Showing the stage (3072x768 here) adds a little to the cpu load (from 2% to 14% usage on the Isadora core, from 1% to 2% on the total cpu usage)

• Playing 2K Blackmagic-multithreaded-decoded movie does not change much on the Isadora core (11%) but obviously adds up on the several cores (5% total cpu usage)
• Connecting the Movie player actor to the Projector actor starts loading quite a bit the Isadora core (52%)
• The load is rather linear with the number of active Projector actors: With 2 onto 2 stages, we get close to 100% (we saturate the Isadora core and it is difficult to check the linearity much longer)
  The behaviour is much more unexpected for the 3D projectors (3D Rect, 3D Quad,...) as the following post will show.

Dear Mehdi, dear DusX,
Would you have the same behaviour on your systems?

I believe it is not such a big deal with standard resolution movies or standard resolution stages, but Isadora seems to process something (even a black video) at the stage or movie resolution that takes the cpu time off.

Any hint?

Ikse

[DusX]

I am most interested by your comment that the projectors take cpu cycles even if not connected..
I will repeat your tests and report back.

[ikse]

Dear DusX, dear Mark,
Please find attached the series of basic tests on the Quad distort actor. There is no magic in the screen shots, nothing hidden, just peculiar and unexpected behaviours, at least to me, with respect to video displaying in Isadora as soon as a stage is set to shown.

Quad distort (no video input)- Without any video input, not even an input connection from a movie player actor, the Quad distort actor, as soon as it is set onto the scene, requires cpu time (~50% load on the core Isadora uses for an output stage of 1280x1024)

• Always or vid-in draw does not change the ~50% cpu load
• The number of output stages does not change the ~50% cpu load
  The fun part is here as the cpu load is not fully related to the size of the output stage, it also depends on its position and how much of it appears onto the hardware monitor
• Setting the output stage as a preview monitor 320x240 reduces the cpu load down to ~23%
• Setting the output stage as a preview monitor 640x480 doubles the cpu load to ~57% (but as you will see, the stage is partially hidden)
• Setting the output stage as a preview monitor 1280x960 gets the cpu load back to ~23% (but this time the preview window is fully shown in the monitor)
• When the preview window is partially hidden at the bottom of the monitor screen, the cpu load rises up to 57%
• When it is nicely set on the side so it is fully apparent, the cpu load gets back to 23%

I rerun the tests on a basic 2-core Dell laptop. The behaviour is similar but the actor load is not put onto a single core but nicely spread between the two cores - which questions on the side the relevance of the 32-core HP workstation for Isadora!

• Hide stage: ~9% total cpu
• Show stage + 1 preview 1440x900: 17% total cpu for 1 Quad distort
• Show stage + 2 preview 1440x900: 24% total cpu for 2 Quad distort

The forthcoming post will report on Quad distort with video in as it is what we want.
I hope these tests will help you, Mark, to figure out what Isadora is processing while, I believe, it should not.

Ikse

[ikse]

Dear Mark, dear DusX,

To continue the tests above, please find three additional screen shots, which show:

• 1 2K movie on in a Quad distort adds a little to the Isadora core (from ~50% to 60%) and a little on every other cores for multithreaded-ffmpeg blackmagic decoding (2% to 7% total cpu usage). The fps is ok (~24).
• 1 2K movie off in 2 Quad distort double the load onto the Isadora core (100%) so it is already saturated so the fps has already dropped down to ~12
• 1 2K movie on in 2 Quad distort keeps the Isadora coreto 100% but the fps drops down to ~10 while the multithreaded-ffmpeg blackmagic decoding remains identical (6% total cpu usage)

I got the same behaviour with the 3D Rect Project. The image processing, which Isadora runs as soon as the Output stages are shown, seems to be the same as the one run when a movie is on at the input of a Projector actor. It seems to be stray processing, which requires a lot of cpu load (at least on the HP workstation but has nothing to do with decoding (multithreadly performed outside Isadora by ffmpeg here) and little to do with effective video displaying (expectedly gpu-handled).

I hope these series of tests will help tracking this cumbersome processing. I do not think it is a computer-related issue but any tests, DusX,  you could run would help stating on this. I will be very happy to see this issue solved - it is obviously a bottleneck for any further development we have planned with Isadora. Time is obviously an issue. We must be ready by June!

Best of luck!

Ikse

[DusX]

I just noticed something that is perhaps related and worth taking a look at.
There is a difference between F24 and F33 as far as just having isadora open.
F24, opens and shows a cpu load of 1% of my total cpu (8% on one core)
F33, opens and shows a cpu load of 12% of my total cpu (showing ~ 50 to 80% on one core)

Obviously a lot can be done accomplished in F33, so the cpu is not maxed right when opened, however; some process is clocking cycles. So these readings may not be very reliable for performance testing the latest version.
@Mark I believe you told me somethings about a change that has a process running from launch now (f32/f33)?

In F24 the reading may still have value, however; video playback has been greatly improved in the newer versions.

[ikse]

Dear Mark,
Have you had the time for checking the processes involved in the display of the projector actors?

As we are moving along, we keep facing this issue as soon as the overall targetted stage matrix is important whatever configuration is set. 55% of a single core cpu usage is a reliable figure we get for just setting the required three Quad distort actors to address three video projectors (864x1152, 1024x576, 1024x576) and projecting black! Same thing for the 3D Mosaic actor...

I hope you will come up with the solution.

Ikse

P.S.: If the Isadora version were the issue, I would be happy to switch to a newer version. But it would mean that you have done something for the image display in the newer ones.
P.P.S.: DusX, Mehdi, did  you have the chance to give it a try? It you had or you could, I would be grateful to u for the outcome :

• A new scene
• Three Quad distort set to three different shown stages (1024x768 each to be standard or higher to enhance the effect)
• Shown stages
• Check out the cpu usage
  That's it! It works with one Quad  but the cpu usage is three time less!
  Thanks a lot.

[mark]

Dear All,

I have started to investigate this issue, and will be able to spend time on it over the next days.

I have some things to say, but first I need to get the most recent version of the pre-release uploaded. If we're going to look into this, we all need to be working with the latest pre-release. I have made some big changes to the way Isadora handles it's timing based on a long back-and-forth with an Isadora user in Uruguay. (There were several ways in which he was pushing the software... and the fact that overloading it would cause a UI freeze was one of the problems he had -- something that is mentioned here.) I should be able to put that up no later than Monday.

But I'll throw in here that I suspect this is an issue of Isadora waiting for the "vertical retrace interval" associated with the many displays you are attempting to render to. While it seems they are all working at 60Hz, they are not necessary synchronized with each other. This could mean, if Isadora ends up waiting for this retrace (which it shouldn't do, but perhaps it is inadvertently be doing so) one would experience significant delays -- even when the 3D Quad Distort has no video input. But when I made three 800x600 stages (admittedly, on one display -- not three separate displays) and added three 3D Quad Distort actors, I saw very little affect on CPU usage. So this is something we're going to need to look at carefully.

One note about CPU usage, which requires some "gory details."

Isadora has a custom-tuned function that attempts to deliver a "clock pulse" the main rendering system as accurately as possible. (The de facto method one can use for this, sending WM_TIMER messages to the application, is highly inaccurate.) This timing mechanism is important to ensure that the delivery of frames matches, as closely as possible, the target frame rate as set in to the Isadora preferences.Let's say we need to wait 15 milliseconds. The way this function works is to wait a certain amount of the desired time using a standard Sleep() function (which is also quite inaccurate) .For our example, let's say it waits 13 mS. Then, it executes a series of Sleep() commands with a delay of 0ms -- this gives other processes a time slice, but allows exit this loop with great accuracy. When you see the CPU usage high, even when Isadora is idle, that's why: it's the time spent in this this Sleep(0) function. It shows the CPU as being busy, but it's actually not that busy -- but it shows up as busy on the CPU monitor.

In any case, the first thing I need to examine to see if there are delays caused by waiting for the vertical retrace. Let me do that, and also get the new version up for you here, and we'll move forward from there.

Best,
Mark

[ikse]

Dear Mark,

This is great!
If it helps, I agree there is little change by adding Quad distort actors as long as they target the same stage. It is another story when they target a larger stage or different stages, namely when the overall stage matrix actually gets bigger. For matrices like 800x600, it does not run too badly, even on three different stages. I have just run the test: 18% of a core while idle but stages are shown, 30% while playing the movie.
As far as I could check, the cpu usage is actual, actual enough to damp the fps and slow down Isadora processing when it gets close to 100% of the used core.
I had tried the f25 but had stability issue with former patches without noticeable improvements so I came back to f24 and was reluctant so far to upgrade. I will install the f33 as you suggest and will check out the outcome. Hopefully, things will come easy and the patch might highly benefit from the improvements anyway (3D renderer etc...).

Thanks a lot for your care and your help in solving this long lasting issue.

Best regards from Paris,

Ikse

[EDIT] removed note about release and pre-release download pages because the comment is no longer true. MFC.

[DusX]

Mark I have a note regarding the UI locking up that might be of interest.
I have been able to consistently break out of a locked UI (Isadora seems completely locked except video output is still live), by using Midi to jump to a simplified snapshot.
So my process has been, to create create first in my video mixer patch a start snapshot that plays to a stage a single video (or stage color), then I proceed to create more complex mixes and creating additional snapshots. I use a midi pad control unit to trigger my snapshots.
My experience has shown that while building mixes, if I get to complex and freeze the UI (usually high cycles and low fps.. nearly equal each other) I am able to still receive midi and trigger the switch to a save snapshot (a simple one will respond most quickly), I have not test with OSC, but keyboard triggers and mouse interactions are not received.
Perhaps understanding why midi still works will help.

[mark]

Regarding the CPU usage: if Isadora is stuck waiting for the Vertical Refresh, then it would eat CPU for sure. This will be my first point of investigation.

Best,

M

[mark]

Dear ikse,

What is your target frame rate setting in the Isadora Preferences?

Best Wishes,
Mark

[mark]

Dear Ikse,

So, I've uploaded a new version, 1.3.1f02\. You can get it here: http://troikatronix.com/download/isadora-pre-releases/ at the bottom of the page.

It has a special feature I want you to test. After installing, open c:\program files (x86)\isadora. Create a text file called "disable-swap-interval.txt" (without the quotes of course) and drag it into the Isadora folder. (You'll probably need to OK this since the folder is usually protected.) 

If this file is present, Isadora will disable the vertical retrace synchronization within OpenGL. This will likely produce ugly artifacts, so I can't recommend you really use this for a production.

Anyway, run Isadora with this file in place and try your simple test with the 3D Quad Distort actors, as well as any other tests you had where performance was suffering. If performance changes noticeably, it will prove that waiting for the vertical retrace interval is the culprit.

Also, as I posted earlier, I want to know your target frame rate as set in the Isadora preferences. Please let me know.

I look forward to your findings.

Best Wishes,

Mark

[ikse]

Dear Mark,

Sorry  I was in a very very long programming meeting. I just got your messages. The framerate was 25 fps for the initial tests but for the latest it was lowered to 6 fps as it is allowed the patches I am currently working on to run.
I will download the updated version and run basic performance tests on it. Crossing fingers, I will send you the results!

Thanks  a lot,

Ikse

[ikse]

Mark,

First tests seem rather positive but I have to dash away for a last train. More spread cpu load. I will double check tomorrow.

Best regards,

Ikse

[mark]

Dear Ikse,

OK. Let me know. But I want to be super clear that the "disable-swap-interval.txt" test I've offered is **not** a solution; as I said you'll likely see "tearing" effects as a result of turning the vertical sync off. (This will appear something like the picture here http://i.stack.imgur.com/qIXX3.jpg -- look at the line between the old frame and the new one, about 20% from the left.) The purpose of this test is simply to find out of the vertical sync is the reason for the slowness.

Also, it's good you're frame rate is set to 25 fps -- I was concerned it was something higher. But, you might try setting it to 30fps -- since this would synchronize with your monitor's refresh rate of 60fps. That might actually make an important difference. So give this a try without the "disable-swap-interval.txt" present to see if it does anything for you.

Best Wishes,

Mark

[ikse]

Dear Mark,

Thanks for the above clarification on the sleep loop and the apparent cpu usage. The first three attached screen shots are pretty explicit. With an empty stage but a performance monitor, the cpu usage rises up from virtually 0% to 82% then 98% for 6, 15, and 30 fps as set in the preferences respectively.

It was also clear to me that the disable file was only for testing and I did run the expected tests on a set of two 2K movies played and projected with either the Projector, 3D Quad distort, or 3D Rect Project actors onto two 2K stages for a targetted fps of 25. Following the attached screenshots :
Projector

• (05) Vblank enable, 1 movie playing: 20 fps (cpu 82%) - 2 movies playing: 8.6 fps (cpu 94%)
• (06) Vblank disable, 1 movie playing: 24.3 fps (cpu 74%) - 2 movies playing: 24.9 fps (cpu 72%)
  3D Quad distort
• (07) Vblank enable, 1 movie playing: 11 fps (cpu 95%) - 2 movies playing: 8.6 fps (cpu 96%)
• (08) Vblank disable, 1 movie playing: 25 fps (cpu 80%) - 2 movies playing: 25 fps (cpu 76%)
  **3D Rect Project
  **- (09) Vblank enable, 1 movie playing: 10 fps (cpu 94%) - 8.6 movies playing: 8.6 fps (cpu 94%)
• (10) Vblank disable, 1 movie playing: 25 fps (cpu 86%) - 2 movies playing: 25 fps (cpu 72%)

The conclusion to be drawn is clear cut: waiting for the vblank is eating up a lot of cpu and consequently a lot of fps. This effect depends on the stage size (onto what the movie is projected) so it is neglible for standard resolution but becomes killing for high definition and above.

The hint was good.

Best of luck for further tracking down!

Ikse

[fifou]

Hello Mark, Ikse, DusX,
Thanks for all your investigations. I've been struggling with this little random stutters problems for years.
My patches were always very simple : one player at a time, no effects, no resize, full 1024768 photojpeg moviefrom ssd drive to a 1024768 stage @ 25 fps . Isadora has been seriously improved above years (thanks to all your hard work Mark) but I still had this stutter problem.
I tried to match isadora's framerate to videocard's refresh rate (isadora 25fps - videocard 50Hz or 75Hz) but it didn't changed anything.

By reading your post (and others) I tried to disable vertical sync (with the help of Quartz Debug) and it works ! No more stutters ! I could believe my eyes : my favorite video application was finally playing really smoothly my 25fps movies. Maybe that's only a 25fps users issue..
I can see sometimes the tearing effect you are talking about Mark, but I think I prefer it to frame drops !
I understand that disabling VSync is not THE solution but that's one for me, for the moment.

Thanks for all your long investigations, (you are about to solve my 4 years old problem.)

Best
Philippe

[mark]

Dear Phillippe and All,

Well, this vertical sync thing is something that's going to require a very careful investigation to address.

 Just to talk it out a bit: It's very strange that this has an effect.The way the rendering cycle goes like this:

1) Execute the "swap buffers" command which tells OpenGL/the GPU to present the frame that was drawn during the last cycle.

2) Execute the actors and draw whatever they need to draw via OpenGL

3) Execute the OpenGL flush command (glFlush), which tells the card to start executing all the commands it just received

4) Wait for the next "timer tick" that indicates it's time to render the next frame -- unless your patch is very heavy and/or you've increased the frame rate to some higher value, this should a decent amount of time. (The more the better)

So when we get back to step 1, the glFlush() command has caused all the rendering to be completed. All that needs to happen now is that the "swap buffers" command should show the results. According to all the documentation I've read, the swap buffers call should not "block" (i.e., wait for the vertical interval.) But I am getting the feeling that it is blocking, and this is the cause of the problem.

In the end, disabling VSync is most definitely not the solution. But I guess if the tearing artifacts are OK with y'all, then use the workaround until I can give you all a better solution.

For sure I will prioritize this and find a solution as soon as I can.

Best Wishes,

Mark

P.S. Phiippe: what setting exactly did you change in Quartz Debug?

[mark]

Dear All,

OK. So I've been looking into this for a couple of hours, and running Isadora with the OpenGL Profiler and Driver Monitor apps to see if I can get a fix on this. So far, there is nothing telling.

But, there is one thought for the Windows users: you might try removing the "disable-swap-interval.txt" file and then turning on "triple-buffering" if your driver supports this. 

This idea was supported by [this post](http://www.anandtech.com/show/2794) and a few others I came across.

I would like to hear the results of this if you can try it.

(Mac OS doesn't seem to support this option at all.)

Best Wishes,

Mark

[ikse]

Dear Mark,

I should be able to give it a try later today and come back to you with some outcomes on triple-buffering. The post seemed rather optimistic.

Best regards from sleeping Paris,

Ikse