Isadora movie player performances with respect to VLC

[mark]

Dear All,

Thank you for this continued, very detailed research. I forgot to say that, after Munich, I was going to the USA, where I am now. But, my travels are almost over I return to Berlin on Monday. Then, finally, I can properly digest and look carefully at all you have to say.

Best Wishes,

Mark

[DusX]

@ikse
Did you happen to take note as to wheather or not any of these codec option allowed for looping. I have been doing a number of windows native + Qt HD test as well (not as well documented) and a big problem for me with most windows options is that the video can not be looped.

[ikse]

Dear DusX,
Looping is not an issue with the VC1 codec. It loops indefinitely under Isadora. I did face some looping issues with lossless avi-contained codec - namely there was no end trigger, so no looping possibility. The player could not figure out that the video had reached its end, whereas, surprisingly enough, it was possible to get its time duration with Media time to percent so I did generate the trigger with a time count down... not very clean.
As it appears more and more clearly to me now, the codec is not that an issue within Isadora. It can be circumvented by the Plays Windows movie formats natively so Isadora does perform as well as another player. At least the movie player actor does conform to this. The problem seems to lie in the projector actor.
I'll come with additional tests on this.
Ikse

[ikse]

Dear Mehdi, Mark, DusX,

Following some Mehdi's advices, please find attached some last results that answer two questions:

• Projecting on a full monitor screen costs more cpu load (mostly onto the core Isadora uses) than projecting on a preview screen. The gpu load is barely affected.
• Projecting several movies on a single preview screen or on separate preview screens do not change the cpu load nor the gpu load
  One could infer from these tests that Isadora runs some additional cpu calculations on the initial image matrix whatever the targeted screen may be. The purpose of such calculations are rather unclear to me as they occur only when the movie player actor is connected to the projector actor whereas the process should be handled by the gpu only.
  I'll hopefully come with some more later today.

Best regards from Paris,

Ikse

[DusX]

Perhaps this issue is related?
http://troikatronix.com/community/#/discussion/638/problem-with-multiple-projectors-and-scenes-drastic-drop-of-framerate-with-each-new-display.
Have any of you tested with the movie player direct?

[ikse]

Dear DusX, Mehdi, Mark,
I did try with the movie player direct and it did not improve the performances. I am convinced now that the limitations come from the projector itself.
I ran another series of comparisons within Isadora this time:
1. Again, the preview seems to be lighter on the cpu than a full screen projection:
    - it is clear and expected between a 1920x1080 monitor and a 320x240 preview window
    - it is clear but much less expected between an 1280x800 monitor and a 1280x800 preview window (the Isadora core is at 44% and 27% for a total cpu usage of 3 and 4% respectively)
2. Blackmagic codec works fairwell as long as the hardrive bandwidth can keep up with the load of data to be transfered (it is certainly a reason for the difficulty of looping mentioned above... the time is over but there are still some jetlagged frames coming in). It actually works very very well with SSD and ffmpeg together. Decoding is very light and well spread on the cores. However the core Isadora uses reaches 100% with four HD movies.
3. Projector, 3D Quad distort, 3D Rect project actors behave more or less the same way in terms of cpu load: 86%, 77%, and 89% on the Isadora core, 9%, 10%, and 12% overall respectively.
4. Projector actors take nearly half of the cpu load as, when removed, the overall usage falls down to 7% and the Isadora core is lowered to 36%.

The last point is striking as one may expect the projector actors to be relatively free from the cpu and handled by the gpu. In summary, I should say that the movie decoding is properly handled and it is only limited by the local software (codecs) and hardware (hardrive access and cpu power). In contrary, the display (projector actors) does not seem to work the way it should and weight too much onto the cpu (essentially onto the core Isadora uses) for large resolution movies (it is rather transparent for standard resolution movies) whereas it is expected to be gpu-handled.

I hope there is reason and a solution behind. I might still be doing things the wrong way and I will be happy to correct that.

Thanks for your help.

Ikse

P.S.: Mark, is there any reason for limiting the vertical size of the 3D renderer today?

[ikse]

Dear Mark,
Please find below an ultimate series of tests on the projector actor family. I hope it will help solving the issue raised in this thread such that Isadora eventually plays as well as VLC or MPC do.
The bug I should report lies in the projector actors once they are active in the scene whether a movie is played at their input or not.
**Projector actor
**- Showing the stage (3072x768 here) adds a little to the cpu load (from 2% to 14% usage on the Isadora core, from 1% to 2% on the total cpu usage)

• Playing 2K Blackmagic-multithreaded-decoded movie does not change much on the Isadora core (11%) but obviously adds up on the several cores (5% total cpu usage)
• Connecting the Movie player actor to the Projector actor starts loading quite a bit the Isadora core (52%)
• The load is rather linear with the number of active Projector actors: With 2 onto 2 stages, we get close to 100% (we saturate the Isadora core and it is difficult to check the linearity much longer)
  The behaviour is much more unexpected for the 3D projectors (3D Rect, 3D Quad,...) as the following post will show.

Dear Mehdi, dear DusX,
Would you have the same behaviour on your systems?

I believe it is not such a big deal with standard resolution movies or standard resolution stages, but Isadora seems to process something (even a black video) at the stage or movie resolution that takes the cpu time off.

Any hint?

Ikse

[DusX]

I am most interested by your comment that the projectors take cpu cycles even if not connected..
I will repeat your tests and report back.

[ikse]

Dear DusX, dear Mark,
Please find attached the series of basic tests on the Quad distort actor. There is no magic in the screen shots, nothing hidden, just peculiar and unexpected behaviours, at least to me, with respect to video displaying in Isadora as soon as a stage is set to shown.

Quad distort (no video input)- Without any video input, not even an input connection from a movie player actor, the Quad distort actor, as soon as it is set onto the scene, requires cpu time (~50% load on the core Isadora uses for an output stage of 1280x1024)

• Always or vid-in draw does not change the ~50% cpu load
• The number of output stages does not change the ~50% cpu load
  The fun part is here as the cpu load is not fully related to the size of the output stage, it also depends on its position and how much of it appears onto the hardware monitor
• Setting the output stage as a preview monitor 320x240 reduces the cpu load down to ~23%
• Setting the output stage as a preview monitor 640x480 doubles the cpu load to ~57% (but as you will see, the stage is partially hidden)
• Setting the output stage as a preview monitor 1280x960 gets the cpu load back to ~23% (but this time the preview window is fully shown in the monitor)
• When the preview window is partially hidden at the bottom of the monitor screen, the cpu load rises up to 57%
• When it is nicely set on the side so it is fully apparent, the cpu load gets back to 23%

I rerun the tests on a basic 2-core Dell laptop. The behaviour is similar but the actor load is not put onto a single core but nicely spread between the two cores - which questions on the side the relevance of the 32-core HP workstation for Isadora!

• Hide stage: ~9% total cpu
• Show stage + 1 preview 1440x900: 17% total cpu for 1 Quad distort
• Show stage + 2 preview 1440x900: 24% total cpu for 2 Quad distort

The forthcoming post will report on Quad distort with video in as it is what we want.
I hope these tests will help you, Mark, to figure out what Isadora is processing while, I believe, it should not.

Ikse

[ikse]

Dear Mark, dear DusX,

To continue the tests above, please find three additional screen shots, which show:

• 1 2K movie on in a Quad distort adds a little to the Isadora core (from ~50% to 60%) and a little on every other cores for multithreaded-ffmpeg blackmagic decoding (2% to 7% total cpu usage). The fps is ok (~24).
• 1 2K movie off in 2 Quad distort double the load onto the Isadora core (100%) so it is already saturated so the fps has already dropped down to ~12
• 1 2K movie on in 2 Quad distort keeps the Isadora coreto 100% but the fps drops down to ~10 while the multithreaded-ffmpeg blackmagic decoding remains identical (6% total cpu usage)

I got the same behaviour with the 3D Rect Project. The image processing, which Isadora runs as soon as the Output stages are shown, seems to be the same as the one run when a movie is on at the input of a Projector actor. It seems to be stray processing, which requires a lot of cpu load (at least on the HP workstation but has nothing to do with decoding (multithreadly performed outside Isadora by ffmpeg here) and little to do with effective video displaying (expectedly gpu-handled).

I hope these series of tests will help tracking this cumbersome processing. I do not think it is a computer-related issue but any tests, DusX,  you could run would help stating on this. I will be very happy to see this issue solved - it is obviously a bottleneck for any further development we have planned with Isadora. Time is obviously an issue. We must be ready by June!

Best of luck!

Ikse

[DusX]

I just noticed something that is perhaps related and worth taking a look at.
There is a difference between F24 and F33 as far as just having isadora open.
F24, opens and shows a cpu load of 1% of my total cpu (8% on one core)
F33, opens and shows a cpu load of 12% of my total cpu (showing ~ 50 to 80% on one core)

Obviously a lot can be done accomplished in F33, so the cpu is not maxed right when opened, however; some process is clocking cycles. So these readings may not be very reliable for performance testing the latest version.
@Mark I believe you told me somethings about a change that has a process running from launch now (f32/f33)?

In F24 the reading may still have value, however; video playback has been greatly improved in the newer versions.

[ikse]

Dear Mark,
Have you had the time for checking the processes involved in the display of the projector actors?

As we are moving along, we keep facing this issue as soon as the overall targetted stage matrix is important whatever configuration is set. 55% of a single core cpu usage is a reliable figure we get for just setting the required three Quad distort actors to address three video projectors (864x1152, 1024x576, 1024x576) and projecting black! Same thing for the 3D Mosaic actor...

I hope you will come up with the solution.

Ikse

P.S.: If the Isadora version were the issue, I would be happy to switch to a newer version. But it would mean that you have done something for the image display in the newer ones.
P.P.S.: DusX, Mehdi, did  you have the chance to give it a try? It you had or you could, I would be grateful to u for the outcome :

• A new scene
• Three Quad distort set to three different shown stages (1024x768 each to be standard or higher to enhance the effect)
• Shown stages
• Check out the cpu usage
  That's it! It works with one Quad  but the cpu usage is three time less!
  Thanks a lot.

[mark]

Dear All,

I have started to investigate this issue, and will be able to spend time on it over the next days.

I have some things to say, but first I need to get the most recent version of the pre-release uploaded. If we're going to look into this, we all need to be working with the latest pre-release. I have made some big changes to the way Isadora handles it's timing based on a long back-and-forth with an Isadora user in Uruguay. (There were several ways in which he was pushing the software... and the fact that overloading it would cause a UI freeze was one of the problems he had -- something that is mentioned here.) I should be able to put that up no later than Monday.

But I'll throw in here that I suspect this is an issue of Isadora waiting for the "vertical retrace interval" associated with the many displays you are attempting to render to. While it seems they are all working at 60Hz, they are not necessary synchronized with each other. This could mean, if Isadora ends up waiting for this retrace (which it shouldn't do, but perhaps it is inadvertently be doing so) one would experience significant delays -- even when the 3D Quad Distort has no video input. But when I made three 800x600 stages (admittedly, on one display -- not three separate displays) and added three 3D Quad Distort actors, I saw very little affect on CPU usage. So this is something we're going to need to look at carefully.

One note about CPU usage, which requires some "gory details."

Isadora has a custom-tuned function that attempts to deliver a "clock pulse" the main rendering system as accurately as possible. (The de facto method one can use for this, sending WM_TIMER messages to the application, is highly inaccurate.) This timing mechanism is important to ensure that the delivery of frames matches, as closely as possible, the target frame rate as set in to the Isadora preferences.Let's say we need to wait 15 milliseconds. The way this function works is to wait a certain amount of the desired time using a standard Sleep() function (which is also quite inaccurate) .For our example, let's say it waits 13 mS. Then, it executes a series of Sleep() commands with a delay of 0ms -- this gives other processes a time slice, but allows exit this loop with great accuracy. When you see the CPU usage high, even when Isadora is idle, that's why: it's the time spent in this this Sleep(0) function. It shows the CPU as being busy, but it's actually not that busy -- but it shows up as busy on the CPU monitor.

In any case, the first thing I need to examine to see if there are delays caused by waiting for the vertical retrace. Let me do that, and also get the new version up for you here, and we'll move forward from there.

Best,
Mark

[ikse]

Dear Mark,

This is great!
If it helps, I agree there is little change by adding Quad distort actors as long as they target the same stage. It is another story when they target a larger stage or different stages, namely when the overall stage matrix actually gets bigger. For matrices like 800x600, it does not run too badly, even on three different stages. I have just run the test: 18% of a core while idle but stages are shown, 30% while playing the movie.
As far as I could check, the cpu usage is actual, actual enough to damp the fps and slow down Isadora processing when it gets close to 100% of the used core.
I had tried the f25 but had stability issue with former patches without noticeable improvements so I came back to f24 and was reluctant so far to upgrade. I will install the f33 as you suggest and will check out the outcome. Hopefully, things will come easy and the patch might highly benefit from the improvements anyway (3D renderer etc...).

Thanks a lot for your care and your help in solving this long lasting issue.

Best regards from Paris,

Ikse

[EDIT] removed note about release and pre-release download pages because the comment is no longer true. MFC.

[DusX]

Mark I have a note regarding the UI locking up that might be of interest.
I have been able to consistently break out of a locked UI (Isadora seems completely locked except video output is still live), by using Midi to jump to a simplified snapshot.
So my process has been, to create create first in my video mixer patch a start snapshot that plays to a stage a single video (or stage color), then I proceed to create more complex mixes and creating additional snapshots. I use a midi pad control unit to trigger my snapshots.
My experience has shown that while building mixes, if I get to complex and freeze the UI (usually high cycles and low fps.. nearly equal each other) I am able to still receive midi and trigger the switch to a save snapshot (a simple one will respond most quickly), I have not test with OSC, but keyboard triggers and mouse interactions are not received.
Perhaps understanding why midi still works will help.

[mark]

Regarding the CPU usage: if Isadora is stuck waiting for the Vertical Refresh, then it would eat CPU for sure. This will be my first point of investigation.

Best,

M

[mark]

Dear ikse,

What is your target frame rate setting in the Isadora Preferences?

Best Wishes,
Mark

[mark]

Dear Ikse,

So, I've uploaded a new version, 1.3.1f02\. You can get it here: http://troikatronix.com/download/isadora-pre-releases/ at the bottom of the page.

It has a special feature I want you to test. After installing, open c:\program files (x86)\isadora. Create a text file called "disable-swap-interval.txt" (without the quotes of course) and drag it into the Isadora folder. (You'll probably need to OK this since the folder is usually protected.) 

If this file is present, Isadora will disable the vertical retrace synchronization within OpenGL. This will likely produce ugly artifacts, so I can't recommend you really use this for a production.

Anyway, run Isadora with this file in place and try your simple test with the 3D Quad Distort actors, as well as any other tests you had where performance was suffering. If performance changes noticeably, it will prove that waiting for the vertical retrace interval is the culprit.

Also, as I posted earlier, I want to know your target frame rate as set in the Isadora preferences. Please let me know.

I look forward to your findings.

Best Wishes,

Mark

[ikse]

Dear Mark,

Sorry  I was in a very very long programming meeting. I just got your messages. The framerate was 25 fps for the initial tests but for the latest it was lowered to 6 fps as it is allowed the patches I am currently working on to run.
I will download the updated version and run basic performance tests on it. Crossing fingers, I will send you the results!

Thanks  a lot,

Ikse

[ikse]

Mark,

First tests seem rather positive but I have to dash away for a last train. More spread cpu load. I will double check tomorrow.

Best regards,

Ikse