Isadora movie player performances with respect to VLC

[ikse]

Hi Mehdi,
I have two main superactors:
1. The first one displaces a large resolution background movie with seven 3D rect Projectors sent to a single channel to be rendered by a 3D renderer. Ideally the resolution of the seven movies and the 3D renderers should match the resolution of the background movie to keep the latter resolved (minimal targeted matrix : 2000x1600). The vertical size of 3D renderer is anyway limited to 768. It obviously freezes when approaching a decent resolution (if I remember well it saturates for something like 512x512).
2. The second one generates seven 3D objects (sphere for now) on the same large resolution background movie with seven 3D players sent to a single stage to be rendered by a 3D renderer. It works fine until I try to play either the background movie or project movies onto the 3D objects. The target obviously is to play both at the same time.

I would agree with you on the expected performances with projectors. However, the projectors might be gpu-based but I guess the associated movie players take over the cpu load. So it freezes even in the second case described above.

The three Quadro cards are independant.

I will post the screenshots and come back to you as soon as I go back to the neticLab, hopefully later today.

Have a nice day!

Xavier

[keftaparty]

Xavier,

I think one problem here is the 3d renderer. It's performance eating on heavy resolutions.

Also, when working with 3d objects, you need to take care to some facts:

-It's "complicated" to share the same 3d object to multiple graphic cards, even on multiple "heads" of a same graphic card.

I've had some issues with that kind of setups in the past.

If you want we can skype ( in french ). Send me a private message with your skype name if you want.

Mehdi

[ikse]

Mehdi,

I have arrived later than expected. Please find below the screenshots. I have not looked into the setting of the graphic cards. Any improvement is certainly welcome.
However, I don't think it is an issueso far for the 3D objects as with the seven of them, it runs smoothly. It is true that there were shared between graphics cards and within a card as they travel through three videoprojectors (2x1280800+1x12801024).

A Skype would be great to sort this out when you have some time.

Meanwhile I will post a late series of tests on cpu loading.

Ikse

[ikse]

Mehdi, Mark,

For a fairer comparison, I have installed K-Lite_Codec_Pack_930 for 32 and 64 bits. It comes with the Media player classic (MPC). It is then possible to run both Isadora with six movie players and six projectors and to compare it to six MPC while making sure that they both use the same decoding process (CPU or GPU hardware accelerated).
I tried several codex configurations (still for the same WVMC1 encoded HD movie). Please find the outcomes in the series of screenshots below (the whole series is compressed in the last file).
The file names are self-explicit. However, in summary, it appears that:
For Isadora

• There is a substantial gain to make use of the ffdshow decoder (the total CPU load falls from 39% down to 18%)
• Isadora does not stand Nvidia Cuvid (GPU acceleration) but runs with DXVA2copyback but without any gain
• Showing the stages cost quite a bit in CPU load (the total CPU load falls by half again from 18-19% down to 10-12%) which is pretty surprising for me. At the same time, the GPU load is substantially reduced (figure again)
• Confining Isadora to 8 cores does not change the overall CPU load - as one can expect
  In summary:
• Native codex 38-39%
• FFv codex 18-19%
• Nvidia Cuvid -> Crashes
• DXVA2copyback 18-19%
• FFv codex + Hidden stages 10-12%
  For MPC
  The overall CPU load is still much less than Isadora
• Native codex 8%
• FFv codex 7%
• Nvidia Cuvid 6%
• FFv codex + Nvidia Cuvid 6%

As it limits our developments, I would be happy to understand why there is such a need of CPU power for playing videos with Isadora and if there are ways to go around. The critical states are obviously exemplified in the screenshots with DXVA2copyback where there is one core (the one that Isadora uses I guess) which is overloaded. It is close to saturate and Isadora, to freeze.

Again, I am hopefully doing something wrong here. The system is as simple as can be with only the required softwares and codex to run the tests (+ the monitoring gadgets).

Thanks for your feedback,

Ikse
3a2030-video_performances_with_mpc.zip

[mark]

Dear All,

Thank you for this continued, very detailed research. I forgot to say that, after Munich, I was going to the USA, where I am now. But, my travels are almost over I return to Berlin on Monday. Then, finally, I can properly digest and look carefully at all you have to say.

Best Wishes,

Mark

[DusX]

@ikse
Did you happen to take note as to wheather or not any of these codec option allowed for looping. I have been doing a number of windows native + Qt HD test as well (not as well documented) and a big problem for me with most windows options is that the video can not be looped.

[ikse]

Dear DusX,
Looping is not an issue with the VC1 codec. It loops indefinitely under Isadora. I did face some looping issues with lossless avi-contained codec - namely there was no end trigger, so no looping possibility. The player could not figure out that the video had reached its end, whereas, surprisingly enough, it was possible to get its time duration with Media time to percent so I did generate the trigger with a time count down... not very clean.
As it appears more and more clearly to me now, the codec is not that an issue within Isadora. It can be circumvented by the Plays Windows movie formats natively so Isadora does perform as well as another player. At least the movie player actor does conform to this. The problem seems to lie in the projector actor.
I'll come with additional tests on this.
Ikse

[ikse]

Dear Mehdi, Mark, DusX,

Following some Mehdi's advices, please find attached some last results that answer two questions:

• Projecting on a full monitor screen costs more cpu load (mostly onto the core Isadora uses) than projecting on a preview screen. The gpu load is barely affected.
• Projecting several movies on a single preview screen or on separate preview screens do not change the cpu load nor the gpu load
  One could infer from these tests that Isadora runs some additional cpu calculations on the initial image matrix whatever the targeted screen may be. The purpose of such calculations are rather unclear to me as they occur only when the movie player actor is connected to the projector actor whereas the process should be handled by the gpu only.
  I'll hopefully come with some more later today.

Best regards from Paris,

Ikse

[DusX]

Perhaps this issue is related?
http://troikatronix.com/community/#/discussion/638/problem-with-multiple-projectors-and-scenes-drastic-drop-of-framerate-with-each-new-display.
Have any of you tested with the movie player direct?

[ikse]

Dear DusX, Mehdi, Mark,
I did try with the movie player direct and it did not improve the performances. I am convinced now that the limitations come from the projector itself.
I ran another series of comparisons within Isadora this time:
1. Again, the preview seems to be lighter on the cpu than a full screen projection:
    - it is clear and expected between a 1920x1080 monitor and a 320x240 preview window
    - it is clear but much less expected between an 1280x800 monitor and a 1280x800 preview window (the Isadora core is at 44% and 27% for a total cpu usage of 3 and 4% respectively)
2. Blackmagic codec works fairwell as long as the hardrive bandwidth can keep up with the load of data to be transfered (it is certainly a reason for the difficulty of looping mentioned above... the time is over but there are still some jetlagged frames coming in). It actually works very very well with SSD and ffmpeg together. Decoding is very light and well spread on the cores. However the core Isadora uses reaches 100% with four HD movies.
3. Projector, 3D Quad distort, 3D Rect project actors behave more or less the same way in terms of cpu load: 86%, 77%, and 89% on the Isadora core, 9%, 10%, and 12% overall respectively.
4. Projector actors take nearly half of the cpu load as, when removed, the overall usage falls down to 7% and the Isadora core is lowered to 36%.

The last point is striking as one may expect the projector actors to be relatively free from the cpu and handled by the gpu. In summary, I should say that the movie decoding is properly handled and it is only limited by the local software (codecs) and hardware (hardrive access and cpu power). In contrary, the display (projector actors) does not seem to work the way it should and weight too much onto the cpu (essentially onto the core Isadora uses) for large resolution movies (it is rather transparent for standard resolution movies) whereas it is expected to be gpu-handled.

I hope there is reason and a solution behind. I might still be doing things the wrong way and I will be happy to correct that.

Thanks for your help.

Ikse

P.S.: Mark, is there any reason for limiting the vertical size of the 3D renderer today?

[ikse]

Dear Mark,
Please find below an ultimate series of tests on the projector actor family. I hope it will help solving the issue raised in this thread such that Isadora eventually plays as well as VLC or MPC do.
The bug I should report lies in the projector actors once they are active in the scene whether a movie is played at their input or not.
**Projector actor
**- Showing the stage (3072x768 here) adds a little to the cpu load (from 2% to 14% usage on the Isadora core, from 1% to 2% on the total cpu usage)

• Playing 2K Blackmagic-multithreaded-decoded movie does not change much on the Isadora core (11%) but obviously adds up on the several cores (5% total cpu usage)
• Connecting the Movie player actor to the Projector actor starts loading quite a bit the Isadora core (52%)
• The load is rather linear with the number of active Projector actors: With 2 onto 2 stages, we get close to 100% (we saturate the Isadora core and it is difficult to check the linearity much longer)
  The behaviour is much more unexpected for the 3D projectors (3D Rect, 3D Quad,...) as the following post will show.

Dear Mehdi, dear DusX,
Would you have the same behaviour on your systems?

I believe it is not such a big deal with standard resolution movies or standard resolution stages, but Isadora seems to process something (even a black video) at the stage or movie resolution that takes the cpu time off.

Any hint?

Ikse

[DusX]

I am most interested by your comment that the projectors take cpu cycles even if not connected..
I will repeat your tests and report back.

[ikse]

Dear DusX, dear Mark,
Please find attached the series of basic tests on the Quad distort actor. There is no magic in the screen shots, nothing hidden, just peculiar and unexpected behaviours, at least to me, with respect to video displaying in Isadora as soon as a stage is set to shown.

Quad distort (no video input)- Without any video input, not even an input connection from a movie player actor, the Quad distort actor, as soon as it is set onto the scene, requires cpu time (~50% load on the core Isadora uses for an output stage of 1280x1024)

• Always or vid-in draw does not change the ~50% cpu load
• The number of output stages does not change the ~50% cpu load
  The fun part is here as the cpu load is not fully related to the size of the output stage, it also depends on its position and how much of it appears onto the hardware monitor
• Setting the output stage as a preview monitor 320x240 reduces the cpu load down to ~23%
• Setting the output stage as a preview monitor 640x480 doubles the cpu load to ~57% (but as you will see, the stage is partially hidden)
• Setting the output stage as a preview monitor 1280x960 gets the cpu load back to ~23% (but this time the preview window is fully shown in the monitor)
• When the preview window is partially hidden at the bottom of the monitor screen, the cpu load rises up to 57%
• When it is nicely set on the side so it is fully apparent, the cpu load gets back to 23%

I rerun the tests on a basic 2-core Dell laptop. The behaviour is similar but the actor load is not put onto a single core but nicely spread between the two cores - which questions on the side the relevance of the 32-core HP workstation for Isadora!

• Hide stage: ~9% total cpu
• Show stage + 1 preview 1440x900: 17% total cpu for 1 Quad distort
• Show stage + 2 preview 1440x900: 24% total cpu for 2 Quad distort

The forthcoming post will report on Quad distort with video in as it is what we want.
I hope these tests will help you, Mark, to figure out what Isadora is processing while, I believe, it should not.

Ikse

[ikse]

Dear Mark, dear DusX,

To continue the tests above, please find three additional screen shots, which show:

• 1 2K movie on in a Quad distort adds a little to the Isadora core (from ~50% to 60%) and a little on every other cores for multithreaded-ffmpeg blackmagic decoding (2% to 7% total cpu usage). The fps is ok (~24).
• 1 2K movie off in 2 Quad distort double the load onto the Isadora core (100%) so it is already saturated so the fps has already dropped down to ~12
• 1 2K movie on in 2 Quad distort keeps the Isadora coreto 100% but the fps drops down to ~10 while the multithreaded-ffmpeg blackmagic decoding remains identical (6% total cpu usage)

I got the same behaviour with the 3D Rect Project. The image processing, which Isadora runs as soon as the Output stages are shown, seems to be the same as the one run when a movie is on at the input of a Projector actor. It seems to be stray processing, which requires a lot of cpu load (at least on the HP workstation but has nothing to do with decoding (multithreadly performed outside Isadora by ffmpeg here) and little to do with effective video displaying (expectedly gpu-handled).

I hope these series of tests will help tracking this cumbersome processing. I do not think it is a computer-related issue but any tests, DusX,  you could run would help stating on this. I will be very happy to see this issue solved - it is obviously a bottleneck for any further development we have planned with Isadora. Time is obviously an issue. We must be ready by June!

Best of luck!

Ikse

[DusX]

I just noticed something that is perhaps related and worth taking a look at.
There is a difference between F24 and F33 as far as just having isadora open.
F24, opens and shows a cpu load of 1% of my total cpu (8% on one core)
F33, opens and shows a cpu load of 12% of my total cpu (showing ~ 50 to 80% on one core)

Obviously a lot can be done accomplished in F33, so the cpu is not maxed right when opened, however; some process is clocking cycles. So these readings may not be very reliable for performance testing the latest version.
@Mark I believe you told me somethings about a change that has a process running from launch now (f32/f33)?

In F24 the reading may still have value, however; video playback has been greatly improved in the newer versions.

[ikse]

Dear Mark,
Have you had the time for checking the processes involved in the display of the projector actors?

As we are moving along, we keep facing this issue as soon as the overall targetted stage matrix is important whatever configuration is set. 55% of a single core cpu usage is a reliable figure we get for just setting the required three Quad distort actors to address three video projectors (864x1152, 1024x576, 1024x576) and projecting black! Same thing for the 3D Mosaic actor...

I hope you will come up with the solution.

Ikse

P.S.: If the Isadora version were the issue, I would be happy to switch to a newer version. But it would mean that you have done something for the image display in the newer ones.
P.P.S.: DusX, Mehdi, did  you have the chance to give it a try? It you had or you could, I would be grateful to u for the outcome :

• A new scene
• Three Quad distort set to three different shown stages (1024x768 each to be standard or higher to enhance the effect)
• Shown stages
• Check out the cpu usage
  That's it! It works with one Quad  but the cpu usage is three time less!
  Thanks a lot.

[mark]

Dear All,

I have started to investigate this issue, and will be able to spend time on it over the next days.

I have some things to say, but first I need to get the most recent version of the pre-release uploaded. If we're going to look into this, we all need to be working with the latest pre-release. I have made some big changes to the way Isadora handles it's timing based on a long back-and-forth with an Isadora user in Uruguay. (There were several ways in which he was pushing the software... and the fact that overloading it would cause a UI freeze was one of the problems he had -- something that is mentioned here.) I should be able to put that up no later than Monday.

But I'll throw in here that I suspect this is an issue of Isadora waiting for the "vertical retrace interval" associated with the many displays you are attempting to render to. While it seems they are all working at 60Hz, they are not necessary synchronized with each other. This could mean, if Isadora ends up waiting for this retrace (which it shouldn't do, but perhaps it is inadvertently be doing so) one would experience significant delays -- even when the 3D Quad Distort has no video input. But when I made three 800x600 stages (admittedly, on one display -- not three separate displays) and added three 3D Quad Distort actors, I saw very little affect on CPU usage. So this is something we're going to need to look at carefully.

One note about CPU usage, which requires some "gory details."

Isadora has a custom-tuned function that attempts to deliver a "clock pulse" the main rendering system as accurately as possible. (The de facto method one can use for this, sending WM_TIMER messages to the application, is highly inaccurate.) This timing mechanism is important to ensure that the delivery of frames matches, as closely as possible, the target frame rate as set in to the Isadora preferences.Let's say we need to wait 15 milliseconds. The way this function works is to wait a certain amount of the desired time using a standard Sleep() function (which is also quite inaccurate) .For our example, let's say it waits 13 mS. Then, it executes a series of Sleep() commands with a delay of 0ms -- this gives other processes a time slice, but allows exit this loop with great accuracy. When you see the CPU usage high, even when Isadora is idle, that's why: it's the time spent in this this Sleep(0) function. It shows the CPU as being busy, but it's actually not that busy -- but it shows up as busy on the CPU monitor.

In any case, the first thing I need to examine to see if there are delays caused by waiting for the vertical retrace. Let me do that, and also get the new version up for you here, and we'll move forward from there.

Best,
Mark

[ikse]

Dear Mark,

This is great!
If it helps, I agree there is little change by adding Quad distort actors as long as they target the same stage. It is another story when they target a larger stage or different stages, namely when the overall stage matrix actually gets bigger. For matrices like 800x600, it does not run too badly, even on three different stages. I have just run the test: 18% of a core while idle but stages are shown, 30% while playing the movie.
As far as I could check, the cpu usage is actual, actual enough to damp the fps and slow down Isadora processing when it gets close to 100% of the used core.
I had tried the f25 but had stability issue with former patches without noticeable improvements so I came back to f24 and was reluctant so far to upgrade. I will install the f33 as you suggest and will check out the outcome. Hopefully, things will come easy and the patch might highly benefit from the improvements anyway (3D renderer etc...).

Thanks a lot for your care and your help in solving this long lasting issue.

Best regards from Paris,

Ikse

[EDIT] removed note about release and pre-release download pages because the comment is no longer true. MFC.

[DusX]

Mark I have a note regarding the UI locking up that might be of interest.
I have been able to consistently break out of a locked UI (Isadora seems completely locked except video output is still live), by using Midi to jump to a simplified snapshot.
So my process has been, to create create first in my video mixer patch a start snapshot that plays to a stage a single video (or stage color), then I proceed to create more complex mixes and creating additional snapshots. I use a midi pad control unit to trigger my snapshots.
My experience has shown that while building mixes, if I get to complex and freeze the UI (usually high cycles and low fps.. nearly equal each other) I am able to still receive midi and trigger the switch to a save snapshot (a simple one will respond most quickly), I have not test with OSC, but keyboard triggers and mouse interactions are not received.
Perhaps understanding why midi still works will help.

[mark]

Regarding the CPU usage: if Isadora is stuck waiting for the Vertical Refresh, then it would eat CPU for sure. This will be my first point of investigation.

Best,

M