Isadora movie player performances with respect to VLC

[keftaparty]

I have no experience in tracking.

Is it so process heavy ?

What video sources and capture material do you use ? This can make a big difference.

What about running it on a non rendering machine and send results to your rendering machine with osc ?

@Mark

Would it be possible to run multiple instances of isadora ?

Windows should distribute the processes on the cores ( am I wrong ? ).

For tracking etc... two instances can communicate with osc.

Mehdi

[ikse]

Mehdi,

You are right again. The tracking in itself is not that heavy. It is actually confined in one of the cores of the computer and performed by a home-programmed script, which sends via osc the position (x,y,z) and velocity (Vx,Vy,Vz) of seven tracked heads using 2 kinects. The heavy part comes when the video streams must be processed according to these parameters independently and according to different scenarii: 7 local displace given by local 3D projector, which position is given by (x,y,z) , generation of associated 3D objects. So the machine we are talking is the rendering one. The first attempts with a low resolution video looked like that www.la-lumiere-ne-s-arrete-pas-la.org

It is possible to run multiple instances of isadora but the last attempt was not very successful when dealing with high definition video streams - it crashed too quickly. I did not see any distribution over the cores. I should certainly give it another try.

Ikse

[mark]

Dear All,

I'm in the midst of the workshop in Munich. I'll need some time to answer these questions with any level of intelligence. I've bookmarked this discussion so I can come back to it next week. Sorry for the delays.

Best Wishes,

Mark

[ikse]

Dear Mark,

Thanks for the forthcoming reflections on the subject - hot and burning for us!

Enjoy the workshop in Munich!

Ikse

P.S.: If you happened to pass by Paris on your way back, I would be happy to welcome you at the neticLab www.neticLab.org

[keftaparty]

Hi Ikse,

Are you manipulating the images only with the projector actors ( 3d quad distort etc... ) ?

Are you using any other actors to manipulate the images ?

Are you using 3d renderers ? If yes at witch resolution ?

Can you post a screenshot of the stage tab of your isadora preferences ?

Can you post a screenshot of your nvidia's driver screens configuration ( The "configurere Surround et PhysX" tab would be perfect for what I want to see... )?

If you use only projector actors, you should not have any performance problems ( they are gpu based ).

On windows, Isadora was a bit capricious when using multiple stages on multiple monitors.

I see you have 3 x quadro cards, are they SLI or are they independant ?

Mehdi

[ikse]

Hi Mehdi,
I have two main superactors:
1. The first one displaces a large resolution background movie with seven 3D rect Projectors sent to a single channel to be rendered by a 3D renderer. Ideally the resolution of the seven movies and the 3D renderers should match the resolution of the background movie to keep the latter resolved (minimal targeted matrix : 2000x1600). The vertical size of 3D renderer is anyway limited to 768. It obviously freezes when approaching a decent resolution (if I remember well it saturates for something like 512x512).
2. The second one generates seven 3D objects (sphere for now) on the same large resolution background movie with seven 3D players sent to a single stage to be rendered by a 3D renderer. It works fine until I try to play either the background movie or project movies onto the 3D objects. The target obviously is to play both at the same time.

I would agree with you on the expected performances with projectors. However, the projectors might be gpu-based but I guess the associated movie players take over the cpu load. So it freezes even in the second case described above.

The three Quadro cards are independant.

I will post the screenshots and come back to you as soon as I go back to the neticLab, hopefully later today.

Have a nice day!

Xavier

[keftaparty]

Xavier,

I think one problem here is the 3d renderer. It's performance eating on heavy resolutions.

Also, when working with 3d objects, you need to take care to some facts:

-It's "complicated" to share the same 3d object to multiple graphic cards, even on multiple "heads" of a same graphic card.

I've had some issues with that kind of setups in the past.

If you want we can skype ( in french ). Send me a private message with your skype name if you want.

Mehdi

[ikse]

Mehdi,

I have arrived later than expected. Please find below the screenshots. I have not looked into the setting of the graphic cards. Any improvement is certainly welcome.
However, I don't think it is an issueso far for the 3D objects as with the seven of them, it runs smoothly. It is true that there were shared between graphics cards and within a card as they travel through three videoprojectors (2x1280800+1x12801024).

A Skype would be great to sort this out when you have some time.

Meanwhile I will post a late series of tests on cpu loading.

Ikse

[ikse]

Mehdi, Mark,

For a fairer comparison, I have installed K-Lite_Codec_Pack_930 for 32 and 64 bits. It comes with the Media player classic (MPC). It is then possible to run both Isadora with six movie players and six projectors and to compare it to six MPC while making sure that they both use the same decoding process (CPU or GPU hardware accelerated).
I tried several codex configurations (still for the same WVMC1 encoded HD movie). Please find the outcomes in the series of screenshots below (the whole series is compressed in the last file).
The file names are self-explicit. However, in summary, it appears that:
For Isadora

• There is a substantial gain to make use of the ffdshow decoder (the total CPU load falls from 39% down to 18%)
• Isadora does not stand Nvidia Cuvid (GPU acceleration) but runs with DXVA2copyback but without any gain
• Showing the stages cost quite a bit in CPU load (the total CPU load falls by half again from 18-19% down to 10-12%) which is pretty surprising for me. At the same time, the GPU load is substantially reduced (figure again)
• Confining Isadora to 8 cores does not change the overall CPU load - as one can expect
  In summary:
• Native codex 38-39%
• FFv codex 18-19%
• Nvidia Cuvid -> Crashes
• DXVA2copyback 18-19%
• FFv codex + Hidden stages 10-12%
  For MPC
  The overall CPU load is still much less than Isadora
• Native codex 8%
• FFv codex 7%
• Nvidia Cuvid 6%
• FFv codex + Nvidia Cuvid 6%

As it limits our developments, I would be happy to understand why there is such a need of CPU power for playing videos with Isadora and if there are ways to go around. The critical states are obviously exemplified in the screenshots with DXVA2copyback where there is one core (the one that Isadora uses I guess) which is overloaded. It is close to saturate and Isadora, to freeze.

Again, I am hopefully doing something wrong here. The system is as simple as can be with only the required softwares and codex to run the tests (+ the monitoring gadgets).

Thanks for your feedback,

Ikse
3a2030-video_performances_with_mpc.zip

[mark]

Dear All,

Thank you for this continued, very detailed research. I forgot to say that, after Munich, I was going to the USA, where I am now. But, my travels are almost over I return to Berlin on Monday. Then, finally, I can properly digest and look carefully at all you have to say.

Best Wishes,

Mark

[DusX]

@ikse
Did you happen to take note as to wheather or not any of these codec option allowed for looping. I have been doing a number of windows native + Qt HD test as well (not as well documented) and a big problem for me with most windows options is that the video can not be looped.

[ikse]

Dear DusX,
Looping is not an issue with the VC1 codec. It loops indefinitely under Isadora. I did face some looping issues with lossless avi-contained codec - namely there was no end trigger, so no looping possibility. The player could not figure out that the video had reached its end, whereas, surprisingly enough, it was possible to get its time duration with Media time to percent so I did generate the trigger with a time count down... not very clean.
As it appears more and more clearly to me now, the codec is not that an issue within Isadora. It can be circumvented by the Plays Windows movie formats natively so Isadora does perform as well as another player. At least the movie player actor does conform to this. The problem seems to lie in the projector actor.
I'll come with additional tests on this.
Ikse

[ikse]

Dear Mehdi, Mark, DusX,

Following some Mehdi's advices, please find attached some last results that answer two questions:

• Projecting on a full monitor screen costs more cpu load (mostly onto the core Isadora uses) than projecting on a preview screen. The gpu load is barely affected.
• Projecting several movies on a single preview screen or on separate preview screens do not change the cpu load nor the gpu load
  One could infer from these tests that Isadora runs some additional cpu calculations on the initial image matrix whatever the targeted screen may be. The purpose of such calculations are rather unclear to me as they occur only when the movie player actor is connected to the projector actor whereas the process should be handled by the gpu only.
  I'll hopefully come with some more later today.

Best regards from Paris,

Ikse

[DusX]

Perhaps this issue is related?
http://troikatronix.com/community/#/discussion/638/problem-with-multiple-projectors-and-scenes-drastic-drop-of-framerate-with-each-new-display.
Have any of you tested with the movie player direct?

[ikse]

Dear DusX, Mehdi, Mark,
I did try with the movie player direct and it did not improve the performances. I am convinced now that the limitations come from the projector itself.
I ran another series of comparisons within Isadora this time:
1. Again, the preview seems to be lighter on the cpu than a full screen projection:
    - it is clear and expected between a 1920x1080 monitor and a 320x240 preview window
    - it is clear but much less expected between an 1280x800 monitor and a 1280x800 preview window (the Isadora core is at 44% and 27% for a total cpu usage of 3 and 4% respectively)
2. Blackmagic codec works fairwell as long as the hardrive bandwidth can keep up with the load of data to be transfered (it is certainly a reason for the difficulty of looping mentioned above... the time is over but there are still some jetlagged frames coming in). It actually works very very well with SSD and ffmpeg together. Decoding is very light and well spread on the cores. However the core Isadora uses reaches 100% with four HD movies.
3. Projector, 3D Quad distort, 3D Rect project actors behave more or less the same way in terms of cpu load: 86%, 77%, and 89% on the Isadora core, 9%, 10%, and 12% overall respectively.
4. Projector actors take nearly half of the cpu load as, when removed, the overall usage falls down to 7% and the Isadora core is lowered to 36%.

The last point is striking as one may expect the projector actors to be relatively free from the cpu and handled by the gpu. In summary, I should say that the movie decoding is properly handled and it is only limited by the local software (codecs) and hardware (hardrive access and cpu power). In contrary, the display (projector actors) does not seem to work the way it should and weight too much onto the cpu (essentially onto the core Isadora uses) for large resolution movies (it is rather transparent for standard resolution movies) whereas it is expected to be gpu-handled.

I hope there is reason and a solution behind. I might still be doing things the wrong way and I will be happy to correct that.

Thanks for your help.

Ikse

P.S.: Mark, is there any reason for limiting the vertical size of the 3D renderer today?

[ikse]

Dear Mark,
Please find below an ultimate series of tests on the projector actor family. I hope it will help solving the issue raised in this thread such that Isadora eventually plays as well as VLC or MPC do.
The bug I should report lies in the projector actors once they are active in the scene whether a movie is played at their input or not.
**Projector actor
**- Showing the stage (3072x768 here) adds a little to the cpu load (from 2% to 14% usage on the Isadora core, from 1% to 2% on the total cpu usage)

• Playing 2K Blackmagic-multithreaded-decoded movie does not change much on the Isadora core (11%) but obviously adds up on the several cores (5% total cpu usage)
• Connecting the Movie player actor to the Projector actor starts loading quite a bit the Isadora core (52%)
• The load is rather linear with the number of active Projector actors: With 2 onto 2 stages, we get close to 100% (we saturate the Isadora core and it is difficult to check the linearity much longer)
  The behaviour is much more unexpected for the 3D projectors (3D Rect, 3D Quad,...) as the following post will show.

Dear Mehdi, dear DusX,
Would you have the same behaviour on your systems?

I believe it is not such a big deal with standard resolution movies or standard resolution stages, but Isadora seems to process something (even a black video) at the stage or movie resolution that takes the cpu time off.

Any hint?

Ikse

[DusX]

I am most interested by your comment that the projectors take cpu cycles even if not connected..
I will repeat your tests and report back.

[ikse]

Dear DusX, dear Mark,
Please find attached the series of basic tests on the Quad distort actor. There is no magic in the screen shots, nothing hidden, just peculiar and unexpected behaviours, at least to me, with respect to video displaying in Isadora as soon as a stage is set to shown.

Quad distort (no video input)- Without any video input, not even an input connection from a movie player actor, the Quad distort actor, as soon as it is set onto the scene, requires cpu time (~50% load on the core Isadora uses for an output stage of 1280x1024)

• Always or vid-in draw does not change the ~50% cpu load
• The number of output stages does not change the ~50% cpu load
  The fun part is here as the cpu load is not fully related to the size of the output stage, it also depends on its position and how much of it appears onto the hardware monitor
• Setting the output stage as a preview monitor 320x240 reduces the cpu load down to ~23%
• Setting the output stage as a preview monitor 640x480 doubles the cpu load to ~57% (but as you will see, the stage is partially hidden)
• Setting the output stage as a preview monitor 1280x960 gets the cpu load back to ~23% (but this time the preview window is fully shown in the monitor)
• When the preview window is partially hidden at the bottom of the monitor screen, the cpu load rises up to 57%
• When it is nicely set on the side so it is fully apparent, the cpu load gets back to 23%

I rerun the tests on a basic 2-core Dell laptop. The behaviour is similar but the actor load is not put onto a single core but nicely spread between the two cores - which questions on the side the relevance of the 32-core HP workstation for Isadora!

• Hide stage: ~9% total cpu
• Show stage + 1 preview 1440x900: 17% total cpu for 1 Quad distort
• Show stage + 2 preview 1440x900: 24% total cpu for 2 Quad distort

The forthcoming post will report on Quad distort with video in as it is what we want.
I hope these tests will help you, Mark, to figure out what Isadora is processing while, I believe, it should not.

Ikse

[ikse]

Dear Mark, dear DusX,

To continue the tests above, please find three additional screen shots, which show:

• 1 2K movie on in a Quad distort adds a little to the Isadora core (from ~50% to 60%) and a little on every other cores for multithreaded-ffmpeg blackmagic decoding (2% to 7% total cpu usage). The fps is ok (~24).
• 1 2K movie off in 2 Quad distort double the load onto the Isadora core (100%) so it is already saturated so the fps has already dropped down to ~12
• 1 2K movie on in 2 Quad distort keeps the Isadora coreto 100% but the fps drops down to ~10 while the multithreaded-ffmpeg blackmagic decoding remains identical (6% total cpu usage)

I got the same behaviour with the 3D Rect Project. The image processing, which Isadora runs as soon as the Output stages are shown, seems to be the same as the one run when a movie is on at the input of a Projector actor. It seems to be stray processing, which requires a lot of cpu load (at least on the HP workstation but has nothing to do with decoding (multithreadly performed outside Isadora by ffmpeg here) and little to do with effective video displaying (expectedly gpu-handled).

I hope these series of tests will help tracking this cumbersome processing. I do not think it is a computer-related issue but any tests, DusX,  you could run would help stating on this. I will be very happy to see this issue solved - it is obviously a bottleneck for any further development we have planned with Isadora. Time is obviously an issue. We must be ready by June!

Best of luck!

Ikse

[DusX]

I just noticed something that is perhaps related and worth taking a look at.
There is a difference between F24 and F33 as far as just having isadora open.
F24, opens and shows a cpu load of 1% of my total cpu (8% on one core)
F33, opens and shows a cpu load of 12% of my total cpu (showing ~ 50 to 80% on one core)

Obviously a lot can be done accomplished in F33, so the cpu is not maxed right when opened, however; some process is clocking cycles. So these readings may not be very reliable for performance testing the latest version.
@Mark I believe you told me somethings about a change that has a process running from launch now (f32/f33)?

In F24 the reading may still have value, however; video playback has been greatly improved in the newer versions.