Isadora, 2013 Mac Pro, dual GPUs and CPU isssues

[dritter]

I have spent many weeks dealing with the following issues and would greatly appreciate if anyone can answer the following questions.
I am working on a new project with Isadora 2.6.1 on a 2013 MacPro (6-core 3.5 GHz, High Sierra) with dual Radeon Firepro D500 GPU’s.  I am controlling 90 texture-mapped .3ds objects(spheres) controlled by 3 Kinects (through two networked Mac Book Pros) on a 4Kx1K stage that is output to 3 HD video projectors, and I am having problems getting the frame rate above 14 fps. I checked the GPUs using OSX’s Activity Monitor >GPU window, and it indicates that Isadora is using only one GPU (see attached screen grab). How can I get Isadora to use the second GPU?

Also, two related questions:

Would  a 2013 Mac Pro with the Radeon Firepro D700 GPU increase the frame rate significantly?

Is it correct that a 2013 Mac Pro with a 6-core 3.5GHz CPU will run Isadora faster than a 8-core 3GHz Mac Pro?
Many thanks
Don

[Fred]

@dritter said:

Firepro D700

 Take a look at this thread:

https://community.troikatronix...

Apple never added any special magic to the OS to just jump out and use 2 gpus, most software manufacturers in industries that could, made their software able to take advantage of it, so rendering video from some versions of premiere or FCPx will really use both, but not a lot else. To be clear, your case is a little worse, the second GPU on your macpro is really only meant for compute purposes, so something like openCL rendering, not real time stuff, it is not even connected to the outputs. The D500 is, unfortunately these days a pretty low powered card, and maybe not fit for the job (I would not use it, but that is me). if you are curious here is a link to a video from an apple developer conference about the 2013 mac pro, at 3 mins and 2 seconds he drops this info clearly.

http://devstreaming.apple.com/...

Isadora is not optimised for multi threading and will not take advantage of high core counts so in many cases higher speed will be better. Some operations that Isadora hands off to the operating system (I guess loading a movie), or if you have one of the kinects on this machine it can eat a few of the idle cores.

Is there an option to change your playback machine? If so comparing the GFX cards even a GTX 1070 should beat this performance and if the majority of the slow down is with 90 3d objects I would guess the GFX card is your bottle neck. 

If not then the next process will be about looking for bottlenecks and trying to optimise things.

I would start with the "90 texture-mapped .3ds objects(spheres)". What is the texture you are mapping to them? Does each sphere have a unique texture? Is the texture a power of 2 size. What is your texture format and size? Maybe you can reduce the size of the sphere models (number of vertexes) and greatly reduce the size of the textures you are using with them (try go to half their current size, and halve the number of vertexes on the sphere or less for both if it still looks ok). Even when the textures are jpgs they need to be decompressed and uploaded and they take up space, but much more than the jpg files and the total of that needs to fit in your GPU memory, and this case the GPU memory of a single card.

[dritter]

Thank you for your suggestions Fred, I know that a Windows machine would perform better, but I am still traumatized by a Windows NT machine from the late nineties that was my first high quality video editing system (total cost $25,000 CAD). That system took up hundreds of hours of my time dealing with non-stop technical issues, and even today I get a rash over my entire body when I hear the phrase "IRQ conflict." Although this Mac Pro is slower than what a PC can do, it is very stable. I still have the option of swapping this computer for a Mac Pro with a D700 GPU for a few hundred dollars more.

86 of the ninety objects are all the same .3DS file (sphere with 60 segments), not copies. The other four object are cone shapes. I have tried reducing the number of segments/vertices in this object, but it seems to make no difference on the frame rate. The texture maps are a single 400x200 png file, which is used for each object. Do you think it would help if I made 90 separate .3ds files with 90 separate PNG maps? What do you mean by "power of 2 size.'

sincerely,
Don

[jhoepffner]

@dritter @fred

Hello,

Dont be afraid of windows... After 30 years on MacOs I just bought a PC laptop with a decent GPU (nvidia 1070) and its really another league. At the moment I find Windows more stable. I have many problems on MacOs with Cuda updates, quicktime and other.

Concerning 3D I dont completly agree with Fred, mesh texturing is made on GPU but vertex creation is made on CPU. If you use 90 time the same object, perhaps you can use 3D Model Particles where Isadora use only 1 model and make many instance. You can put diferent textures on each instance using texture map, where each instance use a different part of the texture. Unfortunately Isadora doesnt recognise 3D texturing but with some management, you can produce 90 objects without charging the CPU. But particles allow only instancing with particles, so you cannot have not moving object.

We have to ask Mark to create instance not on particles but on another mesh, using vertex as positions for instances.

[bonemap]

@dritter 

Hi Don,

I have a very similar project that I have been developing for some time. I have found that using the 3D Model Particles and specifically the ‘group index’ parameter can produce much more efficient representation of multiple 3D models through a single .3DS file. I have had consistent higher frame rates, using a 2015 MacBook Pro, to precisely control 30 - 40 spheres simultaneously through the 3D Model Particles actor. These are 3D files where some initial spatial arrangement of the individual mesh geometry is done in the modeling software and each sphere remains a separate object in the 3D scene before exporting to .3DS file. Isadora can then read each sphere in the file as independent geometry and by timing the 3d Model Particles parameters: ‘group index’ and position x,y,z to synchronise ie. linking all the parameters to the appropriate frequency of a Pulse Generator through a configuration that 'selects' the group index number while simultaneously delivering the position coordinates for each sphere to the x,y,z parameters. In doing this the Kinect OSC motion data can be scaled and linked to the x,y,z of the 3D Model Particles actor through a timed/synchronized delivery of position data at the moment a mesh 'group' instance will spawn/birth. In this way all of the spheres associated with skeleton tracking are aligned to their own mesh 'group' within a single .3ds file. In my experience this is much more efficient and significant frame rate improvement is possible.

I have a couple of the MacPro 3.5 ghz around at the moment, I am unsure if I will have the opportunity to run up my project any time soon but perhaps over the next week or so.

I have also made available a tutorial guide to using the ‘group index’ of the 3D Model Particles actor. It is the third tutorial of the 3D Particles tutorial I distributed to the forum a couple of months ago. The trick is to find the right calibration between the Particle Count and the Hold Time (including Fade-in and Fade-out times) so that the representation of each sphere appears to persist, even though it is continually timing out and spawning new instances, based on synchronizing the frequency of triggers at the 'Add Obj' parameter.

There is a quick screen capture of a prototype system using this technique here. The screen capture shows a prototype patch that simultaneously routes Kinect skeleton data to three different systems , a configuration of 3D Lines actors, a 3D Particles actor and a 3D Model Particles actor.

Best wishes

Bonemap

[Fred]

@jhoepffner if there are less vertexes is there less for the gfx card to do though?

[Fred]

@dritter for po2 texture sizes this article has some info about game engines, but the hardware we use is the same.

https://www.katsbits.com/tutor...

basically GFX cards are optimised to deal with some kinds of math over others. So make the width and height of your textures powers of 2, so 128 * 512 or 256 * 512 would be the closest for you.

[dritter]

thank you Jacques, ok, I am not really afraid of Windows, I just don't have time to get familiar with a  new machine and OS. I still use  windows XP/Bootcamp on my MacBook Pro for 3DS Max.

Don

[dritter]

Thank you Fred, ok, I will try a map of that resolution. I just tried loading copies of all  the 90  .3ds files and their png maps, and also another 180 individual .png files that were used for particle animation. There was no increase in the frame rate.

regards,

Don

[dritter]

@bonemap  thank you, I will try that approach tomorrow.

sincerely,
Don

[jhoepffner]

@fred

No, there is less for the CPU.

The render pipeline is like that:

basic geometry is produced in CPU, if your mesh is still build, like with 3ds files, there is little work to do but if you multiply the same mesh by 90 with different position for each one, there is a bottleneck passing information from CPU to GPU. Its the reason why 3D Model particle is efficient, you only pass 1 mesh and the GPU multiply it with information coming from particles emiter (can be calculated in GPU).

In the GPU

- the received vertices are recalculated in the "vertex shader", adding principaly the point of view, aka camera, to define where are the vertices in the final frame and passing it to the next shader

- the "geometry shader" (not mandatory) can multiply the vertices, making a cube from a point, transforming 1 vertex to 8 vertices

- the "fragment shader" compute what color have each pixel, using texture and uvmap and mixing with information about lighting and texture reflexion. The output is your stage.

@dritter 

There is a huge difference in stability and ease of use between XP and 10. Even in bootcamp, if you have a good GPU, you can feel the difference with 3D studio Max.

[dritter]

@bonemap

Bonemap, I tried to open your file Geometry_Groups.izz using Isadora 2.6.1 on a Mac and I get the message that it cannot be opened because it was created with a newer version of Isadora. Which versions should I be using?

thank you

Don

[dritter]

@bonemap

Bonemap

I downloaded Source Three.zip again and now it works.  Looks good! thank you.

Don

[bonemap]

@dritter

Hi Don,

It is a v2.6.1 file.

I am happy to hear that you have accessed the file.

Best wishes

Bonemap

[dritter]

@bonemap

wow, very innovative solution using groups in the .3ds files. Runs at 38 fps on my Mac Pro

[bonemap]

@dritter said:

@bonemap
wow, very innovative solution using groups in the .3ds files. Runs at 38 fps on my Mac Pro

Hi,

Great to hear that you are getting improved results. We don’t have to give up on the Mac yet!

Best wishes

Bonemap

[dritter]

@bonemap
Bonemap, sorry for the misunderstanding. I meant your patch runs at 38 fps.  I have not yet tried your approach on my patch but I will over the next week.

I am also considering testing my project on a Dell PC  and GeForce GTX 1080. I will be posting some questions about that in the next few hours.

regards,

Don

[bonemap]

@dritter

Hi Don,

OK well it will be interesting to hear about the comparison. There has been a noticeable move to the PC/Nvidia platform and it is entirely understandable.

Best Wishes 

Bonemap 

[dritter]

@bonemap
@fred

Bonemap and Jacques, thank you for the information on using Isadora under Windows. I am now considering this option as my tests on a 3.5 GHz Mac Pro with a D700 GPU did not provide a significant increase over a Mac Pro with a D500. I would greatly appreciate if you or another user could clarify my findings  regarding using Isadora under Windows:  

  1. An Isadora patch created under OSX can be opened in Isadora on a Windows computer, only the container and codec for the video files must be changed.

  2. Continuous multi-channel OSC data can be sent between a Mac and a PC with no problems. (Mac->router->PC)

  3. The best performing/suggested video format for Windows is the AVI container using the HAP codec. (Is it possible to convert a ProRes MOV file to a HAP AVI file on a  Mac, or can this only be done on the Windows side? I have no video software for Windows.)

  4. I am considering a Dell with a Xeon E-2146G, 6 Core 3.5 GHz CPU, 16 or 32 GB RAM, GeForce GTX 1080 (8GB), NVMe SSD , and 460 W power supply. I read other Isadora users are using this GPU or is there something better? Do you think would this setup outperform the 3.5GHz Mac Pro with the D700 GPU?    

Many thanks!

Don

[Fred]

@dritter 

  1. An Isadora patch created under OSX can be opened in Isadora on a Windows computer, only the container and codec for the video files must be changed.

Yes this works fine, as long as the versions are matching, or the move is one directional.

  2. Continuous multi-channel OSC data can be sent between a Mac and a PC with no problems. (Mac->router->PC)

OSC and networking work great on PC and MAC

  3. The best performing/suggested video format for Windows is the AVI container using the HAP codec. (Is it possible to convert a ProRes MOV file to a HAP AVI file on a  Mac, or can this only be done on the Windows side? I have no video software for Windows.)

At the moment this is a little difficult, depending on the content and resolution high data rate H264 may be suitable. Otherwise the conversion is a little annoying. 

  4. I am considering a Dell with a Xeon E-2146G, 6 Core 3.5 GHz CPU, 16 or 32 GB RAM, GeForce GTX 1080 (8GB), NVMe SSD , and 460 W power supply. I read other Isadora users are using this GPU or is there something better? Do you think would this setup outperform the 3.5GHz Mac Pro with the D700 GPU?  

The 1080 is great, you may find a cheap 1080TI these days and it is worth the extra, either are orders of magnitude better than the D700. The CPU is also great, and yes this setup will leave your trash can in the dust.