[INFO] Why Don't the 3D Player and 3D Model Particles Offer Transparency?

[mark]

I wanted to answer a question that came up yesterday that has been asked before: why doesn't the 3D Player offer an 'intensity' input like the Projector actor?

There's a substantial reason behind this seemingly missing capability: to allow transparency when rendering 3D models, you must sort every triangle that makes up every polygon of every model in the scene into the correct z-order based on the camera position. With an interactive system where models can be repositioned every frame, doing this operation except with the simplest possible models proves to be so slow as to make the frame rate unworkable.

You might think the 3D Model Particles could be a solution because it offers a fade time. But regardless of what 3D actor you use in Isadora, there will be problems rendering 3D models with an alpha less than 1.0, or with texture maps that have use images with transparency.

Here are the TL;DR some details if you care to know about them.

Start by opening the attached patch (3d-alpha-issue.zip). It uses two 3D Projector actors to render two rectangles, one rotated 45° to the left, the other rotated 45° to the right, so that the two cross over in the center.

3d-alpha-issue.izz

Watch what happens when the green rectangle has a high alpha (i.e., it's more opaque) and the red rectangle has a low alpha (i.e, it's more transparent.) You'll notice that you never see the left side of the green triangle. This is because it is occluded by the red triangle. even when the alpha of the red color is low. Why? Because all of the polygons of the red triangle are drawn first, and all the polygons of the green triangle drawn second. That means that the polygons of the red rectangle obscure those of the green triangle, even though the the left part of the red rectangle is behind the left part of the green rectangle in 3D space.

Here's a snapshot when the alpha for the green rectangle is 100% and the alpha for the red rectangle is 25%. You should see the green rectangle to the left of the intersection, but, because of this sorting issue, you don't.

If all of the 3D Objects are fully opaque, OpenGL can automatically ensure that everything looks as you'd expect. (You can see this by disconnecting the wave generators and setting the alpha input of both Color Maker actors to 100%.)

But with transparent objects, you'd need to accumulate all of the individual polygons from all of the models being displayed by all active 3D Players and sort them; you need to do this because you must ensure that the polygons that are furthest away from the camera are rendered first and the ones that are closer rendered last. Otherwise transparency doesn't work.

Here's a similar example using the 3D Model Particles actor with a texture map that has transparency. 

3d-model-particles.zip

You'll see the problem is the same. The image on the right is partially faded out, but you can't see the right side of the other rectangle.

Years ago I attempted to make an algorithm to do this, but it proved to be so slow that I gave up on it.

So, while I can understand the usefulness of this capability (and I have desired it myself) I cannot guarantee that we'll solve it because doing so would require a huge amount of programming time, and other features in the pipeline are simply more urgent.

Best Wishes,
Mark

P.S. Just to emphasize how non-trivial this problem is, here's a 47 page scientific paper on the topic.

[Woland]

@mark

Okay, but then why does this work?

2020 - 3.0.7 - Fading 3D Objects.zip

[mark]

@woland said:

Okay, but then why does this work?

Nice try. But unfortunately, in fact, it doesn't work. You just can't notice the problem in your example patch.

Here's a much simplified version, where the first cube doesn't move and the second cube rotates 2.5 to +2.5 degrees along the y-axis. Instead of the transparency working as you expect, you can see the surfaces snap in front of each other.

2020 - 3.0.7 - Fading 3D Objects - Faiilure Example.zip

Best Wishes,
Mark

[Woland]

@mark said:

Here's a much simplified version, where the first cube doesn't move and the second cube rotates 2.5 to +2.5 degrees along the y-axis. Instead of the transparency working as you expect, you can see the surfaces snap in front of each other.

 

Still, better than nothing I suppose.

[bonemap]

@woland

transparency and depth sorting!

[mark]

@bonemap

Well, perhaps it looks fine which is great. Lucas' patch also seemed to be OK visually (though if you watched closely, you could see errors.)

But I can tell you that your patch does not sort multiple models from multiple 3D Model Particle actors at the polygon level. You may be managing the individual models this way, but that doesn't deal with the polygons within the models. It is not a general solution.

My simple example above proves this to be the case.

Best Wishes,
Mark

[bonemap]

@mark

Sometimes limitations become a catalyst for some kind of innovation. In this demonstration patch I am timing the change of the layer hierarchy to allow the 3D Model Players to move in front or behind. It represents a compromise, but in seeking to work through the challenge interesting and generative qualities emerge, where they would not if the path was easy.

It is really nice to have your explanation as to why things are the way they are. It is very special!

Best wishes

Russell

[Fred]

@mark is it difficult to be able to feed the output of 3d actors into a glsl actor (not as a flat image but as a set of vertices)? Then using shaders this problem could have another chance at being solved without needing to change Isadora too much.

[mark]

@fred said:

Then using shaders this problem could have another chance at being solved without needing to change Isadora too much.

From the conclusion of the paper I quoted above:

Neither of the two additional algorithms have the required qualities of a general transparency sorting algorithm and can only be used situational or for limited purposes. Instead the hope lies in the coming algorithms, such as hybrid transparency or adaptive transparency. But even with algorithms performing as well as the per object sorting and without the same distinct visual artifacts other kinds of algorithms will be used. Even if the algorithms only increase the speed in a special cases where the visual quality is not as important, since in real time 3D rendering speed is everything.

Also read this long reply to a similar question about the same problem in Unity.

Or this explanation of the topic from the Khronos group who maintain the OpenGL standard.

If you can find me code that will sort every polygon in the scene in real time, I'm love to see it. But it really is not a trivial problem to solve.

Best Wishes,
Mark

[Fred]

@mark Thanks, that was an interesting read and explains some other anomolies I have seen. I realise now the reason I was able to solve this with a shader is that I was only using 0 or full transparency when I did this.

It would be cool to be able to run shaders on 3d objects though..

[Armando]

@mark Thank you al for the detailed explanation. I get why I just encountered this problem.

Again, a thousand thanks