[LOGGED] effect mixer & alpha channel

[Woland]

I've logged this feature request for internal tracking, but I don't think it will be possible to include this for the initial release of Isadora 3.

Best wishes,

Woland

[artoo]

@woland

coming back...

is anybody here found a solution for blending streams without using the effect mixer (that don't keep alpha)?

FFGL64? or GLSL?

still looking for solution...

artoo.

[Juriaan]

@artoo

Are you still on Isadora 2.6.1 or are Isadora 3.x.x solutions also accepted ;)

[artoo]

@juriaan

oh yes i forgot to mentioned it... the problem is for isadora 3 (i've got cool ffgl32 plug-in to blend streams with alpha layers)

... and i've just found and re-built a cool shader that applies all 25 photoshop blends but without the alpha chanel.

keep searching...

[Fred]

@artoo adding the alpha to the shader should be pretty simple. Want to share the code and I'll have a look on a few days when I have some time.

[Kathmandale]

@artoo @Fred likewise, happy to have a look at that as well. Or, if you fancy having a go yourself, there's a guide I put together for doing exactly that on this thread: GLSL - baby steps (Or "How I Added Alpha Support to Some of TT's Shaders")

I'd quite like a look at that 'all 25 photoshop blend modes mixer' though, so please do share if it's okay to do so.

[artoo]

@Fred , @Kathmandale

thks guys... i will have a look to the guide...

here is the shader i've adapted to use in isadora:

GLSL-FX-blender.iua3

(yes it's a user actor but it's simpler for me to put them in my global toolbox)

artoo.

[Woland]

@artoo said:

(yes it's a user actor but it's simpler for me to put them in my global toolbox)

Any .txt file formatted properly as a GLSL Shader that you put into the folder where Isadora looks for GLSL Shaders on your computer will show up in Isadora in the GLSL Shader actor bin.

[artoo]

@woland

yes i know that!

in fact, it's a kind of routine for me... 'cause (i think) when you want to change someting in the GLSL code (i do it a lot!!) and if it is not in a user actor, you loose the changes if you don't modify the .txt file too. So to prevent this i put all my code in user actors and if i change someting, i just have to save the actor.

am i wrong?

[DusX]

@artoo said:

am i wrong?

 No you are right.

I also generally wrap things into user actors, it provides a few conveniences. Especially for JS scripts.

[artoo]

@Fred @Kathmandale 

sorry i send a wrong one...

here's the one:GLSL-FX-blender.iua3

[Fred]

@artoo Hmm, I went to tackle this and got to an interesting issue, how would you expect these blends to affect the Alpha? Should I always add the alphas together no matter what, so that the output has a combination of both alphas? Should there be a switch on which alpha to use? Kind of cool issue considering how each of the blend modes work, some will leave you with coloured pixels more from each layer.

@mark have you ever encountered or thought about this? It does not seem there is a logical way around this, but it is a case by case issue.

@artoo how would you imagine it worked when the colours are blended together, it seems that the original alpha is not always relevant any more. I could also just check the result and if the pixel is black give it an alpha of 0 and if not give it an alpha of 1? Or if it is not black and alpha of one of the sources? Or of both added together? 

Fred

[artoo]

@Fred :

well...my english is too poor to explain exactly what i expect but here is a picture of my problem:

as you can see the blend made by the effect mixer don't keep the alpha channel (it's lost at the effect mixer's output), where the one made by the ffgl32 plug-in keep it.
i'm just trying to have this possibility in isadora 3. (ie playing movies with no backgrounds and blend them without the addition of a black background)

i understand the problem of the "case by case issue" and how to deal with others kinds of blend (multiply for exemple), but i'm sorry i can't go further with my technical english!!!

anyway thanks a lot.

artoo.

[Kathmandale]

@artoo Just like @Fred I started to have a look and quickly got down to the 'how do you want to apply the alpha in each separate blend mode' question.

I like the idea of having a switch that selects, for example:

1. No alpha - treat alpha as black and run the blends as they are currently

2. Alpha from video A - run the blend mode then re-apply the alpha from video A

3. Alpha from video B - run the blend mode then re-apply the alpha from video B

4. All alpha - run the blend mode then add together the alpha from A and B and re-apply it to the blended video

I imagine in most cases (including in @artoo 's example) option 4 would be the preferred way to do it, but I like the creative possibilities of including the alpha channel in the blend mode calculations. Not quite sure how I'd approach handling it differently in each blend mode, my ideas are all around re-applying the alpha post-blend.

[mark]

@fred said:

@mark have you ever encountered or thought about this? It does not seem there is a logical way around this, but it is a case by case issue.

 @artoo  @Kathmandale  @woland 

As Fred has pointed out, considering how one might make use of the alpha channel in the Effect Mixer is a bit of a strange thing to consider.

The biggest issue is this: the Effect Mixer assumes that the two video streams do not use alpha. It actually premultiplies the input colors, assuming that they are straight alpha, which works as expected as long as the alpha is full -- but this is actually a bit of a bug if the streams actually do have non-full alpha pixels.

Now, for a mode like "add max" it probably makes sense -- whichever alpha is greater "wins" and that's the one passed to the output.

For something like "add ovr" it's bit more weird to consider. What happens if you add two alphas and they "wrap around" to a lower value (e..g, 192 + 128 = 320 which ends up being 64 after the wrap around.)

For something like "and" which performs a binary and operation on the two values, I don't know what would happen to the alpha. Might be pretty bizarre, which is sort of the usefulness of the Effect Mixer anyway.

In any case, I have prepared a patch for you to consider. It recreates the 'addmax' and 'addovr' modes but with two different ways to control the alpha.

In Scenes 1 and 2, the GLSL shader offers the 'alpha 1 amt' and 'alpha 2 amt' inputs. These allow you to mix the amount of alpha from the first image and from the second image. By changing these inputs, you modulate the amount of alpha transferred from the two source images to the output.

In Scenes 3 and 4, the GLSL shader simply selects the maximum alpha value from the two images.

Note that for these examples work properly, the video input to the GLSL Shader actor must be unpremultiplied. I've used the Alpha Tool actor to do this.

See if this inspires any ideas for you.

Best Wishes,
Mark

effect-mixer-alpha.zip