Problems with Straight Alpha

DusX

Being that you see the same behaviour in the original version of Isadora and the latest, it would seem that an outside factor has changed.

Have you updated the mac OS since building the show?

dbengali

@Michel yes the premultiply actor seems to at least make the alpha levels display correctly; is it known that this is required to use straight alpha files? I'm not sure if this will always give the desired result since it would be applied to each file before compositing rather than after all compositing of layers.

@DusX indeed it does seem to be external to isadora but I'm wondering if it is a known issue. I don't *think* there has been a major release update to OSX on the relevant computer since the last time that I saw straight alpha interpreted correctly, but there have definitely been minor release updates installed.

DusX

The premultiply state is likely the only factor that could have changed, so perhaps how QT/AV foundation handles this has changed in one of the point updates.

Either way the premultiply state can be confusing to users, and is not very clear within Isadora. I just replied to another topic where this seems to be the issue as well.

dbengali

Got it, yes, unfortunately I have verified that this is not a solution. The Premultiply actor seems to premultiply with black, which creates a black fringe around the alpha edges of content, changing its appearance and being particularly noticeable when files are layered on top of each other. I got a hold of a computer on an older Mac OS, and see that the straight alpha files look correct on it, and one can compare the look with and without premultiply and see the difference. This fringing is the behavior that straight alpha channels are meant to solve, and have in the past been able to do. Is there any way to get a file with a straight alpha channel to be interpreted correctly again?

Michel

@dbengali

This would best be handed over to @mark

Best Michel

mark

Dear @dbengali + All,

@Michel brought me in to take a look at this. As far as I can tell, alpha works as expected. To let you try it yourself, here are download links for two test files that I use when testing alpha in Isadora:

Apple Pro Res 4444 Straight Alpha (the words are transparent, and a gradation from 0% to 100% in the bar below the words)

Apple Pro Res 4444 Red (50% red background, with rectangular areas at 20%, 40%, 60%, and 80%)

When I render these movies over the output of the Video Noise actor as a background, I see the following, which is exactly what I would expect to see.

Both movies were created in Final Cut Pro X by placing the mask image on a track above the video track, and then compositing the two together using the "Stencil Blend Mode" as shown below.

You can download the alpha mask image I used for the first movie by clicking here.

I don't have Premiere or After Effects, so I can't try their alpha modes to compare. But I think if you try these movies they will render as you would expect.

Perhaps you can open these test movies in your video editing software, and compare the embedded alpha settings to those you are using?

Best Wishes, Mark

dbengali

Thanks @mark for jumping in with these tests. Here is some more info:

I get the same correct-looking behavior when I use your rendered files on my machine

When I pull them into After Effects (where I am building content), they report straight alpha. When I render them back out of after effects, they still seem to work as expected.

But, when I use content rendered out of after effects that was created natively there, straight alpha does not work the way that it used to.

I have attached some screen grabs. I have tried a number of codecs --

From After Effects: ProRes4444 with straight and premultiplied alpha, hap-alpha with straight and premultiplied alpha, and also, png's with alpha rendered out of after effects, encoded to hap alpha using the HapInAVFoundation framework (this last one is my usual method)

It should be apparent in the screen grabs that anything with straight alpha in those test scenes shows up wrong. Along with tests of the content I first noticed the issue with, I tried a simple comp that is just a white solid with some masks with various opacities and feather amounts

There is a screen grab of what the Premultiply actor does, which is to add black fringing.

This is all on OSX El Capitan 10.11.6, Isadora 2.5

I'll try to get a hold of the downgraded computer again, which was on 10.9.5, but basically I don't think I saw these issues there, although I didn't get to run through all the test cases.

Screen grabs are below, and here is a link to a zip file with a patch file and the test media I am working with: Test files

mark

@dbengali

Can you render out a 1 second clip that shows the problem as Apple Pro Res 4444 and post it somewhere so that I can get it?

Also, can you please report the exact settings you're using in After Effects when rendering with alpha so that I can work with someone who has After Effects and replicate the situation?

Best Wishes,
Mark

dbengali

@mark yes definitely. The link from my last post should take you to a zip file that has a file like that, and the patch from the screen grabs above. Here's a copy of the link with full text: https://www.dropbox.com/s/fpv6p0hq4af4ylu/alphaTestShare.zip?dl=0

Here is a link to the simplest file within that archive that meets the spec of 1s long, Pro Res, straight alpha: https://www.dropbox.com/s/cd6un528lhh61kx/AlphaBlur_proResStraight_AEmasks.mov?dl=0

After Effects (version 2017.2) output module settings:

Format: QuickTime

Channels: RGB + Alpha

Depth: Millions of Colors+ or Trillions of Colors+ seem to give the same result. Usually I am working in 8bit project depth, so Trillions does not apply.

Color: Straight (Unmatted)

Format Options - Video Codec: Apple ProRes 4444

Thank you!

all best,

David

mark

@dbengali

Here's some comments from an After Effects expert I know who looked into another user's issues with alpha.

"AE interprets the footage as Premultiplied on Black. It's happened to me before that when changing in between programs, e.g. Nuke and AE, the alpha channels get messed up or misinterpreted.. Nuke interprets the same alpha channel differently, so sometimes a matte choker is required to solve problems with the edges or premultiplication."

Then there's this comment from someone at Adobe in a Creative Cow thread:

"The oversimplified answer about what to use is that straight channels are better but premultiplied channels are compatible with more software."

This item from Adobe's After Effects help might be the "clincher."

"Straight channels retain more accurate color information than premultiplied channels. Premultiplied channels are compatible with a wider range of programs, such as Apple QuickTime Player."

Given this last statement, it might seem that your only option when working with Isadora (which relies on AVFoundation/QuickTime to play movies on Mac OS) is to render the files as "Premultiplied with Black" in AE.

That said, I'll look at the straight alpha movies you provided (thank you!) to see if there is some way Isadora can handle them properly.

Best Wishes,
Mark

mark

@dbengali + All,

One further note here: I used a handy (free) program called Metadata Hootenanny to examine the metadata in the video files you provided, and compare that meta data to the files that provided. There is no information encoded into the file that specifies whether it is straight alpha or premultiplied. Thus the necessity of the "guess" feature in After Effects. It would seem that there is no way for Isadora to discern whether or not the file is straight or premultiplied. In AE, the software has the opportunity to scan the data for the file and make a guess. I suppose this is possible in Isadora as well, but it is not something I want to spend time on unless it is absolutely necessary.

I will be monitoring this thread to see what other users have to say. But at the moment, my previous suggestion about exporting as "premultiplied with black" would seem to be the route to get Isadora rendering the alpha channels correctly.

Best Wishes,
Mark

dbengali

Thanks @mark for all of this!

Well, I guess I would have to say that rendering out as premultiplied with black is not actually a solution, because it adds a black "halo" to objects. Considering two movie files of white objects with soft edges composited on top of each other, there should be no discernible boundary between them where they overlap, but if the alpha is premultiplied with black, there will be a black edge in between

The other thing that is strange is that this used to work in the past. I'm not sure if it is tied to an OS version or update level, but I was able to see straight and premultiplied alpha files correctly detected and rendered by isadora on an older computer running Mavericks (unfortunately I can't get my hands on that machine now). I wonder if something has changed with AVFoundation?

Another useful data point: When I pull the various test files from this experiment back into After Effects, it does not seem to need to guess. It is (correctly) identifying the files rendered as premultiplied and straight alpha with the correct type of alpha channel as per how they were rendered, as shown in the AE project window. For what it's worth, The test files you created are detected as having straight alpha. In other cases (for example, importing TIFF files from screen capture), After Effects says it can't identify the alpha channel and offers the opportunity to guess. So there seems to possibly be something about these files that does identify the type of alpha channel, but it is a bit of a mystery where that might be.

One other thing I noticed which is interesting -- If I play that "AEmasks" test file in Isadora and hover my mouse over the connection from the movie player to the projector, the thumbnail correctly displays the content with checkerboard background appearing as expected in the transparent areas. It is only in the media window and the stage that it is showing up as all white.

best,

David

mark

@dbengali

Well, I'm not quite sure what to do here. If one were to scan through the entire movie file, one could determine if the file was using premultiplied alpha or not. If there was a transparency in the first frame, then one could discover this in the first frame. But it isn't a given: if everything were full alpha until the last frame, then only the last frame would tell you whether the clip was straight alpha or premultiplied. Given that Isadora calls up movies interactively, scanning the entire clip to figure this out is not a viable solution.

From some research, it seems that QLab only accepts straight alpha. My guess is that we're assuming premultiplied, they are assuming straight, because there's no info to tell you in the movie itself which one it is.

Let me have a talk with the team about this. Perhaps our assumption should be straight alpha as well. Or maybe we have to offer an option in the Movie Player to force it one way or another. I put a shout out on FB to see if any of my programmer friends know of a some API call in AVFoundation that would allow me to determine straight vs. premultiplied alpha. But my searching indicates no such all exists.

Best Wishes,
Mark

mark

@dbengali

In investigating this further, it seems clear that adding an option to specify the alpha handling for movies in needed in the Movie Player. I've added this in our internal build, and it works as it should, allowing David's clips to render properly. There really doesn't seem to be a way to determine if a file has straight or premultiplied alpha without scanning the frames in the file... something that After Effects would need to do anyway as part of it's rendering process. (And my After Effects expert tells me that this process is not foolproof in every case.) Given that FCP X and DaVinci Resolve also require that you specify the alpha handling manually, it seems that Isadora follow this model.

Best Wishes,
Mark

dbengali

Thanks @mark!