What does the death of Quicktime mean for our existing Isadora patches exactly?

[DusX]

@bonemap
Currently, if the Movie Player can play the file in 'Performance' Mode and the PB engine read 'AV' the file is supported by AV foundation and you are fine.
This includes (but is not limited to) HAP, ProRes, h264, PhotoJpeg

[mark]

@bonemap said:

files encoded with certain codecs will no longer function correctly?

For MAC OS

Short version: Mojave still allows 32 bit apps, thus we can continue to support QuickTime. As long as 32 bit processes can run on MacOS, we can continue to support Quicktime.

Longer, detailed version:

1) While Apple said that Mojave would not run 32 bit applications at all (e.g., QuickTime) that appears to be false. The beta runs 32 bit stuff fine, though perhaps complaining with the "this application is not optimized for your system" alert the first time. Given that they usually announce at the end of September, I reckon the release build of Mojave will still allow 32 bit applications.

2) On MacOS, Isadora uses "faceless background applications" to play movies. This allows Isadora -- whether compiled as a 32 or 64 bit app -- to play movies with both AVFoundation (the new system, 64 bit only) and QuickTime (the old system, 32 bit only.) So, the new 64 bit Isadora will continue to support QuickTime until such time that Apple really, truly disallows and 32 bits apps.

3) Once MacOS dumps its 32 bit capability, you will most certainly lose certain codecs, with Animation being perhaps the most critical one. But also ancient stuff like Cinepak, etc.

For Windows

As you can see on this link at the Dept. of Homeland Security of the USA, QuickTime for Windows is considered to be a "dangerous" software. Some public universities are prevented from installing QuickTime for Windows because of this warning, and others users have removed it because of (over dramatic) warnings like this. All of this, combined with the the that QuickTime is buggy under Windows and cannot be run in multiple threads, makes its continued use in Isadora extremely problematic.

Thus, 64 bit version currently in development supports only Windows native containers, e.g., AVI, WMV, MP4 etc. I have investigated using FFmpeg to support .MOV playback, but doing so is essentially illegal if you don't pay the patent owners of for the H264, JPEG, and ProRes codecs. We will give you tool to convert .MOV files to AVI files -- this will work only if you have QuickTime installed on your computer.

Common Question: Why won't Isadora 3 support FFmpeg?

The following quote FFmpeg's site should give anyone pause:

Q: Does FFmpeg use patented algorithms?

A: We do not know, we are not lawyers so we are not qualified to answer this. Also we have never read patents to implement any part of FFmpeg, so even if we were qualified we could not answer it as we do not know what is patented. Furthermore the sheer number of software patents makes it impossible to read them all so no one (lawyer or not) could answer such a question with a definite no, those who do lie.

That's a risk I'm not willing to take yet, even though other companies seem to be doing so (Resolume, MAX, etc.) All it would take is one lawsuit to kill Isadora completely.

I will revisit FFmpeg as a possibility after Isadora 3's release. Perhaps if the end user installs the codecs, we'd be safe. That might be one route

I hope that clarifies things.

Best Wishes,
Mark

[bonemap]

@mark 

Thank you for providing the additional detail.

Best wishes

Bonemap

[jtsteph]

Hi, Mark.

I am a windows user and have found that your quicktime engine is the most stable and processor friendly option when interacting with video - especially when using HAP. I am wondering what your plans are for the shift to windows native containers. Will you be supporting HAP in AVI? Is your windows video engine getting an update to allow for more predictable interaction?

I look forward to your thoughts on this.

- J

[Fred]

@jtsteph HAP in AVI has been supported for a while I can only praise the results. The files are big but performance is amazing, better than quicktime for sure. There are some threads about this already, but you will need the codec http://renderheads.com/product... and maybe something to encode, I use virtualDub and despite its odd ancient looking interface it is a pretty incredible tool. I did a very big theatre/mapping project and after some bugs got ironed out by mark during our rehearsals I had only 1 small glitch in 30 performances (multiple 4k videos, alpha layers, multi-channel audio). I definitely had good hardware but the system was eating the video files easily. This uses the direct show engine in windows and you can see the movie player display direct show if you are using it for decode.

[jtsteph]

Thank @Fred . I will need to retest, i think. When planning my show this summer, I tested quicktime and directshow and had determined that quicktime had a more predictable interaction and was lighter on the processor. I tested both on my laptop and a giant workstation. I am playing back 4 channels of video (640X480 to 1280X720) with multiple effects, audio reactive clip timing, and audio reactive postion/zoom etc.

I am currently revising my patches for another show and will work to shift everything over to AVI in light of the changes in V3. It's a bit of an onerous task as I work with hundreds of clips.

I will retest the AVI workflow and report back.

- J

[Fred]

@jtsteph I did not test retiming (that could be a sticking point, but it is going to be on any platform as most modern APIs dont support it that well), and I only used HAP avi (what codec were you using with directshow?), I noticed lower overhead. All the gpu based functions will only be limited by your system and have little to do with the container (so effects and position zoom etc as long as you dont use any CPU effects).  I was playing from a raided SSD array and a 1080ti, but it was not needed. 

[jtsteph]

@Fred . Thanks for the information. Yes, I have been making extensive use of retiming - making short animations that increase in complexity and tying the time of those to audio amplitudes is super engaging to watch. Quicktime does this is spades.  

My effects chain is entirely FFGL, I agree that any CPU effects bring things to a crawl. I did compress to HAP AVI but did not see much difference between it and other codecs. This is where I think I will need to retest and perhaps reconsider using retiming (bummer).

My (by now older) laptop below handled everything just fine using quicktime. My workstaion is an 8C/64gb ram/GTX1080 with NVMe storage. It still had issues with AVI in my initial tests.

I will endeavour to test again in the coming days and will post my results.

- J

[Fred]

@jtsteph Maybe we get media foundation integration with version 3.0, it would be great and give us a lot more codecs.

[jtsteph]

Here is another question. Will V3 support MKV with HAP in windows? I just spoke to the myFFMPEG folks. The new version of FFMPEG  and the next version myFFMPEG will support HAP AVI and HAP MKV. MKV is apparently a more contemporary container according to them.

[Fred]

@jtsteph afaik no, there are few discussions on the forum about integrating ffmpeg into Isadora, but for licensing reasons mark has stated this will not happen. Media foundation would be the next step as it can handle hardware decompression of h265. AVI is an older container, but this should not affect you really. They still play and windows does a pretty amazing job of maintaining backwards compatibility, there is no reason to be afraid of AVI, except for the limited codec support.

[Woland]

@jtsteph  Please see Mark's answer about FFmpeg from earlier in this thread:
@mark said:

Common Question: Why won't Isadora 3 support FFmpeg?

The following quote FFmpeg's site should give anyone pause:
Q: Does FFmpeg use patented algorithms?
A: We do not know, we are not lawyers so we are not qualified to answer this. Also we have never read patents to implement any part of FFmpeg, so even if we were qualified we could not answer it as we do not know what is patented. Furthermore the sheer number of software patents makes it impossible to read them all so no one (lawyer or not) could answer such a question with a definite no, those who do lie.

That's a risk I'm not willing to take yet, even though other companies seem to be doing so (Resolume, MAX, etc.) All it would take is one lawsuit to kill Isadora completely.

I will revisit FFmpeg as a possibility after Isadora 3's release. Perhaps if the end user installs the codecs, we'd be safe. That might be one route

[jtsteph]

@fred and @Woland . Thanks for the information. I just managed to get HAP AVIs up and running on my system (there was a conflict with the ffmpeg LAV filters). I let a clip run for the afternoon with a sine wave generator running the time position on the clip. The primary thing that I noted, to get rid of glitchyness, is to turn "loop" off when running variable timings on HAP AVIs with the DS engine. No crashes, no glitches. I will do a performance test over the weekend. Thanks again for all the information.

[DusX]

@jtsteph

In Isadora 2.6.1 looping is not supported (DirectShow does not offer this support), this is why you found it performed better with it set off.
In the upcoming Isadora 3 changes have been made to allow looping playback of Hap Avi in DS mode :)

[mark_m]

@dusx said:

In the upcoming Isadora 3 changes have been made to allow looping playback of Hap Avi in DS mode :)

 This makes me inordinately happy! I can hardly wait....!

[jtsteph]

@dusx said:

In the upcoming Isadora 3 changes have been made to allow looping playback of Hap Avi in DS mode

 Great. Thanks for the information @DusX .

[mark_m]

@dusx said:

In the upcoming Isadora 3 changes have been made to allow looping playback of Hap Avi in DS mode :)

 Oh, does that include palindromic looping? @DusX I do hope so....