[NOT POSSIBLE] NotchLC codec: "faster than realtime"

[eight]

https://notchlc.notch.one/

NotchLC is a GPU powered codec capable of being used for both intermediary and playback, made by the creators of Notch. It brings the equivalent of 10bit accuracy in a scrubbable codec that is extremely fast to encode and decode, with a compression ratios of around 5:1.

[chimerik]

Thank you for sharing this new codec.

Has anyone tried this codec with Isadora yet? 

I just switched to Catalina, have to abandon MEPG Streamclip, any recommendation on the conversion software for Mac?

Thank you!

[DusX]

@chimerik said:

Has anyone tried this codec with Isadora yet? 

 Like hap, this requires some work to be supported for decompression in Isadora. Additionally the 10 bit format is not yet supported. 

We have been looking at this codec and considering future development options. You can be sure we are looking to provide the best options for playback and presenation. 

[jhoepffner]

@chimerik

For conversion, I recommend Shutter Encoder

https://www.shutterencoder.com...

[chimerik]

@jhoepffner thank you for the recommendation. I will give it a shot. I find AVF Batch convertor also useful. Yet, both of them doesn't give me option for NotchLC codec.

@DusX Thank you, so this means I should wait until NotchLC is supported by Isadora. What codec do you recommend that's most efficient for realtime video processing that's replacing Apple Photo-Jepg?

[Woland]

@chimerik said:

What codec do you recommend that's most efficient for realtime video processing that's replacing Apple Photo-Jepg?

I'm sure you know a lot of this already @chimerik , but I wanted to write it out in case it was useful for anyone else on the forum.

Here's the list of recommended codecs+wrappers from the Isadora 3 release notes fleshed out with what I remember about them:

1. hap.mov
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform
   • Cons: Large file size, gradients aren't great, no alpha channel, not for HQ video
2. hapQ.mov (High Quality)
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform, HQ video
   • Cons: Large file size, gradients aren't great, no alpha channel
3. hapA.mov (Alpha Channel)
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform, alpha channel
   • Cons: Large file size, gradients aren't great
4. hap.avi
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform
   • Cons: Large file size, gradients aren't great, no alpha channel, not for HQ video
5. hapQ.avi (High Quality)
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform, HQ video
   • Cons: Large file size, gradients aren't great, no alpha channel
6. hapA.avi (Alpha Channel)
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform, alpha channel
   • Cons: Large file size, gradients aren't great
7. photoJPEG.avi
   • Can't remember pros/cons because I never use it
   • @chimerik said: Pros: good for interactive playback (different speeds, scrubbing) and realtime video efx 
   • @chimerik said: Con: larger file (but still relatively smaller than HAP), no Alpha.
8. photoJPEG.mov
   • Can't remember pros/cons because I never use it
   • @chimerik said: Pros: good for interactive playback (different speeds, scrubbing) and realtime video efx 
   • @chimerik said: Con: larger file (but still relatively smaller than HAP), no Alpha.
9. Apple ProRes 442.mov (macOS only)
   • Pros: good interactive playback (different speeds, scrubbing), cross-platform, better gradients than hap
   • Cons: Large file size, no alpha channel, macOS only
10. Apple ProRes 4444.mov (macOS only, Alpha Channel)
    • Pros: good interactive playback (different speeds, scrubbing), cross-platform, better gradients than hap, alpha channel
    • Cons: Large file size, macOS only
11. .wmv (Windows only)
    • Cons: Windows-only
    • Can't remember pros/cons because I never use it
12. H264.mp4
    • Pros: Small file size, best option for non-HQ videos with no alpha that you're just playing from start to finish at speed 1
    • Cons: Not great at speeds other than 1, reverse playback, scrubbing, etc.
13. H264.mov
    • Pros: Small file size, best option for non-HQ videos with no alpha that you're just playing from start to finish at speed 1
    • Cons: Not great at speeds other than 1, reverse playback, scrubbing, etc.
14. Mp3 (Audio only)
    • Pros: Running audio through the Movie Player actor gives you more control and options than using the Sound Player actor
    • Cons: Obviously, this is audio-only

---

Other factors that affect playback are: 

• Your computer
  • You will almost always get more bang for your buck with an expensive Windows machine than an expensive Mac.
  • Read/write speed of the internal hard drive (ideally SSD) is a big factor (as noted below).
  • Amount of RAM
  • Some wrappers/codecs only work on certain operating systems, or work better on one than the other.
  • Quality of the graphics card (having multiple graphics cards does NOT help!)
• The number of external displays you're using (and whether you're sending different content to each of them)
  
  • More displays = more work for your computer
  • The higher the resolution of the displays, the harder your computer has to work
  • Individual control of content on the displays is more costly
    • Sending six different 4k movies to six different 4k external displays = most work (six videos sent to six displays) 
    • Sending one 4k video to six different 4k external displays = less work (one video sent to six displays) 
    • Sending one 4k video to a six-way HDMI splitter that sends the signal to six 4k external displays = least work (one video sent to one display [the splitter])
• The read/write speed of the drive you have the media on (the faster the better)
  
  • External disks with a slow transfer speed are going to be worse for playback than a fast internal SSD.
  • SSD > HDD
  • A RAID SSD setup is ideal setup if you're playing a ton of high-quality footage simultaneously.
  • Never play media off of a USB flashdrive.
• How many videos you're playing at once (the fewer the better)
  • If you've got four videos playing in every Scene, keep in mind that during crossfades from one Scene to the next, your computer has to play eight videos at once.
• The resolution of the videos (smaller = faster)
• If your media is has a higher resolution and/or framerate than the display you're outputting to, you're wasting processing power. 
  • If you're playing a six 4k 60fps videos and sending them out to six video projector/beamer that can only do 720p 30fps, you're better off converting the content to 720p 30fps because the content will need to be scaled down to 720p 30fps in order to be displayed by the projector/beamer anyway.  because the output can't even handle 4k 60fps media.
• It's important to match your framerate and media size to the displays you're using whenever possible.
• Ideally, you'd have all your media and your physical displays using the same framerate and resolution so your computer doesn't need to do any scaling.
• The complexity of your patch
  • If you've got a super-complex patch that's doing a lot, it may affect playback. Try to streamline and keep things as efficient as possible.
  • Avoid using the video types CPU and CI whenever possible.
  • If you're having playback problems in a patch that generates its own content (like Shapes, Text Draw, GLSL Shaders, Live Drawing, etc), and you've manually set the resolution of those actors to 1920x1080, try cutting the resolution of those actors down to by 25% (1440x810) or 50% (960x540).

---

If you're playing a ton of videos and having playback problems: 

1. Try putting a faster internal SSD into your computer.
2. Try using a more powerful computer (better graphics card, more ram, more modern).
3. Try using a different codec/wrapper.
4. Try reducing the resolution of all media a bit to see if that improves playback.
5. Reduce the amount of media you're playing back at the same time wherever possible.
6. Make sure that the resolution of your media isn't higher than the resolution of your displays.
7. Try anything you can to reduce the complexity of your patch.

[jhoepffner]

@chimerik

Hello Sammy,

Shutter encoder is a little bit more "universal", no NotchLC at the moment but I asked it to the creator, perhaps can you also ask for it.

For the moment, I use Media Encoder (Adobe…) or TouchDesigner to encode NotchLC.

All the best from Paris!

[chimerik]

thank you very much @Woland , this is a really well organized and comprehensive notes on video codec, really appreciate it. It's also a good reminder for me to go through the release notes, it's really helpful! I can contribute something small to your notes:

photoJPEG.mov

• Pro: good for interactive playback (different speeds, scrubbing) and realtime video efx 
• Con: larger file (but still relatively smaller than HAP), no Alpha.
  
  I had been using this codec for the last decade and thought I might need to abandon it after 64 bit kicks in, so I just did a quick test on izzy 3.0.7 on the new MacBookPro running Catalina, with Photo-Jpeg video codec, it can pull off 10 of 1920 x 1080 videos with 2 efx each (HSL & Contrast), which means 10 HD videos with 20 vefx realtime playback at 30 FPS 90% CPU. That's pretty impressive! I will do some test on HAP as well, but so far Photo-Jpeg seems to be responding pretty well still. I did a report back for izzy 2.0 on bench mark for video playback, I can try to do one for this if there hasn't been much result shared yet :)
  
  thank you @jhoepffner Ok I can try to ask for it too. Thank you for sharing. Lots of love from Vancouver to you, too.

[Woland]

@chimerik said:

I can contribute something small to your notes

 Thanks, I've added the info you provided to my comment up-thread. <3