COMMUNITY POLL: Most Important Video Effects? (Quick Response Please!)

  • Dear All,

    First, I hope you all saw the FFGL plugins I posted the other day. If not, please read here:
    I'm bringing a programmer on board to re-code many of the CPU based Isadora plugins, and the Pete Warden FreeFrame plugins. As you can see from the above post, I've built a hand fulll of replacements. But everyone please "chime in" on this question:
    "What is the most important Isadora video effect plugin?"
    At the top of the list will be HSL Adjust, because when users send me patches, I very often see that one. And I can guess that a nice blur would also be very important.
    But what else? 
    If you respond, it will help us to prioritize the list. He's going to start working next week.
    Thanks in Advance,

  • Threshold and blur

    • DeInterlace for live video sources.
    • Deinterlace double framerate mode (converts a 25fps interlaced stream to a 50fps progressive stream) (this one would be awesome !)

    Thanks !

  • Beta Platinum

    Here is my Christmas list for Santa ;-)

    Motion blur
    Video Delay
    Gaussian blur
    The top would be:
    Interactive Grid like in eMotion where like the Motion brush it changes interactively:  
    See minute 0:27
    [EDIT: Some of the items were not effects and are already rendering on the GPU, so I edited those out. -- MC]

  • 1. "anti fisheye effect" for correcting wide angles camera...very important for good camera tracking.

    2\. Homography transformation effect... to correct perspective.
  • Tech Staff

    Additionally I like using 'Color Measure' and 'brightness'.

  • I forgot a video gpu buffer.

  • Difference 
    Buffer -  (there is a video FFGL delay RGB but it's not as flexible)
    Brightness Calculator
    Eyes- even if this is not an effect

  • GPU Motion blur

    GPU Accumulator with arbitrary amount of frames
    GPU PhotoShop-like blending modes

  • In some form Eyes GPU would be awesome, but the ones I use the most are

    Video Delay
    Blur(any kind)

  • thanks mark for this first package of ffgl plug ins my wish: -deinterlace and maybe -refraction

  • Luminance Key  (I have installed a FFGLLumaKey, but need to test it)

    Calc Brightness
    TimeBlur (FF)
    Wave (FF)
    TimeWarp and/or TimeSlice (FF)
    Bloom (FF)
    Burn (FF)
    Glow (FF)
    Kaleidoscope (FF)
    QC Filmstock is also great... for adjusting video to look like film with parameters Saturation, Contrast, Grain and Vignette.


    thanks Mark!

  • Pixel Mapping (without needing a Lanbox)

  • Thank you all. I'll be passing this information on the programmer.

    I want to do a bit of "thinking out loud" regarding some of the plugins you're requesting.
    I'll break them into two groups:
    **Group A: Requires GPU Readback -- Eyes, Calc Brightness**
    I'm going to need to do some research to see how much of a performance hit something like Eyes and Calc Brightness are going to impose. In those cases, you will need to read from the GPU to the GPU, which is just the absolute worst thing to do when what your goal is performance. GPUs are very good at accepting data _from_ the CPU, but _giving it back_ is not what they're designed to do. (I'm not saying it can't be done, just wondering how much it will affect performance.)
    **Group B: Requires Loads of GPU Memory – Buffer, Shimmer**
    These actors concern me a little because of the limited memory on a graphics card – at least when compared to a CPU. The worst case scenario is if you're exclusively processing full HD video (1920x1080) -- each frame is about 8MB. The GPU memory is going to disappear pretty fast if you start using the Buffer actor to keep those massive textures in memory. 
    The card in my recently bought retina display is 2048MB. That means a total of 256 frames of video. Each Stage output uses 2 to 6 frames depending on what you're doing. Every FFGL effect you use eats one more frame of GPU memory.
    I mean, I quire certain it can be done. But that GPU VRAM is going to dry up really quickly. At the very most, the number of frames you could put in the buffer (again, full HD) would hover somewhere just over 225 (given a few stages and playing a few movies, etc.). That's 9 seconds of video at 25fps, and 7.5 seconds at 30fps.
    Not a lot. But maybe that's all you're expecting?
    Best Wishes,

  • That" what I expected and for me 9 seconds is enough.

  • Beta Platinum

    Interesting thoughts and information Mark.

  • Beta Platinum

    Sorry I missed the chance to comment here - but all the main actors I use have been listed. Mad busy a work yesterday.

  • @Mark Buffer should always be CPU (RAM)  but upload frames as needed to the GPU (GPU only needs what we see in a frame and buffer only shows one frame at a time) I have a buffer style video recorder and looper that lets me load 10 gig of frames into ram that works great and uploads a frame at a time to the GPU as needed)

    Any chance you would switch to using the openCV library for eyes++, it also has a lot lot lot of other great functions and would be an incredible addition to Isadora (Izzy would need a few more types to deal with it but nothing it does not deserve and could not make use of). Although much of the work is CPU, there is a lot of stuff that is now GPU accelerated, it is all cross platform and BSD license. 
    Alternately there are always compute shaders for the GPU or openCL/CUDA acceleration that are widely adopted and work well.

  • @Fred I used GPU Buffer with a "live feedBack program" with glsl in openframeworks with 4k resolution, and a 2GB buffer (not very long) gave the ability (by blending buffer and live input) to slow down and stabilize the effect which is very fast and hard to pilot. And the same effect with CPU buffer slowed down a lot the process which was no more real time.


  • @ioio  The dev branch of OF and the nighlty builds are now 64bit, giving you access to pretty much all the ram in your computer. The system I made has 1 frame latency to input and 1 to output. 4k GPU buffer will eat all your gfx memory for no reason or need and give you less memory to play with. As for making this on GPU, any latency can be minimised with efficient code and threads to handle capture decompression (especially copying or passing a reference of the image to the destination) . I used the blackmagic SDK to get the frames (some OF uses qtkit or older versions quicktime for capture). These libraries can be very slow getting frames and unwrapping them. That is the joy of coding, the efficiency and performance are scalable if you have the time and skills. I think to replace the functionality of the buffer in GPU would be very inefficient especially considering how much RAM machines have as standard vs GFX cards, especially on laptops.