How to manipulate video with sound in Izzy?


  • @joeydrums

    this plugin is useful to give a visual reference to what the frequencies are doing, and to isolate the range and then set up a frequency watcher for each level
    http://www.voxengo.com/product/span/

  • I really enjoy this community. People are very helpful and knowledgeable.  Thanks.  I'm checking out the plug in leben.  I flit in and out of using Isadora so I'm always learning.

    I'm a bit familiar with both Max and PD but never sent them to Isadora.  How dow I pull in a patch I made in one of them into Isadora via OSC or midi. Midi I understand, OSC is a bit more of a mystery.
    Also are you suggesting building a multi band frequency watcher in PD or Max.  Is that what in your conception of this you would use those apps for?  Curious you thoughts.

  • @leben

    Where would you insert Voxengo in the signal flow within Isadora?  It is a live capture. Just use Sound Input actor _before_ Sound Frequency Watcher? Problem is live input is coming through channel 1\.  That is not a choice in Sound input but my built in microphone (which is what live input is set to does come up within Sound input).  Problem is there is no "out" for the audio on Sound input.  Must be my signal flow is wrong?
    AND how do I "see" the Voxengo's UI in Isadora?
    Finally to ALL.  My signal flow currently is:
     Live Input > Sound Frequency watcher.  Movie player > effect (bloom) > Projector.  
    Trigger or/ energy out of Sound Frequency Watcher (still playing with this one) going to the effect (bloom) input called "floor" parameter.  
    This all works fine.  BUT I want that same singular video to have multiple "effects" getting manipulated by the multiple Sound Frequency Watchers isolating different parts of my frequency spectrum.
    Thanks for leben and all in advance!

  • Hello,
    I made a quick sample of frequency analysis in Max (coming from a synth…), a 16 band analyser sending 16 OSC signal to Isadora doing a color mix.
    Pictures and files are here.
    91f6df-capture-decran-2016-02-01-a-23.15.08.png 21c8e4-capture-decran-2016-02-01-a-23.15.30.png 0e1687-testmaxosc.zip


  • Forum.  I have another question.  Has anyone every used Sound Frequency Watcher to monitor a pre recorded and looping audio file?

    It seems SFW only works with Live input?
  • Tech Staff

    @joejdrums

    You could use something like Soundflower or SoundSyphon to route the sound of the movie back into Isadora.

    Best Michel


  • That is a good idea.  I'll check it out.

  • Tech Staff

    I have been experimenting with using Imitone ( https://imitone.com/ )

    It converts sound to midi.. lots of interesting potential.

  • oooh, looks interesting, cheers for this : )


  • @DusX

    Will Izzy pick this up in the input ports of the midi set up, as it would any piece of midi hardware?
  • Tech Staff

    yes..  its just a midi source.


  • Looks cool.  Thanks man


  • Meanwhile you can test existing audio to midi apps. Ones like WIDI has been around for some time. You can convert audio(singing into the mic) to midi in real time and play a synth accordingly.

    http://www.widisoft.com/english/mp3-midi-products.html
    Max/MSP can be set up as such as well
    https://youtu.be/BiEBN17ZnNI
    PD (PureData)
    http://forum.pdpatchrepo.info/topic/9816/audio-to-midi-in-pd

  • @jhoepffner

    This Max patch looks interesting, I know nothing about max, I assume one would be able to tweak this patch so that is sends data from a live audio feed into izzy, instead of from a synth...
    Thank you : )

  • Try WIDI it does it "out of the box". Just choose MIDI output "Isadora..."

     You can handcraft this with Isadora as well: Frequency Watcer for each note(usable voice range is about 2 octaves - 24 notes)

  • Thanks Vanakura.  I have looked into Imitone.  Very, very Beta.  Not too accurate as far as I can tell yet. I will look into the Max patch.

    There is one thing I'm trying to understand with the audio conversion to midi.  This is for a live situation? I think I understand the live signal flow.  
    Audio streams > gets converted to a binary code that Isadora reads and then can process via Sound Frequency Watcher.  
    Yes?  
    Also, Vanakura.  Would you happen to have have any patches you're able to pass along for Max or PD?  

  • look, this may help you

     http://www.ernestomalaca.com/uncategorized/audio-trigger-a-midi-puredata-ableton-live/

  • So is there not a way that voices speaking can be translated into numbers for parameter changes?

    Or something that takes the audio decibel level and uses that to change parameters?

    Thanks, Kedar


  • @Klawrence

    Isadora has actors for that - Sound Level and Sound Freq Watcher.

  • @joejdrums

    I strongly suggest to try to study PD(PureData). It is really rewarding to create something with it. There is a patch that takes mic input and makes midi notes according to sound frequency. You may need help with getting fiddle to work / I have my question answered on their forum. Pure Fun.

    22f56b-mic-to-midi.png