assurance-tunnel
assurance-tunnel
assurance-tunnel
assurance-tunnel

Using frequency monitoring in isadora 3 movie player



  • Re: [Using an audio file in Isadora to trigger effects](/topic/988/using-an-audio-file-in-isadora-to-trigger-effects)

    Hey,


    I am currently working on a project where I want to use the sound of a video to change effect parameters. Following the previous post recommendation I have added 2 frequency bands but I have no reaction on the freq 1 or 2 outputs.


    I am using Isadora 3 for windows. What might be the problem and is there a possible solution?


    thank you in advance,


    diogo


  • Beta Platinum

    @gafias said:

    solution?

     It appears that all you need to change is the ‘optimize’ input of the MoviePlayer to ‘interactive’


    best wishes

    Russell



  • @bonemap

    thank You for your suggestion. 


    I tried it out, both with video and sound files, and the frequency levels still do not vary. Do you have another suggestion?


    best regards,


    diogo


  • Beta Platinum

    @gafias said:

    another suggestion

     Hi,

    If you have audio playback through the file it is intended to output the frequency levels. The next step, if it is not working, would be to check the volume and pan settings. After that we would have to look at file format etc.

    Please try the file attached here: Freq Bands.zip


  • Beta Platinum

    Hi there both,


    I'm not so sure that the freq bands work in Izzy 3.x.x for Windows users, it works in Isadora 2.6.1 but I also can't replicate your patch @bonemap, it defaults to WM (Windows Media Foundation) and not the Quicktime Player that you seem to be using.

    Perhaps @mark can tell us why this is happening, could be that this is a QuickTime only kind of feature. Not sure


  • Tech Staff

    @juriaan said:

    not so sure that the freq bands work in Izzy 3.x.x for Windows users

     Correct. I have an item logged in our sheet named "Movie Player Frequency Bands is MacOS QuickTime Only"



  • @juriaan said:

    I'm not so sure that the freq bands work in Izzy 3.x.x for Windows users

     No, not working for me in Windows.
    Have skimmed the release notes, don't see this listed, but could be part of Window's sound features which are currently being worked on.
    Or it's a bug...


  • Beta Platinum

    @juriaan said:

    QuickTime only kind of feature

     Hi Juriaan, Mark,

    ARRGH! Sorry to hear that frequency bands are not available to Windows users - that is so useful to know - I use this feature so often I take it for granted. It would drive me nuts not being able to hook into audio frequencies through the Movieplayer. I guess Mac users will also loose this with the imminent death of Quicktime. 

    @gafias has provided a screen shot that indicates the playback engine being used is Quicktime (QT) so in my mind I was expecting it to be working for that patch. But alas @gafias with your patch in Windows I can offer no further suggestions.

    best wishes

    Russell


  • Beta Platinum

    If you really need this for you work, get in touch with me using my contact details in my signature. I can make you an Max MSP patch / workaround that allows you to get all the frequency bands through OSC


  • Beta Platinum

    Ok here is a quick workaround built for Windows 10 with Max MSP 7 (32 bit, important (!))
    For an audio installation I use the Analyser~ object that allows us to find the MIDI pitch and all other kind of information regarding the sound that gets played trough our input, it is also capable of sending us the bark list (frequencies from 0hz to 22KHz)

    • Install Virtual Audio Cable (this is needed to create an virtual input / output on Windows that we can use to analyse our sound and route the audio from Isadora to Max MSP)
    • Restart your PC
    • Install Max MSP 7 (32 bit) (https://akiaj5esl75o5wbdcv2a-m...)
    • Install the Analyser~ object found here (https://1drv.ms/u/s!AsGHU5QfIy...), put that inside Documents > Max 7 > Packages
    • Restart / Start Max MSP
    • Open up the file called 'SoundAnalyser"
    • Go to Options > Audio Status, make sure that your input is set to our Virtual Audio cable :)
    • Start your audio by pressing the little power button at Options > Audio Status or in the Left bottom of your patch window
    • In your patch in the Menu Dropdown pick 'Audio' to listen to Channel 1
    • Open up Isadora 3.x
    • Make sure that your set on the same network / adapter
    • Make sure that your receiving OSC port is 1234
    • Go to Stream Setup > Auto Receive Input (Turn this on !)
    • You should see that the values are incoming.

    Please be aware that you need to set your Windows primary Output device to our Virtual Audio cable for Izzy to understand it, Max MSP will receive the stream and from there you can actually do the output of the mix to your speakers.



  • sorry, what a nonsens,

    in our work we have 4 to 8 hours of setup time, most times.

    working with such workarounds based on other software is one more reason for windows users to left the ship, maybe.

    because of nothing with sound is really usable on windows. no asio, no vst, no multichannel output whitout tonnes of workarounds.

    so we would like it to hear this honest and not allways we are working on, the first post we remember with the promise to build in asio is 4 years ago.

    thx 

    r.h.

    #behonestwithwindowsusers


  • Beta Platinum

    @deflost

    Okay, here we go..

    As a Software programmer / company you have to decide what your focus point is, Isadora has a userbase that is mostly working in graphical / theaterical enviroments where we have a performer and that needs to react in a certain way to the video. That is also what Izzy is doing really well, it is a really easy program to quickly prototype for those kind of mediums. (Mind that Isadora knows only programmer 'Mark' , and it is not a company that has 30 programmers working on the software as Resolume / Max MSP / TouchDesigner / etc)

    Windows is a pain to develop for because a lot of the great features that Apple (Linux based) has to offer comes at almost zero cost for the programmer. They got great support for MIDI / Sound devices in general and that makes developing for Mac oriented users a blessing from a developer perspective and that is also why you see a lot more audio features on the Mac version of Izzy, since it is all baked inside the operation system.

    As Isadora (Windows) community we already said against them that they should work on support for Audio interfaces / work on supporting all the great things that Mac has. Some things they can port, others not that is just how the software is built and that is a big process to turn around. 

    Perhaps one of the team members of Troikatronix can tell you more about the things that are currently on the roadmap regarding Audio.

    But perhaps this is not the right place to discuss this topic, we are here in this thread to help a fellow user out. We have plenty of discussion regarding audio and Isadora and feel free to join the discussion there so that you also see what we are thinking about / what we are proposing as artist / users against Mark regarding how it should work / look.



  • @deflost @Juriaan and @gafias 

    Some offerings regarding this conversation.

    But directly to @deflost: yes, we said we'd get it done, but we didn't get it done. I'm sorry for that. I wish that we could afford to hire a team of four programmers to help me, but we simply cannot. I work harder than anyone I know, but it's never enough. All that we can do is to be strategic with our efforts, and converting Isadora to a 64 bit program that would be compatible 64-bit only macOS (Catalina) was a much higher priority than improving Windows audio for a simple reason: a massive percentage of our users are on macOS. In the end, all I can say is we keep trying and we do have plans for audio as noted below. But if Isadora is "unusable" then I guess it's would respectfully suggest you stop using it until we make it more to your liking.

    Sound Frequency Bands

    I apologize for the fact that this feature is only on macOS and only when the optimize mode is set to interactive. Here's how this happened:

    1) We released Isadora v1 which was only based on QuickTime for audio playback. QuickTime offers this frequency band analysis as a built-in feature.

    2) We released Isadora v2, which was based on AVFoundation and QuickTime on macOS, and used both DirectShow and Quicktime on Windows. The sound freq band feature was not available in AVFoundation or DirectShow, because neither offers it. But you could still use it in QuickTime on Windows.

    3) We released Isadora v3 which cannot use QuickTime at all on Windows because we needed to remove QuickTime entirely because of the various security threats that prevented institutions in the US from installing it. That meant that you lost the sound freq band on Windows too, because neither Windows Media Foundation nor DirectShow offered this feature.

    So why didn't we remove this feature entirely? For backwards compatibility. Keeping this feature for QuickTime allowed people to run their old patches that required it successfully.

    I thought that the n/a (not available) marking, which you will see if the feature is not available, would alert people to the fact it was not available. But I guess we should have had something in the online help indicating this. I have now updated the plugin to offer that information.

    Audio Roadmap

    Regarding sound: we know this needs to be better. Our next major item on the roadmap for Isadora 3.1. is to allow fully routable sound in macOS and Windows, both for the Movie Player and the Sound Player. I hope that we'll be in beta with this before the middle of December.

    Movie Player

    Given a movie file with multichannel audio, you will be able to route any channel in the movie to any channel at the output. I believe it will be possible to do this interactively (i.e., change the routing in real-time) but I can't promise this for Windows yet.

    On macOS, this will be done with the built in sound drivers that are part of the operating system. On Windows, this will be accomplished by drivers recognized by Windows Media Foundation, the standard system for playing media on Windows. That means it will not be ASIO because that is not a driver that WMF knows about.

    Sound Player

    Given a sound file file with multichannel audio, you will be able to route any channel in the sound file to any channel at the output. It will be possible to do this interactively (i.e., change the routing in real-time) but I can't promise this for Windows yet.

    The Sound Player use a system called SoLoud to handle multichannel sound on both macOS and Windows. It has various driver interfaces to the sound rendering options offered by the operating system. (On macOS, there is only one; on Windows there a several.) However, ASIO isn't one of the systems it supports. I am open to implementing ASIO as an output system for SoLoud.

    ASIO

    All of you talk about ASIO as sort of the holy grail of audio on Windows. I need someone to educate me about this, because – from my relatively uninformed point of view – if the sound gets out to the multichannel device using Windows drivers, I am unsure what the problem is. If you have the time and want to explain the advantages to me, I am ready to learn. I know it is a common claim that ASIO is the lowest latency system out there, but I have seen this questioned on a couple of programmer sites. I'm not saying it's not true, but I would be interested to see a scientific measurement as compared to, for example, Windows Audio Session API (WASAPI) which would seem to be extremely low latency.

    VST

    We will not offer VST plugin support in Isadora 3.1, because this is a much bigger "fish to fry." On MacOS, the ability to host and use Audio Units is a built-in feature. That's why we have it on macOS – it took little effort to add it. For VST, we must write a custom host which is a complicated undertaking that will consume of a lot of time coding and testing. Before going down this road, I need to know that having this feature is going to sell programs. We must do a cost/benefit analysis for every major feature we add, because we need to know that such efforts will generate a profit and not a loss in the end.

    That's what I can offer you about our audio plans at the moment.

    Best Wishes,
    Mark



  • thank You all for your valid inputs and for the discussion created by the requirements of my project.


    I am quite the amateur. I was invited to work for a play, and being an amateur, I solved the issue by outputting the sound and inputting it through an external sound card line in.

    i am glad to know I paid a license and that @mark even replied to my question. Thank you so much for your program that helped me make create something that will make some kids very happy.

    all of you guys speak about a lot of things. As a retired business consultant I understand @mark. He has to prioritize his assets to develop the most required tools. He can only know that through this forum so it is important that all of us participate on making it better by finding new ways to use it and specific problem.

    Asa beginner very much interested on the capabilities of this software I feel that it is lacking examples online of what you can achieve by using it. Today I finished a project and nobody knew this solution. I was surprised. I needed help and even the founder replied to me. Give more examples, more alternatives, I will, and others for sure help the base of users to grow and therefore the listing of opportunities to develop it even further.

    Again, thank you all for your support. I am not here to to support any discussion that does not provide a solution. (@mark frequency selection said n/a but nevertheless the output was created). I am here to dream, and what you do helps me make my “dreams” come true. So thank you; even if not in a perfect way my creativity took shape.

    i am happy to be part of this community,


    Diogo



  • @gafias said:

    Asa beginner very much interested on the capabilities of this software I feel that it is lacking examples online of what you can achieve by using it.

    Thank you for your thoughtful words and for understanding the need to prioritize things. As sometime trained as an artist, thinking in "business mode" is something that I am still learning how to do well. But it's clear that for the long term health of the software, given that we are a small company, I need to take care about where we allocate our resources.

    Regarding examples: I have started recording a new batch of tutorials for Isadora that will start going online soon. The old tutorials were one of the best ways to learn Isadora, because they provided the guidance you're speaking of. I hope to have the first one online next week, and plan to add one per week after that until we've hit all of the important points for novice and intermediate users.

    Best Wishes,
    Mark


  • Beta Platinum

    @gafias said:

    I solved the issue by outputting the sound and inputting it through an external sound card line in.

     It is good to hear that you found a solution. Did you end up playing the file internally in Isadora and looping through your sound card to the live capture settings and enabling Audio Frequency Analysis?

    Best wishes

    Russell