[ANSWERED] Using frequency monitoring in isadora 3 movie player

[deflost]

sorry, what a nonsens,

in our work we have 4 to 8 hours of setup time, most times.

working with such workarounds based on other software is one more reason for windows users to left the ship, maybe.

because of nothing with sound is really usable on windows. no asio, no vst, no multichannel output whitout tonnes of workarounds.

so we would like it to hear this honest and not allways we are working on, the first post we remember with the promise to build in asio is 4 years ago.

thx 

r.h.

#behonestwithwindowsusers

[Juriaan]

@deflost

Okay, here we go..

As a Software programmer / company you have to decide what your focus point is, Isadora has a userbase that is mostly working in graphical / theaterical enviroments where we have a performer and that needs to react in a certain way to the video. That is also what Izzy is doing really well, it is a really easy program to quickly prototype for those kind of mediums. (Mind that Isadora knows only programmer 'Mark' , and it is not a company that has 30 programmers working on the software as Resolume / Max MSP / TouchDesigner / etc)

Windows is a pain to develop for because a lot of the great features that Apple (Linux based) has to offer comes at almost zero cost for the programmer. They got great support for MIDI / Sound devices in general and that makes developing for Mac oriented users a blessing from a developer perspective and that is also why you see a lot more audio features on the Mac version of Izzy, since it is all baked inside the operation system.

As Isadora (Windows) community we already said against them that they should work on support for Audio interfaces / work on supporting all the great things that Mac has. Some things they can port, others not that is just how the software is built and that is a big process to turn around. 

Perhaps one of the team members of Troikatronix can tell you more about the things that are currently on the roadmap regarding Audio.

But perhaps this is not the right place to discuss this topic, we are here in this thread to help a fellow user out. We have plenty of discussion regarding audio and Isadora and feel free to join the discussion there so that you also see what we are thinking about / what we are proposing as artist / users against Mark regarding how it should work / look.

[mark]

@deflost @Juriaan and @gafias 

Some offerings regarding this conversation.

But directly to @deflost: yes, we said we'd get it done, but we didn't get it done. I'm sorry for that. I wish that we could afford to hire a team of four programmers to help me, but we simply cannot. I work harder than anyone I know, but it's never enough. All that we can do is to be strategic with our efforts, and converting Isadora to a 64 bit program that would be compatible 64-bit only macOS (Catalina) was a much higher priority than improving Windows audio for a simple reason: a massive percentage of our users are on macOS. In the end, all I can say is we keep trying and we do have plans for audio as noted below. But if Isadora is "unusable" then I guess it's would respectfully suggest you stop using it until we make it more to your liking.

Sound Frequency Bands

I apologize for the fact that this feature is only on macOS and only when the optimize mode is set to interactive. Here's how this happened:

1) We released Isadora v1 which was only based on QuickTime for audio playback. QuickTime offers this frequency band analysis as a built-in feature.

2) We released Isadora v2, which was based on AVFoundation and QuickTime on macOS, and used both DirectShow and Quicktime on Windows. The sound freq band feature was not available in AVFoundation or DirectShow, because neither offers it. But you could still use it in QuickTime on Windows.

3) We released Isadora v3 which cannot use QuickTime at all on Windows because we needed to remove QuickTime entirely because of the various security threats that prevented institutions in the US from installing it. That meant that you lost the sound freq band on Windows too, because neither Windows Media Foundation nor DirectShow offered this feature.

So why didn't we remove this feature entirely? For backwards compatibility. Keeping this feature for QuickTime allowed people to run their old patches that required it successfully.

I thought that the n/a (not available) marking, which you will see if the feature is not available, would alert people to the fact it was not available. But I guess we should have had something in the online help indicating this. I have now updated the plugin to offer that information.

Audio Roadmap

Regarding sound: we know this needs to be better. Our next major item on the roadmap for Isadora 3.1. is to allow fully routable sound in macOS and Windows, both for the Movie Player and the Sound Player. I hope that we'll be in beta with this before the middle of December.

Movie Player

Given a movie file with multichannel audio, you will be able to route any channel in the movie to any channel at the output. I believe it will be possible to do this interactively (i.e., change the routing in real-time) but I can't promise this for Windows yet.

On macOS, this will be done with the built in sound drivers that are part of the operating system. On Windows, this will be accomplished by drivers recognized by Windows Media Foundation, the standard system for playing media on Windows. That means it will not be ASIO because that is not a driver that WMF knows about.

Sound Player

Given a sound file file with multichannel audio, you will be able to route any channel in the sound file to any channel at the output. It will be possible to do this interactively (i.e., change the routing in real-time) but I can't promise this for Windows yet.

The Sound Player use a system called SoLoud to handle multichannel sound on both macOS and Windows. It has various driver interfaces to the sound rendering options offered by the operating system. (On macOS, there is only one; on Windows there a several.) However, ASIO isn't one of the systems it supports. I am open to implementing ASIO as an output system for SoLoud.

ASIO

All of you talk about ASIO as sort of the holy grail of audio on Windows. I need someone to educate me about this, because – from my relatively uninformed point of view – if the sound gets out to the multichannel device using Windows drivers, I am unsure what the problem is. If you have the time and want to explain the advantages to me, I am ready to learn. I know it is a common claim that ASIO is the lowest latency system out there, but I have seen this questioned on a couple of programmer sites. I'm not saying it's not true, but I would be interested to see a scientific measurement as compared to, for example, Windows Audio Session API (WASAPI) which would seem to be extremely low latency.

VST

We will not offer VST plugin support in Isadora 3.1, because this is a much bigger "fish to fry." On MacOS, the ability to host and use Audio Units is a built-in feature. That's why we have it on macOS – it took little effort to add it. For VST, we must write a custom host which is a complicated undertaking that will consume of a lot of time coding and testing. Before going down this road, I need to know that having this feature is going to sell programs. We must do a cost/benefit analysis for every major feature we add, because we need to know that such efforts will generate a profit and not a loss in the end.

That's what I can offer you about our audio plans at the moment.

Best Wishes,
Mark

[gafias]

thank You all for your valid inputs and for the discussion created by the requirements of my project.

I am quite the amateur. I was invited to work for a play, and being an amateur, I solved the issue by outputting the sound and inputting it through an external sound card line in.

i am glad to know I paid a license and that @mark even replied to my question. Thank you so much for your program that helped me make create something that will make some kids very happy.

all of you guys speak about a lot of things. As a retired business consultant I understand @mark. He has to prioritize his assets to develop the most required tools. He can only know that through this forum so it is important that all of us participate on making it better by finding new ways to use it and specific problem.

Asa beginner very much interested on the capabilities of this software I feel that it is lacking examples online of what you can achieve by using it. Today I finished a project and nobody knew this solution. I was surprised. I needed help and even the founder replied to me. Give more examples, more alternatives, I will, and others for sure help the base of users to grow and therefore the listing of opportunities to develop it even further.

Again, thank you all for your support. I am not here to to support any discussion that does not provide a solution. (@mark frequency selection said n/a but nevertheless the output was created). I am here to dream, and what you do helps me make my “dreams” come true. So thank you; even if not in a perfect way my creativity took shape.

i am happy to be part of this community,

Diogo

[mark]

@gafias said:

Asa beginner very much interested on the capabilities of this software I feel that it is lacking examples online of what you can achieve by using it.

Thank you for your thoughtful words and for understanding the need to prioritize things. As sometime trained as an artist, thinking in "business mode" is something that I am still learning how to do well. But it's clear that for the long term health of the software, given that we are a small company, I need to take care about where we allocate our resources.

Regarding examples: I have started recording a new batch of tutorials for Isadora that will start going online soon. The old tutorials were one of the best ways to learn Isadora, because they provided the guidance you're speaking of. I hope to have the first one online next week, and plan to add one per week after that until we've hit all of the important points for novice and intermediate users.

Best Wishes,
Mark

[bonemap]

@gafias said:

I solved the issue by outputting the sound and inputting it through an external sound card line in.

 It is good to hear that you found a solution. Did you end up playing the file internally in Isadora and looping through your sound card to the live capture settings and enabling Audio Frequency Analysis?

Best wishes

Russell

[gafias]

@mark, just shared my feelings regarding all the discussion my question started. I have the reverse problem, a businessman starting to do art. It is hard to use "both sides" of the brain.

Where will those video tutorials be available? I am looking forward to watch them.

Sorry for the late reply but I was fortunately away from technology.

Best regards,

Diogo

 

[gafias]

@bonemap, exactly. I wanted to use the volume variation to affect the projection and it worked as i required.

Best Regards,

Diogo

[tomthebom]

@gafias

Hi Gafias, regarding "businessman starting to do art": I know what you are talking about (me: "engineer to puppeteer").

Something that really really helped me was reading and doing Betty Edwards "How to draw on the right side of the brain".
After many years of not knowing how to deal with my double education, this book finally helped me to get conscious about which part of the brain I am using and how to switch...

Best regards, Tom

[chimerik]

Hi all,

I believe it maybe due to Catalina update/new machine, the freq bands in movie player stopped functioning. I've found this thread and understand that it doesn't support Windows, but for Mac after Catalina, is there something that we need to download to enable QuickTime to function in Isadora?

Right now I can think of a workaround by using the old machine to freq bands values via OSC to the new machine, but just like to check in to see if there's any updates on this. Thank you!

[Michel]

@chimerik

Unfortunately not, the Quicktime engine is 32 Bit and will not work on Catalina.

Best Michel

[jfg]

@chimerik

you have to use the way over "Soundflower" or " Blackhole" and the sound Frequency Band or Sound frequency watcher actors.

best

Jean-François 

[chimerik]

Thank you @Michel  

Right, I remember Mark mentioned that a while ago. So this "freq band" feature won't function after Catalina. I look forward to future replacement of this feature :) In the meantime, I figured out using the old machine to run the stem audio tracks and send each track's value via OSC to the new machine. It's working well so far :) 

@jfg thank you for the suggestion! I'm actually using the freq band feature for a specific reason. Just sharing it for those who might be interested using similar technique. I basically take different stem tracks and use the volume data of each stem to drive the visuals. 

with love,

[bonemap]

@chimerik said:

take different stem tracks and use the volume data of each stem to drive the visuals

Thanks so much for sharing your technique it is really cool!

The big question is what happens to frequency bands in the Movie Player? It was a big part of my workflow with Isadora pre Catalina - so what now post Catalina? I have kept an older MacPro with OSX Mojave because the freq bands function of the MoviePlayer is so quick and convenient to deploy in a patch when prototyping. Using the Sound Frequency Watcher or Frequency Bands with Live Capture frequency analysis adds a lot of extra steps and system configuration to implement to get to the same place in Catalina. 

It would be great to get this resolved soon. My understanding is that the frequency band functionality of the MoviePlayer on the Mac pre Catalina will be lost in future implementations of the new sound/audio channel system currently being developed in the beta program, because it was provided by Quicktime.

best wishes

Russell

 

[Woland]

@bonemap said:

what happens to frequency bands in the Movie Player? It was a big part of my workflow with Isadora pre Catalina - so what now post Catalina?

I've logged this as a feature request but since it's more of an advanced feature I predict that we won't be able to take care of this as soon as you would like. We try to prioritize adding features and improving things that will give a better experience for as many users as possible, so unfortunately many things that we need to rank higher on our to-do list are not power-user features like sound frequency analysis. As Mark loves to say though, the squeaky wheel gets the grease, so if you and others remind us of this feature request often enough, it improves the chances of it being ranked higher in terms of priority.

Best wishes,

Woland

[jfg]

@woland

I remind you of this feature and will do it (one a week is OK :-) ? ) 

best

Jean-François

[deflost]

hello,

we also whish to have a future with this feature...

r.h.

[chimerik]

Thank you everyone. I also hope there are ways to bring this feature back when time is right for izzy. In the meantime, I just did a test on the workaround, and I thought it might be helpful for people to see how it works. Feel free to watch and use this as your workaround to achieve very similar result. 

https://youtu.be/McuZL128MOc

summary: I recorded this demo to show the workaround for the Movie Player's freq band output trick that worked before Catalina, and for Window users too. Since the original Movie Player's freq band uses QuickTime engine, and QuickTime engine isn't supported 64 bit, post Catalina OS won't work. Isadora community users has suggested this workaround. Originally I thought it would be a lot more work due to the many audio tracks to capture live, but since Blackhole supports 16 channel virtual audio capture, and you can actually identify each channel in the sound level watcher actor, so this workaround is actually feasible.