Audio Features for Isadora: What Do You Want?

bonemap

Hi,

I am a big fan of movie player integrated audio frequency outputs. Being able to use integrated audio frequency bands straight out of a movie player has been the foundation of many of my projects and interactive patches. It has been so critical when keeping audio and video in sync while routing audio responsiveness around a patch. I will have a huge back catalogue of Isadora repertoire that will be cumbersome to adapt without the integrated audio frequency bands that we have enjoyed with Quicktime based codecs on Mac. But is there an alternative? what is the future of that feature?

It was great to use multichannel video files recently and facilitate a projection project where audio speakers were distributed around the space, mostly behind projection surfaces but some being flown. This has been available for the Mac version of Isadora and will be an exciting option for WIN users when it is released. It was tricky to format the multichannel video files for a recent work as it required duplicate stereo tracks being panned on output from Premiere Pro and a master track being specified before the project file was initialised. Some of the issue was due to being given tracks embedded in stereo rather than all mono tracks to work with. It was a beautiful thing in the end.

Having an integrated option to re-route audio from video files would be super useful, as suggested previously, being able to avoid partial workarounds such as using Soundflower on the Mac would be absolutely amazing.

best wishes

Russell

Maximortal

1: asio

2: asio

3: asio

4: audio oscillator

mark

@maximortal said:

asio asio asio

OK, you want ASIO and others like like @jhoepffner have argued for it also. But we need to discuss this.

We use Windows Media Foundation to play movies on Windows, and WMF uses Windows Audio Session API (WASAPI) to send audio to the audio device. To allow output from WMF to ASIO is going to take at least a six weeks of focused work (i.e., no other features being added and no other bugs getting fixed), and testing it will take another four to six weeks (maybe more because we'll have to try it with a bunch of devices.) That puts the release of 3.1 much further into the future than we would like.

I must admit, not being a Windows user for audio, I don't understand why ASIO is so critical. Every up-to-date audio device is going to have a standard set of WASAPI drivers that are going to be compatible with Windows Media Foundation, and -- if you choose to use WASAPI in dedicated mode (where no other apps can share the output hardware) -- the bits bypass the Windows mixer, meaning that the the fidelity is the same as ASIO, and the latency (from what I've read) is pretty much the same as ASIO.

So, given that the sound will get to your speakers using WASAPI, why is it important that it be ASIO instead?

This is not the only place where a debate on this has occurred. The discussions listed below are just four I've found -- there are many more actually.

https://www.audiosciencereview...

http://forum.cakewalk.com/Wasa...

https://forum.audiogon.com/dis...

https://www.reddit.com/r/headp...

I'll just point out that the reddit thread starts with "There was a time when windows sound drivers sucked. Big time. That's the reason why ASIO was popularized, to get away from the crappy Windows Kmixer kernel that mangled sound like a testicular vice..." and ends with the comment "WASAPI, or ASIO (if the drivers are stable), will feed a bit-perfect signal to the DAC."

My main takeaway from reading several of these debates is that no one really seems to scientifically prove that ASIO is better.

If we're going to delay the release of v3.1 by more than two months beause of ASIO, I need a really convincing reason to use this API.

Best Wishes,
Mark

Maximortal

asio Is so important because on my experience any audio interface sound better with asio driver than wasapi: lower latency, better stability. As you said during werkstatt Isadora what to be the people's media server and to do that output quality is important. Actually the audio quality with Isadora don't satisfy my needs and I use others software for that. Maybe with asio we can use Isadora for all purposes.

jhoepffner

Hello,

I agree with Mark, the most important function (for me) is routing from input to output.

Input : sound file, movie file, live feed (sound card, computer, soundflower etc.), kinect, internet …

Output : physical (sound card, audio jack, hdmi), virtual (internet, Dante, NDI…), recording (audio file, video file)

With the possibility to change it smoothly during performance.

I know by experience that sound is so much easy and dependable on Mac but for live video creation nVidia cards are so much better that I have to use both systems.

And the asio debate.

On my experience, mostly with Touchdesigner on Windows, with asio driver I can easily reach every input/output for every physical/virtual input/output on my computer.

With other drivers (directSound in TD) you I only reach the first input/output pair on each named device.

I use mostly Dante Via to route the sound inside and outside my computer and I am very happy with the possibilities but it is a paying software and not very user friendly.

The another problem on Windows asio (compared to MacOs) is that you have to be very careful not to mix 44 with 48 sound, at least with TD.

For the other possibilities, I thing the 'minimum vital' of sound treatment is mandatory, mainly equalizer and limiter/compressor.

VST filters and oscillators would be great but for my part I greatly prefer to use Ableton Live or Max/Msp, be on separate or the same machine.

In any case, thank you for your work and the listening to our needs.$

Jacques

jfg

VU-meter vor control pane. I use slider but it is not so chic

soniccanvas

Awesome :)

1) a good flexible built in mixer that allows easy routing from multi inputs to multiple outputs

2) support for poly wave files

3) support for some kind of ambisonics workflow

4) support for Au and VST plug-ins :)

Thanks!

David

kdobbe

My use of Isadora in theatre is what would be at the top of the list. Triggered sound/music/FX are heavy and actors can be built; but it would be WAY cool to have the following built in:

In-and down indicator (after start at 100%: a time before lowering to x-percent) with adjustable time (i.e. 100%-30% over 5 seconds)
Secondary loop area: ability to loop to middle of audio AFTER the whole file is played (this allows for the In-and down action

I use this all the time in theatre applications and have done so with building actors... the actors become VERY complicated to manage and this would allow simpler control.

Does anyone else use audio in this way?

mark

@kdobbe said:

In-and down indicator (after start at 100%: a time before lowering to x-percent) with adjustable time (i.e. 100%-30% over 5 seconds)Secondary loop area: ability to loop to middle of audio AFTER the whole file is played (this allows for the In-and down action

For the "in and down" are you speaking about volume?

For the looping, assuming the loop point is 50%, it would be like this

1) play 0 to 100%
2) then play 50% to 100%, 50% to 100%, etc

right?

Best Wishes,
Mark

tomthebom

Could It be an idea to improve the possibilities around MIDI? I am thinking of something similar to the Delta MIDI Composer (), just within Isadora.

https://www.youtube.com/channe...

knowtheatre

@mark said:

For the looping, assuming the loop point is 50%, it would be like this
1) play 0 to 100%
2) then play 50% to 100%, 50% to 100%, etc
right?

Yeah, I would second this. I often need to create a "vamp:"

1) play from 0% to say, 30%

2) then play from 25% to 30% on a loop

3) then release that loop and continue play from 30% to 100%

Also, I have no idea how possible this would be, but a way to display the wave form would be awesome. A visual way to identify quickly start/end/loop points would be helpful. As of now, I usually just edit the file externally...

anibalzorrilla

I think that something like a "sound watcher" and a "sound generator" actors, and some basic audio process and settings like delays, compressor, filters (FFT), reverbs, equalizers, (or audio operators like [+~] in MaxMSP and others similar) would be fantastic. And all the audio routers and select that you can imagine. I use PD with OSC communication, but I would like to have all of that in Isadora. Best, AZ.

Juriaan

@tomthebom

Could you tell us what you are missing from Isadora that you find inside this program ?

mark_m

Another vote for vamping.

And aside from that I would like first and foremost a way of playing back an audio file or a movie file and for Isadora to be able to analyse its frequencies, so that I can create images that respond to the beat.

I would also like there to be an energy level monitor, so that I can get Isadora to 'listen' to and respond to live sounds of a certain volume level and / or pitch.

I really really want to understand better how to integrate Isadora with, say, Ableton Live. People keep threatening to write tutorials, but they haven't appeared yet.

And finally, I'd like Isadora to be able to respond to voice commands, like my GoPro does. Or my iPhone... "Hey, Isadora, go to next scene"

I'm not serious about that last one.

Michel

@mark

Please make the sound level setting in movie player and sound player switchable to dB instead of percentage. Its sometimes embarrassing if someone asks to put up the level a few dB's and I say I can't I must do it in percentage.

eight

@mark Buffer and Delay actors, similar to those in Video department.

Michel

@mark

It would be great if Isadora could read 24bit audio files. It does work when I embed them into a movie. I guess not many will ever hear the difference between 16bit and 24bit be it because of the hardware or their ears. But we have sound designer students and they would probably swear they hear it. The main difference is explained in this youtube video, starting at 6min 45 sec:

kdobbe

@mark

Yes on both... I should have been a bit clearer.

bonemap

Hi,

From what I have learnt about audio input and routing in Isadora (I could have easily missed something) there appears to be currently the following options for the Mac version:

Audio sources:
1. Live Capture: Microphone/ Line in (4)
2. Audio File
3. Video File
4. Sound Input plug-in (device selectable)

Audio matrix routing is currently configurable in three places within Isadora Mac version:
1. Sound Output Setup: Sound Output Routing: Channel 1 - 16 /Ext. 1-32 (in pairs)
2. MoviePlayer: audio trks /trk vol /trk pan /trk out (1-32 in pairs)
3. AUMatrixMixer: /inputs (64) /in channels (64) /out channels (16)

Really it would be shame to lose any of these options from the Mac version, but it would be great to see them ported to the Windows version in some way.

I don't know how possible it is to have a cross-platform audio plug-ins, but access to a suite of 3rd party plugins including audio generators would also be a good outcome.

best wishes

Russell

mark

@bonemap said:

Really it would be shame to lose any of these options from the Mac version, but it would be great to see them ported to the Windows version in some way.

Dear Russel,

None of the Core Audio features will be removed from MacOS. So nothing you mention related to that will change. (Sound Input plug-in, AUMatrixMixer.)

The Movie Player routing will change, so this affects #1 and #2 in your second list. But it's going to be be better (in my opinion) because the pair limitation will be removed -- you'll simply be able to send any movie audio channel to any output on the device you desire.

And while I hear several of you mentioning cross-platform audio processing via plugins, remember that's something we will have to consider for a future release. As I said at the outset such a feature will not be part of Isadora 3.1.

Best Wishes,
Mark