Audio File Functionality - Opinions Please

[mark]

Dear All,

Actually, if you don't mind, I'd like to ask for some opinions on how the whole "Audio Files" business works in Isadora, because this area of the software needs to be improved.

**Gory Details**

When I originally wrote this part of Isadora way back in the olden days of 1999 with Mac OS 9, my intent was to use the Audio files like a mini-sampler. The idea was that you could load in short samples of uncompressed audio and play trigger them instantaneously. For my work at the time with piece that used the MidiDancer system to interactively trigger my musical compositions, it was important.

So this is how it worked: when you entered a scene, it started loading samples into RAM. This was done in a in the background process, starting the 'loop start' point. If you triggered the sound, it would start playing from the 'loop start' point and the samples would be ready since the OS loaded the samples faster than you could play them. This worked fine for all the stuff I was doing at the time.

And why in RAM? To support the crossfade feature and chunk feature. Remember, before 2002, there was only one Isadora user: me. These special features were used in pieces I was making at the time. But does anybody use them now?

Another oft asked question: why do MP3 and other compressed audio files end up in the Movies section? Because, for the 'crossfade' and 'chunk features' to work, you need the sound in RAM. If these compressed audio files were imported into the Audio Files section of the Media Window, I'd have to decompress the whole file. That's going to be  which is going to be even slower than if it's already uncompressed. (Not to mention developing the code to do the decompression -- not so hard given that it's in QuickTime, but still, yet another task.) I never intended anyone to use Audio Files to play long stretches of sounds since that the samples are being loaded into RAM. I'd alway use an audio only movie for that. But since most people don't read the manual, they don't often realize this is a "best practice."

One final problem with this approach – especially for longer files – would occur when one used an Enter Scene Trigger connected to a Random actor to randomize the 'loop start' point when entering a Scene. In this case, Isadora would dutifully behave as described above, loading samples from the  'loop start' point she saw when the Scene was activated. But then, if you changed the 'loop start' once the scene was active, the samples at that point wouldn't be ready.

**Solutions**

I still think the idea of having samples that can be instantaneously triggered is a good one. But obviously there is confusion above using long files, and problems when you change the loop start point. So I'm seeing three possibilities.

Option 1: Preload "On the Fly"

This is how it works now, though I've radically changed the preloading code for 1.3.0f25 to make it work faster and thus reduce the amount of time it takes to get the samples into the buffer in the first place.

As before, if you don't preload the sound, but simply Jump, playback will be fine if you don't modify the 'loop start' point. If you intend to immediately modify the 'loop start' point, you would need to use the Sound Preload actor to ensure the entire sound is in RAM before you jump. 

A further problem with this approach: when you're using two sound players to play the same sound with two different 'loop start' points. In this case, the Sound Player actor that is higher in the Scene wins: it's 'loop start' point would determine where the load begins. For longer files, that means the samples for the second Sound Player may not be ready.

I must admit, unless you really know what you're doing, this approach lacks elegance.

Option 2: Preload the Whole Sound

Isadora's preloading mechanism has gotten a lot more sophisticated since 1.3.0f17 -- e.g., when you Jump to a new a scene, all the movies are preloaded in the background before the Scene changes. So one answer is to do this with the sound files: ensure the **entire** sound file is loaded before you jump into the next scene. For files of a few dozen megabytes, this is no problem. But if you have an hour long (600MB+) audio file, it can literally take 15+ seconds or more to load the entire sound into memory on my spiffy Mac Book Pro i7\. In this latter case, Isadora seems "stuck" because it's won't jump until the preload is complete.

As I have been experimenting with this idea, I added a warning dialog that shows up when you import an AIFF or WAVE file that is 100MB or larger. It warns you that it may take a long time to load the Scene, and encourages you to read Tutorial 8 in the manual which explains the Audio File section of Isadora.

There are two advantages to this method: first, no matter what you do so the 'loop start'/'loop length' inputs once your in the scene, it will work, because the entire sound file is already in RAM. Second, if you import a long AIFF/WAVE file, you'll learn (from the warning dialog) that they work in a bit differently than other media, and maybe even read the manual about that topic. At the least, users will know something is not as they might expected; at best, they'll be prompted to get a better understanding of how the whole thing works.

Option 3: Revamp the Whole Thing

The final approach would be to revamp the whole thing. Dump the "play in RAM" approach, lose the crossfade and chunk features, and simply play the files from disk. Thing is, you can do this already by using an audio only movie. But because there is generally a misunderstanding of the difference between how the Audio Files part of Isadora works, it would at least result in a consistency that feels better.

Under this option, mp3 files and other compressed audio would end up in the Audio Files section of the media Window, which would certainly make a lot more sense to most people.

But of course, we lose something that existed in previous versions. I am always loathe to do that.

----------------

Well, that's my little diatribe for today. If anyone has some clever thoughts about it, I'd be happy to hear your comments.

Best Wishes,

Mark

[vanakaru]

Thank you for this nice and clear explanation.

As I see it we have all the tools available already. If I can use preload small audio samples and play these without a problem as done before and have .mov for longer files I will be satisfied with this the most.

Option 2 would be fine too with warning dialog. And we have this discussion to point our confused users to.

Maybe there should be an Actor that is actual Movie Player(Sound Movie Player) but with different name(like Mp3 Player or Long Audio Player)  without ”video” output and listed under Audio section. And when importing Mp3 a dialog that will say where this file needs to be placed(Video files).

[Fred]

Yes thanks for the explanation. I had a suggestion a little outside of your scope. Could it be possible to have a load to ram tag in the media bin. This way we could select which files (sound or even video) we load into ram. Whichever of the files that are selected to load into ram will be loaded to ram as soon as the session is open. If you import a new file and say load to ram it will pause and load to ram when you select the load to ram box and it will keep the loading out of the performance runtime. This could obviously get complicated with a lot of file and not enough ram but appropriate warnings could be given. If the files should only be loaded close to the time that they are used maybe the load to ram tag in the media bin could have a load to ram for *****scene only, where you let the system know what you want in ram for each scene.

It would be quite some work but here you could also have an option to load an mp3 to ram and at the session start up (or if you select this check box) you could convert the file back to uncompressed and from then on treat it like a real audio file even though it started as an mp3.

Just some thoughts

Fred

[jjkanin]

Thanks for the explanation.

The important thing for a user like me is that all the audio files end up in the same bin, to be easily choosen with the bin picker.

Then I can chose the player that suits me best, One that preload into RAM or one that does not. When using a bin-picker I think i don't need to load the file into RAM. I prefer to use the sound player rather than the sound movie player only because of the progress bar at the bottom of the player.

That's a beginners opinion

/Jan

[lpearse]

Options, options...

Would there be a way to combine these choices?  Having the option to preload entirely into RAM would be great, but especially when working with older machines with less RAM it would be good to be able to stream from disk, wouldn't it? 

I like Fred's suggestion about making it an option in the Media Bin.  Could there be an audio only bin, even if it is playing back some files like a movie? Just for clarity. 

[CraigAlfredson]

I think the whole thing could be resolved simply by allowing the sound media player to play aif's and wav's (or having a carbon copy of the actor to maintain separate bins).  Perhaps a change in nomenclature would help novices choose the right actor, for example rename the Sound Player to Sound Looper or something and change the Sound Movie Player to Sound Media Player.

Incidentally, I almost always just use regular old Movie Players to play my audio (as mp3's or audio only movies) since it shows me the progress bar and I can use the frequency bands to affect other parameters.

[DusX]

I like the idea of keeping the current functionality, and perhaps following the above naming recommendations to make things more clear.
If another bin type was added to differentiate native audio from compressed audio that might also be usefull?

[CitizenJoe]

Hi Mark,

I like Fred's suggestion, too.  But...

Would it be an idea to keep what you have (perhaps updating the code as you wish) and create a completely new actor, "Audio File Player" or "CD Player" or something, that offers the functionality of a CD player? It might have Play, Pause, Stop, forward, reverse controls.  I'd want to keep all the other things that are there now, loop enable, play start, play length, etc., but not the chunk stuff.  It would be really, really lovely if it had a timer built into it, as well!  Just like a CD Player.  This would be really simple, operationally, for people who simply want to plplayn audio track.

I'd also love to seen an Audio Mixer Actor in which we could adjust all the things that one does in a mixer, such as EQ...  Or maybe, to keep it simple an "EQ Actor"?  Actually, an EQ Actor makes sense.  I don't have a clue what that means from a programming point of view, though!

I realize this doesn't directly answer your question, but it does address the things I really miss in the audio section of Isadora.

Cheers,

Hugh

[Michel]

I like the idea of CitizenJoe, leave as it is and add a new actor that acts like a cd player.
@citizenjoe the audio core version has an EQ.

Best,
Michel

[Skulpture]

I agree with @Michel on @citizenjoe comments. A CD like actor would fit the bill for me.

I think it would be more intuitive for new users/student also.

[CitizenJoe]

@Michel,  I laughed when I read your comment about The Core version having an EQ Actor.  Of course, I'm on Win7!

Cheers,

Hugh

[CraigAlfredson]

I would also love to see the frequency band outputs that exist on the movie player actor on the sound player and the AU sound player.

While we are on the subject, it would be great to have an audio output on the movie player, so I could run the audio on movies through core audio effects.