Recognizing spoken words

Armando

Hello,

any idea on how to make Isadora recognize spoken words? Talking in a microphone is izzy capable to recognize the word and trigger something?

Thanks

Armando

Skulpture

I imagine you would have to use a third party somewhere and somehow write to a file and then import them... maybe a .txt file? Data array? Not sure really. That's a big task. Sounds like something @Fred would do ;)

Maximortal

This is a workaround: use some software like IMITONE (they transform your sing in midi notes)then use a different pitch tp pronunce the words, it recognize not the word but tonality than map isadora's triggers to different midi note.

crystalhorizon

Thats clever

jandraka

Hello,

There is a section in System Preferences named "Dictation". You can turn it on and select the input. If you use Quartz Composer you can use a composition as attached and the "Text" field activate the Dictation function (Fn twice). Talk and takes seconds to recognize it. You press intro and say another phrase. If you use "Syphon to texture" you can handle the text as a layer.

I hope it helps.

095d55-text.zip

Skulpture

@jandraka way is the kind of route i'd take; but i imagine there is a delay and it wont be 100% perfect.

Fred

Apart from this Kinect V2 (proper sdk for MS only) has pretty good voice recognition and it is not to hard to use- you can send OSC to Izzy to let it know that a voice command has been activated, or of course the old mac system. My advice would be to use a lapel mic to avoid some confustion. I had voice controlled Izzy several years ago (using apple script) it worked ok, was clunky as the voice recognition supplied was. Also google have a service you can upload files to (well kind of you need to do some work but I think you can get there without complex code) to do speach to text. The recognition is a lot better and contextual, I used it a few times, you can see a way to use it in this code

https://github.com/fx-lange/ofxGSTT

Pretty simple and you can do it without OF if you need. Or you could do it with OF and get the strings back- this engine understands more accents and does a better job.

Fred

Maximortal

I found another solution using your smartphone:

the chain is: Tasker now - tasker - usb OTG cable - USB midi adapter

Tasker now is a plugin for tasker that intecept words in google now

Tasker is...all that you want :-)

for more infos look at http://tasker.dinglisch.net/userguide/en/midi.html

EDIT: you can do it without a midi interface..

you record two different sine waves (e.i. 200 Hz and 15 kHz) and store into phone

then you set in tasker to play the first when you say "first" and second to "second"

then you connect you smartphone in the audio input of isadora and use sound frequences analisys actor

next steps are obiouvs

Of course you need an android phone with tasker and tasker now

Armando

Thank you all.

Just to give a feedback to so many generous suggestions I've tried MAXIMORTAL suggestion. But it doesn't woks for me because peuple should really sing a specific note associate with a word. And I Don't want to teach music and text, And I'l like people to express more freely.

Second maximortsl option is very interesting, But android only.....

Fred as always has solutions that make me dream (and sewat) but it seems like a think that could work although out of my reach. (I'll hire people to develop and report to the community in case I go this way).

I"d go: text to speach, Data array text comparator.

I'll tell when I try

DusX

Heres a python module that offers some Windows options: https://github.com/t4ngo/dragonfly

Adding an OSC module for communication with Isadora would make this a workable option.

Armando

Thanks DusX,

but I'm on a mac plus I don't lump how to use this But thanks anyway. This can help windows people after all.

Ciao

jandraka

Hi Armando,

I do not know if this can help you. It is a video in which I use Dictation, Quartz Composer and Isadora. I use the words "dos", "tres" and "cuatro" in Spanish to toggle the video sources. You must enter a sound to insert text in Quartz Composer, but the sound can be more discreet. In Dictation you can choose the audio input source: Internal microphone, line input ... Dictation is quite fast recognizing the words but not infallible.

If it helps, I can share the Quartz Composer patch.

Sorry for my English.

Best regards.

e523a7-20150701_002731.mp4.zip

Armando

Wow. video is impresing. I'll try

Thanks

gapworks

any more details or maybe an example about you quartz patch! your concept sounds really interessting!

best

jandraka

Hi,

I use several chained applications. The result would be better if I could automatically insert text in Text Edit.

I use Dictation, Text Edit, Quartz Compser and Isadora.

First, you have to make some adjustments in Dictation, in System Preferences. You have to activate Dictation (I have selected the option to activate by pressing "fn" twice).

Then, also in the System Preferences, Accessibility, go to Dictation and then to Dictation Commands, "+" press and create a new user command. This command allows you to save changes in Text edit by slight murmur (see sceenshoots 1, 2, and 3).

When these adjustments are made, we can open Text files recogn.txt, .qtz, .izz. Now with Dictation function (appears a small square with a microphone), and from the Text Edit file, we have to go inserting any text. By slight breathing after the text, we will saving Text Edit document and therefore entering text in Quartz Composer. Here, in Quartz Composer, comparison with our keywords occurs and if the result is true, a MIDI event is triggered.

It is important that Kineme String With File patch matches the file path to the .txt document.

The Kineme plugin, you must install it in user / Library / Graphics / Quartz Composer Patches

I understand it's not a very clear settings but it works. The biggest problem is that Dictation sometimes understand anything.

I hope this can be of some use to you.

2aaa44-text-recognition.zip