Using sensors to control Isadora

[ccrabtree10]

Hi all,

I'm currently studying an MSc in Computing and my dissertation is on a sensor-to-Midi interface for live performance. I'm interested in what kind of technologies people use to get sensor data into Isadora.

Does anyone use I-CubeX? OSCulator? Miditron? MidiSense?
Do people program their own software for use with Arduino or Phidgets sensors?
What is good / bad about the technology you use? (e.g. ease of use, GUI, latency, reliability, compatibility, how well is it supported).

Any help would be much appreciated!!
Thanks,
Chris.

[Skulpture]

I have used the I-CubeX before. Its good but expensive!

[dbini]

i use midi controllers (Korg NanoKontrol, Behringer BCR and M Audio Oxygen 8) directly into Isadora.

Wiimote via OSCulator

LeapMotion via GECO

Kinect via NImate

and i'm considering getting a MYO for a 2015 project

Kinect is probably the most flexible, but its not very accurate and has limited range.

[Fred]

Personally I go for Arduino directly and program it myself. Form here you get a lot of options, getting the data in from ethernet (osc) with an ethernet arduino (or over wifi with a wifly- non osc formatted UDP), serial, or if you want you can convert your arduino to be a midi (or HID) device directly.

This way you can choose how you want to use the data and scale it or smooth it inside arduino. You can also use whatever combination of sensors you want. It is also possible to run arduino code on a texas instruments launchpad making things a lot cheaper.

This level of control allows you to use interrupts and callbacks to read values, which is a lot more efficient than just polling every loop. 

My real preference is to go with standard firmata and use an openframeworks template made for this to get data in and forward it over  osc- this way it runs as fast as possible in a background application and never causes a problem in your running software. These systems are still connected by serial to the host. This is not the fastest interface but even when you use ethernet on the arduino you have the same speed limitations due to the onboard controller.

From these comments, latency and resolution are my main concerns, as well as ease of programming. The above pathway takes minutes to set up and lets you get on with your actual work. It is also mac/windows/linux and raspberry pi compatible and as it sends over OSC it can send to pretty much any device, including smartphones and tablets, and any software, with low latency and high resolution. (on the PI you can also bypass the arduino and go directly form the IO pins on board).

In the end there is no real need for an interface or GUI, once you have the data output organised no GUI is needed you just get the data in and do what you want with it. The gui part is much more up to the host.

These system are so simple already the idea of another layer of abstraction (like I-cubex) is a waste of time, limiting the use to a set of sensors that may not fit your needs as well as lowering the resolution to midi standards and costing way way too much. A texas instruments launchpad and some sensors and wire can cost €30, one tenth the cost of the inflexible and lower resolution I-cube system.

I-cube, Miditron and Midisense made more sense before arduino, but now with a huge community and forums and such a simple system these products don't really have a place (creating a circuit board, USB programmer, power regulators and programming PIC chips was a big deal some years ago - now the whole system is available in a pretty much fool proof box).

Osculator will convert some types of data input to others, only needed if your system does not natively accept OSC, or neither your outputting system or inputting system allows for editing of commands for control

For an interesting setup check out Onyx Ashanti and his beat jazz controllers. 

From my experience, in the end the hardest part of the process (and one that could do with some streamlining) is all in physically mounting the sensors, especially if you want to attach them to existing sets or props. Having a flexible mount, box or clamp system for sensors and encoders would be more of an advantage in the field, as the programming and electronics part is now totally streamlined, you may have a high res rotary encoder working in a few minutes, but it can take hours to get it reliably mounted on a wheel and counting revolutions. Same goes for other encoders, badly made boxes or mounts don't live up to stage use and then your data is bad and your scene can fail.

Fred

[ccrabtree10]

Hi Graham,

Thanks for your response speedy response! What did you use the I-CubeX for? Were you using it to generate midi and send to Isadora?

Do you have any experience with other sensor-to-Midi/OSC (or another protocol?) interfaces?

Thanks.

[Skulpture]

Hi,

Yes I used various sensors to send MIDI into isadora.

I use Kinect, Leap Motion, software such as Ni-mate, Z-vector also.

I also use the [Makey Makey](http://www.makeymakey.com/) a lot too. 

This may be a good read for you: http://vjskulpture.wordpress.com/2011/02/13/the-problems-with-motion-tracking

[ccrabtree10]

Hello again Graham, thanks for your response, I'll have a read of your article tonight after work.

dbini - thanks for your response. When you use those applications/hardware, what would you say is good/bad about them? and could anything be improved? I'm trying to gather some user requirements for my own product at the moment, so I'm interested to know what is good and bad about existing systems.

Thanks Fred, for your very detailed response! However, I'm interested to know why you think systems like I-CubeX don't (or shouldn't) have a place in media performance. The setup you described is technically better than some systems such as I-CubeX, but to build it, you must have a good knowledge of hardware/computing. Many performance artists don't know (and don't want to know) anything about computers, so doesn't I-CubeX offer a layer of abstraction to those individuals, so they can use the system, without having to get into too much detail about how it actually works?

Once again, thanks for all your responses, they've been a great help!

[vanakaru]

I never got I-CubeX. It was way over my budget and if I would have not find Arduino I would use cameras for sensors only. For years!

So while I-Cubex is real PRO product - reliable and understandable and well documented. - its cost may be become an obstacle for an artist who just starting explore the possibilities of interaction.

Arduino is 10 times less expensive. And you need to program I-Cubex as well(although not by writing the code). C++ is not that difficult to learn and you can find working codes on internet so easy now that you can start playing almost right away. Also if you do not wish to deal with soldering stuff you can buy ready made sensor shields. For me the product ”off scratch” is more fun than nice packaged Pro stuff.

And when OSCulator came about it opened up Wii and all that send OSC. I have done so many fun installations by just using WiiMotes.

BTW Makey Makey is made with the same hardware as Arduino. I have used the same sensor idea with Arduino and 16 sensors instead of 9.

So I use mostly Arduino for specific physical sensors - I like piezo triggers.

Wii and OSCulator has many possibiliteis however Wii itself is not so easy to hack and mount.

And simple webcam is probably the most flexible and easy sensor to use. It is not MIDI sending per se but with Isadora you can make it to send midi if needed to control other apps of tools. I have done many projects with controlling video in Modul8 using webcam-Isadora-MIDI.

[Armando]

I've also use a lot of wireless sensors from interface-z the 11 bit 20148 values midi hf (existing very reliable actors for Isadora and very cheap 98 euros for transmitter and receiver, goodbye arduino. Ah and reliable long distance)  wiwi (max/msp only) but also the very professional Kroonde. Cube too (no direct osc in Isadora but objects for max.....). Best in town? And the winner is.... Mini hf by interface-z because transmitter is a little bigger than a 9v Battery. 8 analog sensors plus 10 buttons possible on one transmitter. 2 frequencies in hf this means 2 different sets on 2 different dancers (if it is a dance project) Long distance (regular theatre) low price (400 euros for nicely package transmitter + receiver, custom actors in isadora and 4092 values fast midi. What do you need more?

[gapworks]

@ Skulpture

makey makey is a great tool but have you found a way to ground objects within themselves? as i´m working on an installation i would need this option. 

[vanakaru]

@gapworks

The idea of Makey does not allow this. You have a connection bridged with very large resistor 10MOhm or more that Arduino is set to read 0\. Your body resistance is less than that and thus Arduino reads more than 0 when you bridge the connention. If you set Arduino to react to the value over some threshold you get a trigger(MIDI note, switch etc). I have thought about this too and the only solution I found for my installation/object was to have person touch the ground connection without notice.

Look fot capacitance sensors for more discreet setup.