Arduino Firmata Actor

mark

Are other boards suitable?

This actor relies the Firmata protocol, which allows things like asking the board what it's capabilities are (e.g. how many pins do you have? what can they do?) So if the board (whatever it is) runs Firmata, it would work.

So the process would be grab the Firmata Standard Arduino Code and load it onto whatever flavour of board is at hand - using the Arduino IDE or similar. Connect the board through USB to the Serial Setup in Isadora as usual. Run the Firmata actor and check for a connection by observing the 'firmware' output. Determine what input and output ports are required for the project, enter these parameters and click the generate button - start linking data throughput with the rest of the patch.

Yes, that's it.

What happens if I decide I need more ports after I have generated the initial ones can I go back and add through the same process or do I need to add through the edit text parser?

Well, right now it would overwrite whatever's there. If you've not modified it by hand, no biggie. But otherwise you'd lose your changes.

But if @tonyschuite agrees that this would make it easy for the students he teachers, I'll get this ready for distribution so people can try it.

Best Wishes,
Mark

jfg

@mark

Great and very useful. A lot of students will love it.

best

Jean-François

DusX

@mark this is awesome!!! Really fantastic!!

bonemap

@mark

Hi,

The Actor is amazing as is but Some additional reflecting on the procedure outlined in your video. There is an element of guess work in letting the actor report back on the availability of ports. I think I would want to know what a board is capable of before making the setting port parameters and generating them. If the board rejected a parameter setting I would want to know why.

A little online research into the specification of a particular board would probably be all that is required. Another approach might be to consider having an option to reveal the full capacity of the board through the actor- so there is some clear guideline to what the board is capable of before setting port assignments. Could the process be, using the existing actor as proposed, to trigger settings with an aim to reveal all of the the boards potential ports and then pair back/assign the ports on a second pass?

Or perhaps a dialog is populated with the boards port type and capacity when I select the port number... then I would be equiped with the information required to make appropriate port assignments for the board..

The demonstrated actor is great as is - so these are just additional thoughts.

Best wishes

Russell

Armando

@mark Amazing work Mark !

The only thing I don't understand in the video is why outputs are on the right in the actor and inputs on the left....

Ciao !

bonemap

@armando said:

why outputs are on the left in the actor and inputs on the right

Hi,

I know! I did a second take of that as well! But if you think of the actor module as a simulacrum/stand-in for the board itself it makes more sense. If I have a sensor attached to the Arduino it is an input therefore it is labeled as an input in the Isadora module. If I wanted to send current information to a servo motor or a relay it is an output on the board and therefore labelled an output in the module.

Best Wishes

Russell

mark

@armando said:

The only thing I don't understand in the video is why outputs are on the right in the actor and inputs on the left....

I debated this because I knew someone would feel like it's backward. But just think about it for the second

The definition is talking about PINS, which are physical input or output pins on the Arduino board.

For a physical input pin on the Arduino board, data will enter the input pin and the board will pass the incoming data on to Isadora. Thus, you need an output on the actor.

For a physical output pin on the Arduino board, Isadora must to send data to the Arduino so it can pass that data to the output pin. Thus you and you need an input on the actor.

Refer again to the Firmata Test App screen shot:

This is defining the PINS on the Arduino board, as either inputs or outputs. The specification does the same.

But the fact that someone as experienced as you found this confusing does not encourage me that I've made the right choice. Hmmmmmm.

Best Wishes,
Mark

mark

@bonemap said:

Or perhaps a dialog is populated with the boards port type and capacity when I select the port number... then I would be equipped with the information required to make appropriate port assignments for the board..

This is a reasonable idea, but it is not doable because there is no ability for actors to show a dialog. (We're leading up to the Big Sur release, and the code is frozen while we go through final tests. I simply can't add that capability to the 3.0.8 release, which means such a new feature will need to wait for the next version.)

So what I've done is as follows: here's the new automatically generated output when you trigger the 'gen spec' input. It now inserts a list of the board capabilities.

Additionally, if one or more of the requested pins could not be generated, it inserts a comment describing which pins and how many could not be generated (see GEN SPEC ERROR.) It also shows "error: not enough pins" at the status output instead of "ready."

// GEN SPEC ERROR: Some of the requested pins could not be generated.
// GEN SPEC ERROR: for pin mode ENCODER OUTPUT, none of the 3 pins requested were generated.
// GEN SPEC ERROR: for pin mode PWM OUTPUT, only 2 of the 4 pins requested were generated.
// START OF BOARD CAPABILITIES
//
// PIN 2: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 3: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 4: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 5: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 6: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 7: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 8: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 9: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 10: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 11: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 12: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 13: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 14: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 15: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 16: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 17: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 18: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / I2C OUTPUT / PULLUP INPUT
// PIN 19: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / I2C OUTPUT / PULLUP INPUT
//
// END OF BOARD CAPABILITIES
// AUTOMATICALLY GENERATED PIN SPECIFICATION
//
PIN 14 "pin 14: analog input"  ANALOG INPUT                "No help yet available."
PIN 15 "pin 15: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 16 "pin 16: analog input"  ANALOG INPUT                "No help yet available."
PIN 17 "pin 17: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 18 "pin 18: analog input"  ANALOG INPUT                "No help yet available."
PIN 19 "pin 19: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 2  "pin 2: digital input"  DIGITAL INPUT               "No help yet available."
PIN 3  "pin 3: digital input"  DIGITAL INPUT   0     1     "No help yet available."
PIN 4  "pin 4: digital input"  DIGITAL INPUT               "No help yet available."
PIN 5  "pin 5: digital input"  DIGITAL INPUT   0     1     "No help yet available."
PIN 6  "pin 6: digital output" DIGITAL OUTPUT              "No help yet available."
PIN 7  "pin 7: digital output" DIGITAL OUTPUT  0     1     "No help yet available."
PIN 8  "pin 8: digital output" DIGITAL OUTPUT              "No help yet available."
PIN 9  "pin 9: digital output" DIGITAL OUTPUT  0     1     "No help yet available."
PIN 10 "pin 10: pwm output"    PWM OUTPUT                  "No help yet available."
PIN 11 "pin 11: pwm output"    PWM OUTPUT      0     255   "No help yet available."

That ought to cover all the bases here.

Best Wishes,
Mark

Armando

@mark Got it. Thanks for taking the time

mark

Dear All (including @bonemap @armando @tonyschuite @juriaan @Eva)

Thanks for all your input. You'll see it reflected in this first version of the plugin for Mac and Windows.

You all get zero points if you don't read both the READ ME because it contains these critical instructions:

First you need to understand how to attach switches, sensors, LEDs, motors and whatever else you desire to the physical input and output pins on your Arduino board. If you do a Google search for Arduino tutorials you'll find dozens upon dozens of tutorials online (both video and written) that can help you learn more about how to do this. You can also jump directly to the tutorials on the main Arduino site.

Second, you need to have a basic understanding of how Firmata system works. We found this tutorial on instrucibles.com to be a good one. If you have attached devices to your Arduino input and output pins as described above, you can use the Firmata Test App to manually test the hardware connected to your Arduino input and output pins.

Once you've accomplished the above, then you can the Arduino Firmata actor to your patch and double-click it. In the editor dialog that appears, click the "Help" button to learn how setup the serial port in Isadora and automatically generate a specification that will link the input and output pins on your Arduino.

Once you've created a specification, the Arduino Firmata actor will receive data from the Arduino input pins, and send data to the Arduino output pins in a way that's similar to the Firmata Test App – except that you now have easy access to those pins in Isadora.

The first two paragraphs are there because we cannot be responsible to for teaching Isadora users how to create the hardware part of their Arduino setup. Likewise, there's an excellent tutorial about setting up and trying Firmata, and while Juriaan suggested we should write our own, for now I'm going to use the already excellent resources that exist out in the world.

So please read not only the READ ME, but also the updated help you'll find when you click the "help" button in the editor dialog.

*KNOWN BUG* This evening I discovered that Isadora 3.0.7 won't show the help file when you click the help button on Windows. I've put the HTML help file (firmata-actor-help.html) at the same level as the READ ME, and updated the READ ME to acknolwege this issue. The 3.0.8 release will address this problem. Sorry for the inconvenience.

Let me know how it goes.

~~arduino-firmata-v0.8.1.zip~~

I've moved the download to the first post at the top of this thread.

Best Wishes,
Mark

mark

Dear All (including @bonemap @armando @tonyschuite @juriaan @Eva)

I just re-uploaded v0.8.0 at 8:17pm CETon February 22, 2021. If you downloaded before this time, please download again.

The first upload of v0.8.0 had the wrong html for the help.

Then I discovered a bug showing the Help file under Windows in Isadora 3.0.7; the same help file is now at the top level of the download to allow users to work around this and the READ ME has been updated to acknowledge the bug.

Best Wishes,
Mark

bonemap

@mark said:

Let me know how it goes.

Hi Mark,

It worked! I was able to get going with the actor very quickly based on loading an Arduino UNO board with the Firmata Standard code from Arduino 1.8.3 program. The following are some quick tests...

The procedure to connect all the elements appeared to require a stepped process, which has the Firmata actor dropped into a patch AFTER setting up the Isadora Serial preferences. Is this the expected behaviour - activating port with the correct device, setting the correct baud rate and then going back into the Communications menu to check Enable Serial Ports. The above process appears to be required before the Firmata actor goes into a patch otherwise, it does not initialise effectively?

I quickly tested with two sensors: an analogue pulse sensor and a digital PIR sensor. I was unable to get both sensors reading into the same Firmata actor. They worked just fine independently and concurrently within their own Firmata actor as represented in the screengrab here.

A couple of other things that need some more testing: I also noticed an immediate crash when attempting to cut and paste the Firmata actor between patches that already had Serial in Watcher. I was also unable to assign different port numbers and devices to the Serial in Watcher and run the Firmata actor concurrently, but this might require some additional testing. Or we just say the two formats don't mix in an open patch?

Best Wishes

Russell

mark

@bonemap said:

The procedure to connect all the elements appeared to require a stepped process, which has the Firmata actor dropped into a patch AFTER setting up the Isadora Serial preferences. Is this the expected behaviour - activating port with the correct device, setting the correct baud rate and then going back into the Communications menu to check Enable Serial Ports. The above process appears to be required before the Firmata actor goes into a patch otherwise, it does not initialise effectively?

I don't think this order is required. When serial communication is disabled or undefined:

When the Firmata actor is activated (i.e, by dropping it into a Scene or by entering a Scene that contains it), it requests the firmware version from the board. In this case the 'firmware' output will say "???" because it received no response.
Similarly, if you click the 'gen spec' input, the actor obviously can't receive a response regarding the board capabilities, and the 'status' output will be stuck on 'get board info'.

Both of these would be the expected behavior if the serial port isn't properly set up and enabled.

But after dropping the actor into the scene enabling the serial ports caused my Arudino (running StandardFirmata.ino v2.5) to send the firmware report automatically. So for me the following sequence worked in a new patch.

Create New Patch and add Arduino Firmata actor.
Add Arduino Firmata - firmware output says '???' because it is not receiving data from the board.
Choose Communications > Serial Port Setup... and setup serial port and baud rate.
Choose Communications > Enable Serial Ports

After a moment, the board sends its firmware data and the 'firmware' output changed to 'v2.5'. In addition, because the default settings on the actor requested four analog inputs, and I have a sensor on A0, the "pin 14: analog input" started outputting data ope the actor.

quickly tested with two sensors: an analogue pulse sensor and a digital PIR sensor. I was unable to get both sensors reading into the same Firmata actor. They worked just fine independently and concurrently within their own Firmata actor as represented in the screengrab here.

Did you try using those two inputs at the same time in the Firmata Test App mentioned in the documentation? Do they work there?

I also noticed an immediate crash when attempting to cut and paste the Firmata actor between patches that already had Serial in Watcher.

I tried the reproduce this but was unable. Crash reports are appreciated if you can zero in on the report related to this crash. I did however get multiple dialogs saying The input buffer for the Serial In Watcher - Text' actor is full! Accumulated data has been discarded. That's because the Serial In Watcher - Text actor is going to be very confused by the Firmata data, as it has no line endings. This error would be expected if the two actors are on the same port and you're receiving Firmata data on that port.

Or we just say the two formats don't mix in an open patch?

As long as they are on different serial ports, it should be OK.

I was also unable to assign different port numbers and devices to the Serial in Watcher and run the Firmata actor concurrently, but this might require some additional testing.

I would be surprised if this is the case, but frankly I don't have multiple serial devices to test.

Best Wishes,
Mark

bonemap

@mark said:

try using those two inputs at the same time in the Firmata Test App

Hi,

[EDIT: we now have our own version of the Firmata Test App that is compatible with Big Sur. If you're on any version of macOS, you should be using this version. The latest version can always be found in the releases section of Firmata Test App on GitHub]

The link to the Firmata Test App appears to lead to a MacOS .dmg file from 2012 that will not load onto my computer.

I have a bit of gear here Uno and Micro boards with a variety of sensors and relays. Keen to test this out and get a great result from it. Is there a Firmata Test App for Mac OS 11 that I have missed? or do I need to find an older operating computer to test with?

Best Wishes

Russell

mark

@bonemap said:

The link to the Firmata Test App appears to lead to a MacOS .dmg file from 2012 that will not load onto my computer.

The minimum system version inside the app says 10.4. What version of macOS are you trying to run it on? I haven't tried it on Big Sur, but it works great for me on Mojave.

Is it a security issue? Did you try control-clicking the app and choosing "Open" and then saying it's OK to run it?

Best Wishes,
Mark

bonemap

@mark said:

tried it on Big Sur

Yes, I am on Big Sur with these patches because I am trying to help you out with beta testing. Anyway, I have persisted with the testing on Mac OS 11.1 so far this is what I have...

Tested PIR sensor to single digital input pin - appears to be working well.

Tested Pulse Sensor to single analogue pin - this worked initially but now I have a persistent signal noise on only one analogue pin. the noise persists even no sensor connected to the boards but I can’t appear to make any other analogue input pins active. I have deleted the plugin and reinstalled ... deleted preferences/ restarted computer but the noise on only a single analogue pin persists. It also appears the same on a second port with a Firmata Actor when placed in the scene editor. Because of this I can't recreate the success I had this morning reading an analogue Sensor.

I have been unable to use two boards with the Firmata actor at all. I can assign the boards in the serial port setup and can access them concurrently through Serial in Watcher actors when setup in a separate Isadora file. But with the Firmata actor only one board appears to initialise through one of the Firmata actors the other Firmata actor remains with "???" or "getting board info". I have tried this about a dozen times with no success.

I have also had no success with any analogue inputs since first using the plugin this morning with a Pulse Sensor. I have several Pulse Sensors and I can get none of them to work anymore on analogue pins.

I have also attempted to use 3 pin accelerometers with the analogue pins through the Firmata actor to no avail. Just to be sure I ran two boards with accelerometers each using Serial In Watchers and this worked without any problem so I know the boards and the sensors are all functioning correctly with standard Arduino sketches and serial communication.

I also got an error in the text editor for Line 44 but I could not scroll any further than Line 43 - so I don't know if there was a Line 44 that was outside the bounds of the scroll or if something else was going on.

I was able to get digital output to operate a relay and PMW to operate a led without any problems.

This is looking good.

Best Wishes

Russell

Eva

Hi:)
So, for now, I have tested with an Arduino Mega 2560 and at least there is a signal comming through. But also I had problems getting anything out of the pulse generator. I get one signal, that is definetly not my pulse... (it the little heart beat thingy;)

I also, today, tested a bare conductive board, but with that, It tells me that Standard Firmata is uploaded, but I cannot test it afterwards, all I see is a blank app when I choose the board. Also, the board doesn't work if I try in Isadora. I tried to look it up, and it seems that one needs to do something else with the Bare Conductive board because it has its own programming.
I had some problems replying here, but now it somehow works...

All the best
Eva

bonemap

@mark said:

it works great for me on Mojave

OK! I have wound my testing back to my MBP with 10.14.6 Mojave and incorporated testing with the Test Firmata app. I can see now why the test Firmata app is so critical, but it is hardly future proof having to go back 2x Mac OS releases to access a critical test app - it is not going to be great moving forward. We really need a solution for Catalina and Big Sur - don't we?

I could not find any clear specification online about which boards are supported by Firmata protocol and which are not. I have a couple of Arduino Micro boards (which I love using because they are so small, using a magnifying glass to read the PIN numbers etc.), but these don't appear to be recognised by the Firmata Test app in Mac OS Mojave. However, the Arduino Micro I tested on Big Sur machine did allow me to successfully control PMW and digital outputs through Isadora - just not the analogue inputs.

The Arduino Uno appeared in the Firmata Test app on Mojave and I could see the analogue pins working with an accelerometer attached. So it is unclear why the Arduino Uno failed to read the analogue inputs when attempting to access through the Firmata actor in Isadora. On both test machines I could only get a reading from one analogue input at a time - perhaps I have something wrong with these inputs?. Not having the Firmata test app for Mac OS beyond Mojave 10.14 makes the task a bit hit and miss.

Best Wishes

Russell

liminal_andy

@mark said:

@tonyschuite @bonemap @liminal_andy @juriaan
So, I've listened to all you have to say, and made substantial changes to the actor. I've explained how it's working now in this video, as it was easier to do this than to write it all out.
Please let me know if you think will help make this accessible to beginners and students who might use this actor.
Best Wishes,
Mark

I think you nailed it :). These are precisely the changes needed to open up the actor to a lot of entry-level use cases. Absolutely love it. It may still be interesting in the future to have a dedicated firmware file and user actor that, while rigid and restrictive, can extend to boards that do not support Firmata specifically but do support the general firmware language (thinking of some of those ESP32s, etc.).

Fabulous actor! I need to aside aside more time to play with it...

mark

@liminal_andy said:

but do support the general firmware language (thinking of some of those ESP32s, etc.).

So, ESP32's do not support Firmata? (I know zero about these boards. Thanks for educating me.)

Best Wishes,
Mark