Arduino Firmata Actor

mark

What are the advantages of serial communication for the opportunistic prototyper?

That's a separate discussion, which you can most certainly start a thread about. Let's stick to the question at hand, OK? (Most Arduinos come with a USB serial interface connector that appears as a serial connection, as you know.)

Best Wishes,
Mar

mark

@bonemap said:

And I would say there is good, workable documentation and instruction for using Arduino with the Serial In Watcher, Send Serial Data actors. It is available through Isadora knowledge base. At least enough for a non-programmer like myself to succeed.

I'm mostly going on the multiple requests I've gotten on this over the years. On the Facebook post I made about this, several educators were immediately enthusiastic, so I would see merit in finishing this up. (It was basically two days of time while I waited for the team to test other stuff.)

Best Wishes,
Mark

bonemap

@mark

Super happy to get hands on with the Arduino Firmata implementation - if it is ready let’s try it.

Best wishes

Russell

tonyschuite

@fubbi

by knowing about ESP32 you already know about the Arduino more than the focus group for this actor and I would recommend just using the serial actors.

@marc

Reading the comments here, I think there is some good insights. as to the practical question of how you would implement an auto populated actor: after populating you would set the ins and outs to "none". If you then set one of the pins to be output, you disable the input field and vice versa, so it cannot be modified anymore.

Also, in regards to it being too many options: there are a fair few actors that have this. I don't think it is that much of a worry. however, you could go the route of only show the first 4 pins and add in the "help" section that you can enable more capabilities.

mark

@tonyschuite @bonemap @liminal_andy @juriaan

So, I've listened to all you have to say, and made substantial changes to the actor. I've explained how it's working now in this video, as it was easier to do this than to write it all out.

Please let me know if you think will help make this accessible to beginners and students who might use this actor.

Best Wishes,
Mark

Juriaan

@mark

Chapeau. That looks amazing and simple enough for students/people that never touched Arduino before. Just tell the system what you want, and be done with it :)

I could not wish for anything more, I love that it also labels the names of the Input / Output of the actor with the Pin / Type of pin that you generated. PWM, Analog, Digital, Encoder and Servo.

bonemap

@mark said:

I've explained how it's working now in this video

That appears very straight forward! nice solution with the innovation of the button to generate/automate connection ports, which I don't recall being apart of any other Isadora actor. Is it just Arduino's or are other boards suitable?

So the process would be grab the Firmata Standard Arduino Code and load it onto whatever flavour of board is at hand - using the Arduino IDE or similar. Connect the board through USB to the Serial Setup in Isadora as usual. Run the Firmata actor and check for a connection by observing the 'firmware' output. Determine what input and output ports are required for the project, enter these parameters and click the generate button - start linking data throughput with the rest of the patch.

Great !

What happens if I decide I need more ports after I have generated the initial ones can I go back and add through the same process or do I need to add through the edit text parser?

Best Wishes

Russell

mark

@bonemap said:

Are other boards suitable?

This actor relies the Firmata protocol, which allows things like asking the board what it's capabilities are (e.g. how many pins do you have? what can they do?) So if the board (whatever it is) runs Firmata, it would work.

So the process would be grab the Firmata Standard Arduino Code and load it onto whatever flavour of board is at hand - using the Arduino IDE or similar. Connect the board through USB to the Serial Setup in Isadora as usual. Run the Firmata actor and check for a connection by observing the 'firmware' output. Determine what input and output ports are required for the project, enter these parameters and click the generate button - start linking data throughput with the rest of the patch.

Yes, that's it.

What happens if I decide I need more ports after I have generated the initial ones can I go back and add through the same process or do I need to add through the edit text parser?

Well, right now it would overwrite whatever's there. If you've not modified it by hand, no biggie. But otherwise you'd lose your changes.

But if @tonyschuite agrees that this would make it easy for the students he teachers, I'll get this ready for distribution so people can try it.

Best Wishes,
Mark

jfg

@mark

Great and very useful. A lot of students will love it.

best

Jean-François

DusX

@mark this is awesome!!! Really fantastic!!

bonemap

@mark

Hi,

The Actor is amazing as is but Some additional reflecting on the procedure outlined in your video. There is an element of guess work in letting the actor report back on the availability of ports. I think I would want to know what a board is capable of before making the setting port parameters and generating them. If the board rejected a parameter setting I would want to know why.

A little online research into the specification of a particular board would probably be all that is required. Another approach might be to consider having an option to reveal the full capacity of the board through the actor- so there is some clear guideline to what the board is capable of before setting port assignments. Could the process be, using the existing actor as proposed, to trigger settings with an aim to reveal all of the the boards potential ports and then pair back/assign the ports on a second pass?

Or perhaps a dialog is populated with the boards port type and capacity when I select the port number... then I would be equiped with the information required to make appropriate port assignments for the board..

The demonstrated actor is great as is - so these are just additional thoughts.

Best wishes

Russell

Armando

@mark Amazing work Mark !

The only thing I don't understand in the video is why outputs are on the right in the actor and inputs on the left....

Ciao !

bonemap

@armando said:

why outputs are on the left in the actor and inputs on the right

Hi,

I know! I did a second take of that as well! But if you think of the actor module as a simulacrum/stand-in for the board itself it makes more sense. If I have a sensor attached to the Arduino it is an input therefore it is labeled as an input in the Isadora module. If I wanted to send current information to a servo motor or a relay it is an output on the board and therefore labelled an output in the module.

Best Wishes

Russell

mark

@armando said:

The only thing I don't understand in the video is why outputs are on the right in the actor and inputs on the left....

I debated this because I knew someone would feel like it's backward. But just think about it for the second

The definition is talking about PINS, which are physical input or output pins on the Arduino board.

For a physical input pin on the Arduino board, data will enter the input pin and the board will pass the incoming data on to Isadora. Thus, you need an output on the actor.

For a physical output pin on the Arduino board, Isadora must to send data to the Arduino so it can pass that data to the output pin. Thus you and you need an input on the actor.

Refer again to the Firmata Test App screen shot:

This is defining the PINS on the Arduino board, as either inputs or outputs. The specification does the same.

But the fact that someone as experienced as you found this confusing does not encourage me that I've made the right choice. Hmmmmmm.

Best Wishes,
Mark

mark

@bonemap said:

Or perhaps a dialog is populated with the boards port type and capacity when I select the port number... then I would be equipped with the information required to make appropriate port assignments for the board..

This is a reasonable idea, but it is not doable because there is no ability for actors to show a dialog. (We're leading up to the Big Sur release, and the code is frozen while we go through final tests. I simply can't add that capability to the 3.0.8 release, which means such a new feature will need to wait for the next version.)

So what I've done is as follows: here's the new automatically generated output when you trigger the 'gen spec' input. It now inserts a list of the board capabilities.

Additionally, if one or more of the requested pins could not be generated, it inserts a comment describing which pins and how many could not be generated (see GEN SPEC ERROR.) It also shows "error: not enough pins" at the status output instead of "ready."

// GEN SPEC ERROR: Some of the requested pins could not be generated.
// GEN SPEC ERROR: for pin mode ENCODER OUTPUT, none of the 3 pins requested were generated.
// GEN SPEC ERROR: for pin mode PWM OUTPUT, only 2 of the 4 pins requested were generated.
// START OF BOARD CAPABILITIES
//
// PIN 2: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 3: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 4: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 5: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 6: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 7: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 8: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 9: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 10: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 11: DIGITAL INPUT / DIGITAL OUTPUT / PWM OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 12: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 13: DIGITAL INPUT / DIGITAL OUTPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 14: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 15: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 16: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 17: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / PULLUP INPUT
// PIN 18: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / I2C OUTPUT / PULLUP INPUT
// PIN 19: DIGITAL INPUT / DIGITAL OUTPUT / ANALOG INPUT / SERVO OUTPUT / I2C OUTPUT / PULLUP INPUT
//
// END OF BOARD CAPABILITIES
// AUTOMATICALLY GENERATED PIN SPECIFICATION
//
PIN 14 "pin 14: analog input"  ANALOG INPUT                "No help yet available."
PIN 15 "pin 15: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 16 "pin 16: analog input"  ANALOG INPUT                "No help yet available."
PIN 17 "pin 17: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 18 "pin 18: analog input"  ANALOG INPUT                "No help yet available."
PIN 19 "pin 19: analog input"  ANALOG INPUT    0     1023  "No help yet available."
PIN 2  "pin 2: digital input"  DIGITAL INPUT               "No help yet available."
PIN 3  "pin 3: digital input"  DIGITAL INPUT   0     1     "No help yet available."
PIN 4  "pin 4: digital input"  DIGITAL INPUT               "No help yet available."
PIN 5  "pin 5: digital input"  DIGITAL INPUT   0     1     "No help yet available."
PIN 6  "pin 6: digital output" DIGITAL OUTPUT              "No help yet available."
PIN 7  "pin 7: digital output" DIGITAL OUTPUT  0     1     "No help yet available."
PIN 8  "pin 8: digital output" DIGITAL OUTPUT              "No help yet available."
PIN 9  "pin 9: digital output" DIGITAL OUTPUT  0     1     "No help yet available."
PIN 10 "pin 10: pwm output"    PWM OUTPUT                  "No help yet available."
PIN 11 "pin 11: pwm output"    PWM OUTPUT      0     255   "No help yet available."

That ought to cover all the bases here.

Best Wishes,
Mark

Armando

@mark Got it. Thanks for taking the time

mark

Dear All (including @bonemap @armando @tonyschuite @juriaan @Eva)

Thanks for all your input. You'll see it reflected in this first version of the plugin for Mac and Windows.

You all get zero points if you don't read both the READ ME because it contains these critical instructions:

First you need to understand how to attach switches, sensors, LEDs, motors and whatever else you desire to the physical input and output pins on your Arduino board. If you do a Google search for Arduino tutorials you'll find dozens upon dozens of tutorials online (both video and written) that can help you learn more about how to do this. You can also jump directly to the tutorials on the main Arduino site.

Second, you need to have a basic understanding of how Firmata system works. We found this tutorial on instrucibles.com to be a good one. If you have attached devices to your Arduino input and output pins as described above, you can use the Firmata Test App to manually test the hardware connected to your Arduino input and output pins.

Once you've accomplished the above, then you can the Arduino Firmata actor to your patch and double-click it. In the editor dialog that appears, click the "Help" button to learn how setup the serial port in Isadora and automatically generate a specification that will link the input and output pins on your Arduino.

Once you've created a specification, the Arduino Firmata actor will receive data from the Arduino input pins, and send data to the Arduino output pins in a way that's similar to the Firmata Test App – except that you now have easy access to those pins in Isadora.

The first two paragraphs are there because we cannot be responsible to for teaching Isadora users how to create the hardware part of their Arduino setup. Likewise, there's an excellent tutorial about setting up and trying Firmata, and while Juriaan suggested we should write our own, for now I'm going to use the already excellent resources that exist out in the world.

So please read not only the READ ME, but also the updated help you'll find when you click the "help" button in the editor dialog.

*KNOWN BUG* This evening I discovered that Isadora 3.0.7 won't show the help file when you click the help button on Windows. I've put the HTML help file (firmata-actor-help.html) at the same level as the READ ME, and updated the READ ME to acknolwege this issue. The 3.0.8 release will address this problem. Sorry for the inconvenience.

Let me know how it goes.

~~arduino-firmata-v0.8.1.zip~~

I've moved the download to the first post at the top of this thread.

Best Wishes,
Mark

mark

Dear All (including @bonemap @armando @tonyschuite @juriaan @Eva)

I just re-uploaded v0.8.0 at 8:17pm CETon February 22, 2021. If you downloaded before this time, please download again.

The first upload of v0.8.0 had the wrong html for the help.

Then I discovered a bug showing the Help file under Windows in Isadora 3.0.7; the same help file is now at the top level of the download to allow users to work around this and the READ ME has been updated to acknowledge the bug.

Best Wishes,
Mark

bonemap

@mark said:

Let me know how it goes.

Hi Mark,

It worked! I was able to get going with the actor very quickly based on loading an Arduino UNO board with the Firmata Standard code from Arduino 1.8.3 program. The following are some quick tests...

The procedure to connect all the elements appeared to require a stepped process, which has the Firmata actor dropped into a patch AFTER setting up the Isadora Serial preferences. Is this the expected behaviour - activating port with the correct device, setting the correct baud rate and then going back into the Communications menu to check Enable Serial Ports. The above process appears to be required before the Firmata actor goes into a patch otherwise, it does not initialise effectively?

I quickly tested with two sensors: an analogue pulse sensor and a digital PIR sensor. I was unable to get both sensors reading into the same Firmata actor. They worked just fine independently and concurrently within their own Firmata actor as represented in the screengrab here.

A couple of other things that need some more testing: I also noticed an immediate crash when attempting to cut and paste the Firmata actor between patches that already had Serial in Watcher. I was also unable to assign different port numbers and devices to the Serial in Watcher and run the Firmata actor concurrently, but this might require some additional testing. Or we just say the two formats don't mix in an open patch?

Best Wishes

Russell

mark

@bonemap said:

The procedure to connect all the elements appeared to require a stepped process, which has the Firmata actor dropped into a patch AFTER setting up the Isadora Serial preferences. Is this the expected behaviour - activating port with the correct device, setting the correct baud rate and then going back into the Communications menu to check Enable Serial Ports. The above process appears to be required before the Firmata actor goes into a patch otherwise, it does not initialise effectively?

I don't think this order is required. When serial communication is disabled or undefined:

When the Firmata actor is activated (i.e, by dropping it into a Scene or by entering a Scene that contains it), it requests the firmware version from the board. In this case the 'firmware' output will say "???" because it received no response.
Similarly, if you click the 'gen spec' input, the actor obviously can't receive a response regarding the board capabilities, and the 'status' output will be stuck on 'get board info'.

Both of these would be the expected behavior if the serial port isn't properly set up and enabled.

But after dropping the actor into the scene enabling the serial ports caused my Arudino (running StandardFirmata.ino v2.5) to send the firmware report automatically. So for me the following sequence worked in a new patch.

Create New Patch and add Arduino Firmata actor.
Add Arduino Firmata - firmware output says '???' because it is not receiving data from the board.
Choose Communications > Serial Port Setup... and setup serial port and baud rate.
Choose Communications > Enable Serial Ports

After a moment, the board sends its firmware data and the 'firmware' output changed to 'v2.5'. In addition, because the default settings on the actor requested four analog inputs, and I have a sensor on A0, the "pin 14: analog input" started outputting data ope the actor.

quickly tested with two sensors: an analogue pulse sensor and a digital PIR sensor. I was unable to get both sensors reading into the same Firmata actor. They worked just fine independently and concurrently within their own Firmata actor as represented in the screengrab here.

Did you try using those two inputs at the same time in the Firmata Test App mentioned in the documentation? Do they work there?

I also noticed an immediate crash when attempting to cut and paste the Firmata actor between patches that already had Serial in Watcher.

I tried the reproduce this but was unable. Crash reports are appreciated if you can zero in on the report related to this crash. I did however get multiple dialogs saying The input buffer for the Serial In Watcher - Text' actor is full! Accumulated data has been discarded. That's because the Serial In Watcher - Text actor is going to be very confused by the Firmata data, as it has no line endings. This error would be expected if the two actors are on the same port and you're receiving Firmata data on that port.

Or we just say the two formats don't mix in an open patch?

As long as they are on different serial ports, it should be OK.

I was also unable to assign different port numbers and devices to the Serial in Watcher and run the Firmata actor concurrently, but this might require some additional testing.

I would be surprised if this is the case, but frankly I don't have multiple serial devices to test.

Best Wishes,
Mark