[SOLVED] Input Parser size limit?

bonemap

@robhblack

We will have to make a feature request for this.

Best wishes

Russell

mark

Dear All,

So I recompiled the Text Parser to have a buffer of 256K instead of 2k. It solves your problem. But the issue on MacOS is, if you install it, the "code signature" of the app will become invalid and -- depending on the version of macOS you're running -- it might not allow Isadora to run. On Windows, it should not be an issue.

So, attached you will find the Mac and Windows plugins in a .zip file. Follow the instructions about Manually Installing a Plugin which say Isadora 1 + 2, but are valid for Isadora 3 too.

Text Parser.zip

If the operating system refuses to launch Isadora after installing the plugin, re-install Isadora using the standard installer.

I hope it helps.

Best Wishes,

Mark

photogramdude

@mark said:

Manually Installing a Plugin

IT WORKS!!!

Much appreciated, I'll dive back into this tomorrow and share progress.

bonemap

@mark said:

I hope it helps.

Thanks @mark

I was able to get the new module to parse the data to the last line. I am using Mac OS Mojave and Isadora 3.07 stable.
I did have one issue: I was unable to parse the ° degree symbol (shift-option-8) Isadora hard crashed most times I tried to include the symbol in the character-set range.

I have marked the data and string in red on this test patch: smartcitizenParse_degreeSymbol.izz

I have a Mac OS Catalina machine arriving in a couple of days, I might be onto other things by then otherwise I will try the new Text Parser there as well.

Best Wishes

Russell

photogramdude

@bonemap Thanks, this is what I came up with, the degree symbol is not posing a problem albeit my iMac has pretty much latest MacOS. Your version is more elegant with less redundancies. My plan was to torture test with a few different sensor networks (some have soil sensors etc so would need a javascript logic based interpreter to grab additional individual component sensors. currentcitizen.izz

bonemap

@robhblack said:

the degree symbol is not posing a problem

It could be the cause of my ° symbol issue is that my keyboard is set to 'Australian'? I tried your string but still not parsing it.

I have also put together a 'dynamic data' demonstration patch that uses open data from the World Bank: DynamicData.zip

best wishes

Russell

mark

@bonemap said:

I did have one issue: I was unable to parse the ° degree symbol (shift-option-8) Isadora hard crashed most times I tried to include the symbol in the character-set range.

Yes, this is no going to work because this actor is not really unicode savvy. (It converts all strings to UTF-8, and the ° symbol ends up being more than one byte long in that reprepsentation. So you can't actually include it in the character set you're using inside of the square brackets (i.e., [-\" a-z _ A-Z 0-9:[]] )

But I feel in general that the technique you're using to skip a fixed number of bytes is maybe not the best way to do it. It would be better to parse for the label and then the data after it. Take a look at the simplified example file below:

This is the input string

{"description":"Temperature","unit":"ºC"}

Here is the parsing code with comments (which cannot of course be included in the real parsing string.)

"{" -- skip the starting bracket
"\"description\":" -- skip text "description":
"\"" -- skip opening double quote
desc : string=[^\"] -- accept all characters until the next double quote and output that string to the desc output
"\"" -- skip ending double quote
"," -- skip comma
"\"unit\":" -- skip text "unit:"
"\"" -- skip opening double quote
unit : string=[^\"] -- accept all characters until the next double quote and output that string to the unit output
"\"" -- skip ending double quote
"}" -- skip ending bracket

as you can see, this correctly grabs "°C" because it is not depending on matching particular characters like a-z, etc.

You could write this even more elegantly as follows

"{"
"\"" [^\"] "\":" -- skip any text in the form "xxx":
"\"" desc : string=[^\"] "\"" -- grab a parameter in the form "xxx" and output as desc
"," -- skip a comma
"\"" [^\"] "\":" -- skip any text in the form "xxx":
"\"" unit : string=[^\"] "\""-- grab a parameter in the form "xxx" and output as unit
"}"

Maybe this will allow you to improve you parsing and grab the stuff you need.

Best Wishes,
Mark

parsing-suggestion.izz

bonemap

@mark said:

this will allow you to improve you parsing

Hi @mark

Yes it will allow some improvement. That is a much appreciated insight. I did attempt something like that but will definitely give it another go.

best wishes

Russell

mark

@robhblack and @bonemap

I've just made your life a lot easier.

Get the new JSON Parser Actor

Best Wishes,
Mark

bonemap

@mark said:

Get the new JSON Parser Actor

Hi Mark,

That is remarkable - I am spinning!!

Best wishes

Russell