assurance-tunnel
assurance-tunnel
assurance-tunnel
assurance-tunnel

[SOLVED] Input Parser size limit?


  • Beta Platinum

    @robhblack

    We will have to make a feature request for this.

    Best wishes

    Russell



  • Dear All,

    So I recompiled the Text Parser to have a buffer of 256K instead of 2k. It solves your problem. But the issue on MacOS is, if you install it, the "code signature" of the app will become invalid and -- depending on the version of macOS you're running -- it might not allow Isadora to run. On Windows, it should not be an issue.

    So, attached you will find the Mac and Windows plugins in a .zip file. Follow the instructions about Manually Installing a Plugin which say Isadora 1 + 2, but are valid for Isadora 3 too. 

    Text Parser.zip

    If the operating system refuses to launch Isadora after installing the plugin, re-install Isadora using the standard installer.

    I hope it helps.

    Best Wishes,

    Mark



  • @mark said:

    Manually Installing a Plugin

     IT WORKS!!!

    Much appreciated, I'll dive back into this tomorrow and share progress.


  • Beta Platinum

    @mark said:

    I hope it helps.

     Thanks @mark 

    I was able to get the new module to parse the data to the last line. I am using Mac OS Mojave and Isadora 3.07 stable.
    I did have one issue: I was unable to parse the ° degree symbol (shift-option-8) Isadora hard crashed most times I tried to include the symbol in the character-set range.

    I have marked the data and string in red on this test patch: smartcitizenParse_degreeSymbol.izz

    I have a Mac OS Catalina machine arriving in a couple of days, I might be onto other things by then otherwise I will try the new Text Parser there as well.

    Best Wishes

    Russell



  • @bonemap Thanks, this is what I came up with, the degree symbol is not posing a problem albeit my iMac has pretty much latest MacOS. Your version is more elegant with less redundancies. My plan was to torture test with a few different sensor networks (some have soil sensors etc so would need a javascript logic based interpreter to grab additional individual component sensors. currentcitizen.izz


  • Beta Platinum

    @robhblack said:

    the degree symbol is not posing a problem

     It could be the cause of my ° symbol issue is that my keyboard is set to 'Australian'? I tried your string but still not parsing it. 

    I have also put together a 'dynamic data' demonstration patch that uses open data from the World Bank: DynamicData.zip

    best wishes

    Russell



  • @bonemap said:

    I did have one issue: I was unable to parse the ° degree symbol (shift-option-8) Isadora hard crashed most times I tried to include the symbol in the character-set range.

     Yes, this is no going to work because this actor is not really unicode savvy. (It converts all strings to UTF-8, and the ° symbol ends up being more than one byte long in that reprepsentation. So you can't actually include it in the character set you're using inside of the square brackets (i.e., [-\" a-z _ A-Z 0-9:[\]] )

    But I feel in general that the technique you're using to skip a fixed number of bytes is maybe not the best way to do it. It would be better to parse for the label and then the data after it. Take a look at the simplified example file below:

    This is the input string

    {"description":"Temperature","unit":"ºC"}

    Here is the parsing code with comments (which cannot of course be included in the real parsing string.)

    "{" -- skip the starting bracket
    "\"description\":" -- skip text "description":
    "\"" -- skip opening double quote
    desc : string=[^\"] -- accept all characters until the next double quote and output that string to the desc output
    "\"" -- skip ending double quote
    "," -- skip comma
    "\"unit\":" -- skip text "unit:"
    "\"" -- skip opening double quote
    unit : string=[^\"] -- accept all characters until the next double quote and output that string to the unit output
    "\"" -- skip ending double quote
    "}" -- skip ending bracket

    as you can see, this correctly grabs "°C" because it is not depending on matching particular characters like a-z, etc.

    You could write this even more elegantly as follows

    "{"
    "\"" [^\"] "\":" -- skip any text in the form "xxx":
    "\"" desc : string=[^\"] "\"" -- grab a parameter in the form "xxx" and output as desc
    "," -- skip a comma
    "\"" [^\"] "\":" -- skip any text in the form "xxx":
    "\"" unit : string=[^\"] "\""-- grab a parameter in the form "xxx" and output as unit
    "}"
    

    Maybe this will allow you to improve you parsing and grab the stuff you need.

    Best Wishes,
    Mark

    parsing-suggestion.izz


  • Beta Platinum

    @mark said:

    this will allow you to improve you parsing

     Hi @mark 

    Yes it will allow some improvement. That is a much appreciated insight. I did attempt something like that but will definitely give it another go.

    best wishes

    Russell



  • @robhblack and @bonemap

    I've just made your life a lot easier.

    Get the new JSON Parser Actor

    Best Wishes,
    Mark


  • Beta Platinum

    @mark said:

    Get the new JSON Parser Actor

     Hi Mark,

    That is remarkable - I am spinning!! 

    Best wishes

    Russell