• Isadora
  • Get it
  • Forum
  • Help
  • ADD-ONS
  • Newsletter
  • Impressum
  • Dsgvo
  • Impressum
Forum

Navigation

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Popular
    • Tags

    [ANSWERED] Pulling realtime numerical data from a webpage

    How To... ?
    5
    13
    686
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • dbini
      dbini last edited by mark

      Hello folks,


      Is it possible to get realtime data into Isadora from the variables in a website? specifically the radiation data from Fukushima found here: https://realtime.safecast.org/

      John Collingswood
      taikabox.com
      2013 MBPR 2.3GHZ i7 OSX11.6.4 16GB
      & 2019 MBPT 2.6GHZ i7 OSX12.3 16GB

      1 Reply Last reply Reply Quote 0
      • Woland
        Woland Tech Staff last edited by

        @DusX is the king of this if I recall correctly

        TroikaTronix Technical Support
        New Support Ticket Link: https://support.troikatronix.com/support/tickets/new
        TroikaTronix Support Policy: https://support.troikatronix.com/support/solutions/articles/13000064762
        TroikaTronix Add-Ons Page: https://troikatronix.com/add-ons/

        | Isadora 2.6.1 + 3 | Mac Pro (Late 2013), macOS 10.14.6, 3.5GHz 6-core, 1TB SSD, 64GB RAM, Dual AMD FirePro D700s | Macbook Pro (Retina, 15", Mid 2015), macOS 10.11.4, 2.8GHz Intel Core i7, 16GB RAM, Intel Iris Pro 1536 MB |

        dbini 1 Reply Last reply Reply Quote 0
        • dbini
          dbini @Woland last edited by

          @woland

          i assumed Ryan would be the person to ask, but i felt a bit cheeky asking him directly. i guess it can be done with Javascript or Websockets or something that i don't have a clue about.....

          John Collingswood
          taikabox.com
          2013 MBPR 2.3GHZ i7 OSX11.6.4 16GB
          & 2019 MBPT 2.6GHZ i7 OSX12.3 16GB

          1 Reply Last reply Reply Quote 1
          • Juriaan
            Juriaan Tech Staff last edited by

            It depends a bit. If the webpage can be queried using JSON it is quite simple to do inside Isadora, if this isn't the case the question is or we can ask for the entire webpage using an TCP call and scrap the contents of the returned query to look for the content that we need. (Normally we search for the DIV since most of the time they are unique enough) and then get the elements behind that. 

            Tl;dr; If it isn't JSON, possible but requires some thoughts / some knowledge regarding parsing strings using JavaScript and knowledge about the TCP actors of Izzy

            Isadora 3.1.1, Dell XPS 17 9710, Windows 10
            Interactive Performance Designer, Freelance Artist, Scenographer, Lighting Designer, TroikaTronix Community moderator
            Always in for chatting about interaction in space / performance design. Drop me an email at hello@juriaan.me

            1 Reply Last reply Reply Quote 2
            • Juriaan
              Juriaan Tech Staff last edited by

              https://api.safecast.org/

              They have an JSON API :)

              Isadora 3.1.1, Dell XPS 17 9710, Windows 10
              Interactive Performance Designer, Freelance Artist, Scenographer, Lighting Designer, TroikaTronix Community moderator
              Always in for chatting about interaction in space / performance design. Drop me an email at hello@juriaan.me

              DusX 1 Reply Last reply Reply Quote 3
              • DusX
                DusX Tech Staff @Juriaan last edited by DusX

                @juriaan

                Just what I was going to post.

                @dbini you can use the 'Get URL Text' actor to grab this data, and then process the data in a Javascript actor (this will allow you to grab the bits you want).

                This URL get you the recent measurements: https://api.safecast.org/measu...

                And this tutorial will help you deal with the data quickly: https://support.troikatronix.c...
                You will want to ungroup it.

                Troikatronix Technical Support

                • New Support Ticket Link: https://support.troikatronix.com/support/tickets/new
                • Isadora Add-ons: https://troikatronix.com/add-ons/
                • My Add-ons: https://troikatronix.com/add-ons/?u=dusx

                Running: Win 10 64bit, i7-8750H, M.2 PCIe SSD's, 16gb DDR4-2666, nVidia GTX 1070 | located in Ontario Canada.

                1 Reply Last reply Reply Quote 2
                • dbini
                  dbini last edited by

                  Ryan, Juriaan, thankyou, that's a top quality result. I'm on it. wish me luck.....

                  cheers,

                  john

                   

                  John Collingswood
                  taikabox.com
                  2013 MBPR 2.3GHZ i7 OSX11.6.4 16GB
                  & 2019 MBPT 2.6GHZ i7 OSX12.3 16GB

                  1 Reply Last reply Reply Quote 0
                  • DusX
                    DusX Tech Staff last edited by

                    Here is a quick showing of the data coming in.

                    Note I am using 2 versions of the Javascript Split to break out the data.. 
                    The second is the same as in the tutorial. The other 'upgraded for objects' is new, and includes a loop to deal with data groupings seen as objects in javascript.

                    Troikatronix Technical Support

                    • New Support Ticket Link: https://support.troikatronix.com/support/tickets/new
                    • Isadora Add-ons: https://troikatronix.com/add-ons/
                    • My Add-ons: https://troikatronix.com/add-ons/?u=dusx

                    Running: Win 10 64bit, i7-8750H, M.2 PCIe SSD's, 16gb DDR4-2666, nVidia GTX 1070 | located in Ontario Canada.

                    1 Reply Last reply Reply Quote 1
                    • DusX
                      DusX Tech Staff last edited by DusX

                      Below is the upgraded code to handle the Objects. (note I only use this is required since the loop adds a little extra processing)

                      function debug(printString){
                          if (DebugMode){
                              print(printString + "\n");  
                          }
                      }
                      

                      function main()
                      {
                      DebugMode = false; //example usage: debug("string" + var + "\n");

                      // ensure input is not EMPTY
                      if (arguments[0].length < 1){
                          debug("arg0 length: " + arguments[0].length + "\n");
                          return array;
                      }
                      var parsed = JSON.parse(arguments[0]);
                      var array = [];
                      
                      for (elem in parsed ) {
                          if (typeof parsed[elem] == 'object') {
                              parsed[elem] = JSON.stringify(parsed[elem])
                          }
                         array .push(parsed[elem]);
                      }
                      array.unshift(array.length); // make the first output value 
                                                  //a count of additional outputs needed
                      return array;
                      

                      }

                      That should all be 1 script.. not sure why the format breaks it up.

                      Troikatronix Technical Support

                      • New Support Ticket Link: https://support.troikatronix.com/support/tickets/new
                      • Isadora Add-ons: https://troikatronix.com/add-ons/
                      • My Add-ons: https://troikatronix.com/add-ons/?u=dusx

                      Running: Win 10 64bit, i7-8750H, M.2 PCIe SSD's, 16gb DDR4-2666, nVidia GTX 1070 | located in Ontario Canada.

                      bonemap 1 Reply Last reply Reply Quote 3
                      • bonemap
                        bonemap Izzy Guru @DusX last edited by bonemap

                        @dusx

                        I love this thread!

                        But how is that Trigger Text output being formatted?

                        Does it need a Text Formatter to get the final output?


                        Best Wishes

                        Russell

                        http://bonemap.com | https://github.com/bonemap | Australia
                        Izzy 3 STD/USB 3.2.1 | MBP 16” 2019 2.4 GHz i9 64GB AMD Radeon Pro 5500 8 GB 4TB SSD | OSX 12.2 Monterey

                        1 Reply Last reply Reply Quote 1
                        • Juriaan
                          Juriaan Tech Staff last edited by

                          Hi there, 

                          I just spent some time on creating this little user actor that allows you to specify the area that you want to look and the distance. If you get any results it will return the smallest, biggest, sum, average and amount of sensors in that area.

                          This is meant as an example file that you can dig apart and see how to add new filters (like for example perhaps you want to get the readings of a few months ago, you only have to add the filter to the URL and you will get that data back from the system)


                          Let me know if you have any questions :)

                          IzzySafeCast.izz

                          Isadora 3.1.1, Dell XPS 17 9710, Windows 10
                          Interactive Performance Designer, Freelance Artist, Scenographer, Lighting Designer, TroikaTronix Community moderator
                          Always in for chatting about interaction in space / performance design. Drop me an email at hello@juriaan.me

                          1 Reply Last reply Reply Quote 3
                          • DusX
                            DusX Tech Staff last edited by DusX

                            Attached are TXT versions of the 2 SplitOutputs Scripts in a Zip (since dbini had some trouble getting it from the page).

                            JS_SplitOutputs_scripts.zip

                            You should be able to just copy past the code in to the JavaScript actor :)

                            @Juriaan s example is perfect for showing you how to create URLs for specific data sets, as well as processing that data. 
                            However, I suspect you will want to use the Split Output scripts first since they allow you to more easily see the data. 
                            Another option would be to copy the entire data set output from the get URL text actor, and paste it into an editor that give you JSON formatting.
                            https://jsonformatter.org does a nice job of this. Paste the copied JSON on the left, read the data on the right.

                            Troikatronix Technical Support

                            • New Support Ticket Link: https://support.troikatronix.com/support/tickets/new
                            • Isadora Add-ons: https://troikatronix.com/add-ons/
                            • My Add-ons: https://troikatronix.com/add-ons/?u=dusx

                            Running: Win 10 64bit, i7-8750H, M.2 PCIe SSD's, 16gb DDR4-2666, nVidia GTX 1070 | located in Ontario Canada.

                            1 Reply Last reply Reply Quote 0
                            • dbini
                              dbini last edited by

                              Thanks Ryan and Juriaan, I've studied your actors, but I'm afraid the internal workings of the Javascript actor still remain a complete mystery to me. 

                              As far as I can tell, the data from each of the radiation monitors consists of something like this:

                                {
                                  "id": 78490936,
                                  "user_id": 759,
                                  "value": 18,
                                  "unit": "cpm",
                                  "location_name": null,
                                  "device_id": 100012,
                                  "original_id": 78490936,
                                  "measurement_import_id": null,
                                  "captured_at": "2017-01-22T07:23:26.000Z",
                                  "height": null,
                                  "devicetype_id": null,
                                  "sensor_id": null,
                                  "station_id": null,
                                  "channel_id": null,
                                  "latitude": 37.659,
                                  "longitude": 140.459
                                },

                              some of them seem to be measuring different things or working with different units, so its not sensible to sample all of the data on the page at the same time.

                              I think that what would be useful for this project would be to have an actor where i could trigger a specific ¨id¨ to an input and receive the associated current  ¨value¨ - maybe with the option to sample a number of devices simultaneously (like 8 inputs and 8 outputs)

                              would one of you be able to make me an actor (or TXT) that does that? i would be eternally grateful..... :)

                              (i tried triggering specific longditude/latitude numbers into Juriaan's actor but it was outputting something that didn't correlate, and i don't know why)

                              John Collingswood
                              taikabox.com
                              2013 MBPR 2.3GHZ i7 OSX11.6.4 16GB
                              & 2019 MBPT 2.6GHZ i7 OSX12.3 16GB

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post