TCP Actors & REST Interface to Database (Neo4j)

Jim_Nevin

Hi Mark,
Attached is a text file into which I pasted the full actual text from the Monitor window.
For background info, I'm testing with Isadora 1.3.1.f02 on Mac OS X 10.7.5, 8GB RAM.
The TCP-Send-Data actor contains the string: "GET http://localhost:7474/db/data/\n\nAccept: application/json\n"
...which you will also see in the top portion of the Monitor text.
The TCP-In-Watcher (Text or Binary) actors do not appear to be firing the "msg rcv" outputs, nor am I showing any parsed parameter data from the TCP response.
Thanks for taking a look,
Jim Nevin
d20afd-tcp-actors-rest-monitor-capture.txt

mark

Dear Jim,

Looking at this data, the end of line character seems to be decimal 10 / hex 0a -- which is the linefeed character. (This would mean Unix style linefeeds.) What happens if you set the eom (end of message) character to 10?

Best Wishes,

Mark

P.S. Is there a public IP address where I can try this? Or only localhost on your machine?

Jim_Nevin

Hey Mark,

I set the eom in my TCP-In-Watcher-Text actor to 10 with the same result. The extraction pattern I used to test in this actor to test is currently set to:

testmsg:string="extensions"

...and for sanity, I connected its msg-rcv output to a Trigger Text actor just to make sure I wasn't missing the fired trigger.

Now only localhost -- I'll see if I can get it somewhere accessible on a public IP address and let you know.

mark

Well, the other thing you could do is to put the request in your browser (maybe) and just post the text that results. If I can get that, I can simulate the transaction here without getting a public IP address.

Does it work if you just enter

[http://localhost:7474/db/data/\n\nAccept](http://localhost:7474/db/data/\n\nAccept): application/json\n

into your browser? If so, copy and paste the text into a text file and send it.

Best Wishes,

Mark

mark

Dear Jim,

Nevermind. I wrote an AppleScript to convert the hex data from the Monitor Window into a text file so I can work on this. More soon.

Best Wishes,

Mark

mark

Dear Jim,

OK. I have some advice for you. First, let's just say you want to output each line of the input text separately. In the TCP In Watcher Text you would enter:

output:string={00-FF}

the pattern {00-FF} will match any sequence of characters between hex 00 and hex FF -- in other words, _any_ character. Feeding your input text into this will output each individual line in rapid succession.

But what you really want to do is to get the data for a specific line. Let's use this line as an example:

"execute_query" : "http://localhost:7474/db/data/ext/CypherPlugin/graphdb/execute_query"

The key to understanding how this works is to remember that 1) whitespace is what separates the "tokens", and 2) all whitespace is ignored. With this in mind, we have the following tokens in the above line of text:

"execute_query"

:

"http://localhost:7474/db/data/ext/CypherPlugin/graphdb/execute_query"

(Note that the double-quotes here are literal -- they are part of the token.) To output this line only, you must match the first three tokens, and then use a variable pattern to match the final token (the URL). So, in the TCP In Watcher - Text actor you would enter

{20}

"\"execute_query\""

": "

output:string={00-FF}

For the first token you use {20} – this will match any number of spaces. (Note that there must be at least one space character for this to work! If there are no leading spaces, then the pattern match will fail and the line will be ignored.)

For the second token, you need to match the double-quotes that are part of the text _execute_query_. (The double-quotes are part of the text because they are not separated from the text by whitespace.) So you must use the "escape" character (the backwards slash) to include a double-quote inside the text you wish to match. Thus the pattern _"\"execute_query\""_ which means: a double-quote, followed by the characters _execute_query_ followed by a double-quote.

The next pattern matches the third token, the colon character followed by a space. (We need to match the space because the next pattern will match _any_ character, and we don't want to include the space.)

Finally, the fourth line will match any characters up to the end of the line, which is the URL.

When you feed your entire text response into the TCP In Watcher - Text actor using the above pattern, the output will be the URL including the double quotes that surround it.

To remove the quotes, you can use a Text Parser actor (which works exactly like the TCP In Watcher parser). Inside the Text Parser actor, the pattern would be

{22} output:string={00-21, 23-FF}

The pattern {22} matches a double-quote character, which is hex 22. (Good to have an[ ASCII lookup table](http://web.cs.mun.ca/~michael/c/ascii-table.html) handy for this stuff.) The pattern {00-21, 23-FF} matches any character _except_ a double-quote. In other words, we skip the first quote and then read characters until we find another double-quote and then we stop. Thus, the output of the Text Parser is the URL without the quotes.

So, there you go. I've attached an example patch which uses the Text Parser actor as indicated above. Note that the _eom char_ is 13 (a carriage return) instead of 10\. Keep the _eom char_ in the TCP In Watcher set to 10.

In conclusion, I'll say that I know this stuff isn't easy -- these actors were designed to be able to parse _any_ kind of data, and so they are not very friendly if you're not a programmer. But hopefully this example will serve to help others use these actors more effectively.

Best Wishes,

Mark

3c095c-parse-complex-text.izz

Jim_Nevin

Mark,

Thanks so much for the amazing support and the great mini-tutorial on parsing the TCP response content. This will definitely help!

There's actually another different problem:

... regardless of which EOM character I use for the Text TCP-In-Watcher, or if instead I use the Binary TCP-In-Watcher and say set the "msg len" input to 2048 (with TCP Stream Control buf size at 2048 or 4096) the Watcher actor is not firing "msg rcv" or showing matched-pattern on the Watcher actor's output (even using your excellent patterns.)

This is the case even though, when I trigger the request in the TCP-Send-Data actor, I see the correct response in the Izzy Monitor.

Shouldn't I at least see "msg rcv" firing in this case?

Is there an example Izzy patch for the TCP actors I can use for a sanity-check verification?

Again, thanks!

Jim_Nevin

FYI, I'm seeing the above issue while using Izzy 1.3.1f02 on Mac OS 10.7.5 8GB RAM

mark

Dear Jim,

If you set the EOM to 10 and you use the parsing pattern

output:string={00-FF}

you don't see the "msg rcv" blinking?

Well, because of the need to use TCP/IP, I don't think I can help further unless you find some way to let me get at the IP address from here in Germany. I need to be able to try it live.

Best Wishes,
Mark

Jim_Nevin

Nope, no msg-rcv blink.

I'm working on setting a TCP thingie you can reach.

Will keep you posted.

Thanks again,

j

mark

Dear Jim,

You should be able to set this up using [dyndns.org](http://dyn.com/dns/)

Once you do this, don't publish the sever name here... send it to me in a private message and let then I'll see what I can find out.

Best Wishes,

Mark