Error retrieving JSON data part null

Aaroneden · 2018-12-26 16:56:23 UTC

1) Give a description of the problem
I am trying to get the “sid” portion of the data response, but every variation I try seems to fail with error " Error retrieving JSON data part null"

2) What is the expected behavior?
I expect to the my $sid variable populated with $response.data.sid

3) What is happening/not happening?
Errors

4) Post a Green Snapshot of the piston

5) Attach any logs (From ST IDE and by turning logging level to Full)
12/26/2018, 9:52:02 AM +296ms
+2ms ╔Starting piston… (v0.3.109.20181207)
+563ms ║╔Subscribing to devices…
+644ms ║╚Finished subscribing (88ms)
+676ms ╚Piston successfully started (674ms)
12/26/2018, 9:43:06 AM +765ms
+0ms ╔Received event [Home].test = 1545842586764 with a delay of 0ms
+61ms ║RunTime Analysis CS > 21ms > PS > 27ms > PE > 13ms > CE
+64ms ║Runtime (37912 bytes) successfully initialized in 27ms (v0.3.109.20181207) (61ms)
+65ms ║╔Execution stage started
+72ms ║║Cancelling statement #1’s schedules…
+91ms ║║Sending external web request to: XXXXANONOMYZEDXXXXX
+895ms ║║Executed virtual command httpRequest (811ms)
+905ms ║║{“data”:{“is_portal_port”:false,“sid”:“tuwwxL2R7GWgo1730PVN232400”},“success”:true}
+906ms ║║Executed virtual command log (2ms)
+912ms ║║Error retrieving JSON data part null
+918ms ║║Executed virtual command setVariable (3ms)
+929ms ║║Executed virtual command log (2ms)
+935ms ║╚Execution stage complete. (870ms)
+938ms ╚Event processed successfully (937ms)

eibyer · 2018-12-26 17:06:46 UTC

I wonder if the fancy " is throwing it off?

Edit: Yup, I just pasted it into a json parser and it didn’t like it, replace all the fancy " and it accepted it as a valid json

Aaroneden · 2018-12-26 18:28:54 UTC

So I tried doing a replace and the ParseJSON is definitely executing without error now. I still can’t get to the Sid value though. Any other suggestions?

Log
12/26/2018, 11:26:53 AM +460ms
+0ms ╔Received event [Home].test = 1545848813459 with a delay of 0ms
+68ms ║RunTime Analysis CS > 24ms > PS > 26ms > PE > 18ms > CE
+71ms ║Runtime (38615 bytes) successfully initialized in 26ms (v0.3.109.20181207) (69ms)
+72ms ║╔Execution stage started
+80ms ║║Cancelling statement #1’s schedules…
+99ms ║║Sending external web request to: wiki.aaroneden.com:5000/webapi/auth.cgi?api=SYNO.API.Auth&method=Login&version=4&account=admin&passwd=VictoriaElena13!&session=SurveillanceStation&format=sid
+962ms ║║Executed virtual command httpRequest (870ms)
+975ms ║║Executed virtual command setVariable (3ms)
+983ms ║║{‘data’:{‘is_portal_port’:false,‘sid’:‘a0HHJovbwPMc.1730PVN232400’},‘success’:true}
+985ms ║║Executed virtual command log (2ms)
+989ms ║║Executed virtual command parseJson (1ms)
+997ms ║║Executed virtual command setVariable (3ms)
+1005ms ║║Variable ‘$cleanResponse.data.sid’ not found
+1007ms ║║Executed virtual command log (2ms)
+1012ms ║╚Execution stage complete. (939ms)
+1014ms ╚Event processed successfully (1014ms)

eibyer · 2018-12-26 18:30:29 UTC

Will try to import yours… one moment

eibyer · 2018-12-26 18:33:44 UTC

Try this in the replace() function…

replace($response,'“','"','”','"')

Aaroneden · 2018-12-26 18:48:46 UTC

Thank you very much for your help. It looks like it is being parsed now, but cleanResponse.data.sid doesn’t seem to have a value. I also tried {sid} = {cleanResponse.data[0].sid} and no dice there either.

Any other suggestions very welcome.

eibyer · 2018-12-26 19:35:38 UTC

I’m running into ternary operator errors. I think the nested data maybe the causing havoc. Maybe @ipaterson can give more guidance.

ipaterson · 2018-12-26 20:07:54 UTC

The curly quotes in the JSON seem to have appeared only when you pasted (or when webCoRE displayed) the logs, if replacing " and ' actually worked exactly as written in your piston then those are just regular straight " quotes in $response which is what you need for valid JSON.

So if webCoRE is giving you a $response formatted as text rather than an object try passing that response without modification to Parse JSON data $response then try to access $json.data.sid.

In order for webCoRE to handle that response as JSON so that you can just do $response.data.sid the server needs to identify it as JSON rather than plain text. It seems like the server may be sending a Content-Type header with a value other than Content-Type: application/json.

eibyer · 2018-12-26 22:05:03 UTC

@ipaterson is right, somewhere along the way the data is getting screwed up… try calling

https://webcore.free.beeceptor.com/sid with your original piston

|+158ms|║║Sending external web request to: webcore.free.beeceptor.com/sid|
|+555ms|║║Executed virtual command httpRequest (398ms)|
|+563ms|║║Executed virtual command setVariable (4ms)|
|+570ms|║║[data:[is_portal_port:false, sid:a0HHJovbwPMc.1730PVN232400], success:true]|
|+571ms|║║Executed virtual command log (1ms)|
|+579ms|║║Executed virtual command setVariable (3ms)|
|+586ms|║║a0HHJovbwPMc.1730PVN232400|
|+587ms|║║Executed virtual command log (2ms)|
|+590ms|║╚Execution stage complete. (449ms)|
|+591ms|╚Event processed successfully (591ms)|

Aaroneden · 2018-12-27 15:53:51 UTC

I fought with this for a couple of more hours last night and could not get it to work as JSON. Since all I was trying to do was extract the SID for a 2nd web call I went old-school and just extracted from the string.

Ultimately, this creates an integration between WebCore and Synology Surveillance Station. I can now set Home Status on and off from WebCore based on multiple presence sensors and phones that I don’t want to install special apps on (I track them based on them connecting to our Google Wifi).

Also - I thought others (and the WebCore documentation) might benefit from a couple of other insights I had from this project:

The moment that you try to utilize a string which WebCore believes is a JSON object, the parser will leap into action. You can use wrap any reference to the variable in string( to keep this from occurring.
The dynamic $response variable created after a web Get request is only active once. After you refer the variable once it will no longer have any data. If you want to use the value multiple times you need to store it in a variable.
This community rocks! Thanks to everyone for your help.

eibyer · 2018-12-27 18:40:03 UTC

That’s good that you got it worked out. I had to do some string manipulation on a weather alert also the other day. I’m still curious as to what is happening in that reply you’re getting back from the server. In my testing yesterday I just mocked the json reply from your log and it worked properly.