Parsing json get request - parsing data question

allrak · 2018-02-07 20:45:14 UTC

I’m making a GET request, and I’m able to pull the data into $response, which shows up in the console.
I’m having a problem parsing the actual data, for example, $response.router_time

Does it have to go into an array or something?

edit: I’m guessing it’s not working right due to the double-colon (::) in the data I’m parsing

Here’s what a browser request returns:

{router_time::Wed, 07 Feb 2018 14:36:34}{mem_info::,'total:','used:','free:','shared:','buffers:','cached:','Mem:','260644864','63578112','197066752','0','6000640','15876096','Swap:','0','0','0','MemTotal:','254536','kB','MemFree:','192448','kB','MemShared:','0','kB','Buffers:','5860','kB','Cached:','15504','kB','SwapCached:','0','kB','Active:','12580','kB','Inactive:','10900','kB','MemAvailable:','209732','kB','Active(anon):','2116','kB','Inactive(anon):','0','kB','Active(file):','10464','kB','Inactive(file):','10900','kB','Unevictable:','0','kB','Mlocked:','0','kB','HighTotal:','131072','kB','HighFree:','119788','kB','LowTotal:','123464','kB','LowFree:','72660','kB','SwapTotal:','0','kB','SwapFree:','0','kB','Dirty:','0','kB','Writeback:','0','kB','AnonPages:','2116','kB','Mapped:','2616','kB','Shmem:','0','kB','Slab:','17656','kB','SReclaimable:','1340','kB','SUnreclaim:','16316','kB','KernelStack:','592','kB','PageTables:','236','kB','NFS_Unstable:','0','kB','Bounce:','0','kB','WritebackTmp:','0','kB','CommitLimit:','127268','kB','Committed_AS:','4628','kB','VmallocTotal:','1949696','kB','VmallocUsed:','0','kB','VmallocChunk:','0','kB'}{clkfreq::1000}{uptime:: 14:36:34 up 43 days, 2:11, load average: 0.01, 0.02, 0.00}{ip_conntrack::182}{cpu_temp::CPU 59.7 °C / WL0 48.3 °C / WL1 54.8 °C}{voltage::}{ipinfo:: IP: 11.11.11.145}{nvram::43 KB / 128 KB}

2/7/2018, 3:37:51 PM +241ms
+1ms	╔Starting piston... (v0.2.102.20180116)
+282ms	║╔Subscribing to devices...
+385ms	║╚Finished subscribing (114ms)
+415ms	╚Piston successfully started (415ms)
2/7/2018, 3:37:22 PM +352ms
+1ms	╔Received event [Home].time = 1518035843086 with a delay of -734ms
+156ms	║RunTime Analysis CS > 17ms > PS > 46ms > PE > 92ms > CE
+158ms	║Runtime (37387 bytes) successfully initialized in 46ms (v0.2.102.20180116) (157ms)
+159ms	║╔Execution stage started
+160ms	║╚Execution stage complete. (1ms)
+161ms	╚Event processed successfully (161ms)
2/7/2018, 3:37:03 PM +643ms
+1ms	╔Received event [Home].wc_async_reply = httpRequest with a delay of 0ms
+112ms	║RunTime Analysis CS > 21ms > PS > 29ms > PE > 62ms > CE
+114ms	║Runtime (37392 bytes) successfully initialized in 29ms (v0.2.102.20180116) (113ms)
+115ms	║╔Execution stage started
+128ms	║║{router_time::Wed, 07 Feb 2018 15:37:03}{mem_info::,'total:','used:','free:','shared:','buffers:','cached:','Mem:','260644864','62832640','197812224','0','6000640','15679488','Swap:','0','0','0','MemTotal:','254536','kB','MemFree:','193176','kB','MemShared:','0','kB','Buffers:','5860','kB','Cached:','15312','kB','SwapCached:','0','kB','Active:','12388','kB','Inactive:','10900','kB','MemAvailable:','210264','kB','Active(anon):','2116','kB','Inactive(anon):','0','kB','Active(file):','10272','kB','Inactive(file):','10900','kB','Unevictable:','0','kB','Mlocked:','0','kB','HighTotal:','131072','kB','HighFree:','119928','kB','LowTotal:','123464','kB','LowFree:','73248','kB','SwapTotal:','0','kB','SwapFree:','0','kB','Dirty:','0','kB','Writeback:','0','kB','AnonPages:','2116','kB','Mapped:','2616','kB','Shmem:','0','kB','Slab:','17064','kB','SReclaimable:','1332','kB','SUnreclaim:','15732','kB','KernelStack:','592','kB','PageTables:','236','kB','NFS_Unstable:','0','kB','Bounce:','0','kB','WritebackTmp:','0','kB','Co...[TRUNCATED]
+131ms	║║Executed virtual command log (3ms)
+135ms	║║Error retrieving JSON data part null
+140ms	║║Executed virtual command log (2ms)
+142ms	║╚Execution stage complete. (27ms)
+143ms	╚Event processed successfully (143ms)

eibyer · 2018-02-07 21:12:39 UTC

No idea if it will work but have you tried logging $response.router_time: ?

allrak · 2018-02-07 21:21:18 UTC

Yep, that doesn’t work, it fails… Now another way I was thinking to get the data… If I set a variable $output to the value of $response, I get this:

You can see that it did populate the $output variable. I guess this can be a way to get around the non-json-compliant return data. However, I’m not sure how to parse that variable out and just grab something like “mem_info” or “router_time”

eibyer · 2018-02-07 21:23:23 UTC

You can probably squeeze it out of there using a couple of the webcore text functions, mid(), etc.

allrak · 2018-02-07 21:26:30 UTC

Yeah, was just looking at the functions page on the wiki, and now my brain hurts.

eibyer · 2018-02-07 21:40:42 UTC

I was going to try and save your output as text string and see if I could extract info but something in that output is triggering an error about a variable not found.

allrak · 2018-02-07 21:47:20 UTC

I was able to grab it using ‘mid’

dynamic output_formatted = {mid((output) , 14, 25)}

That evaluates with the correct date. I guess using ‘mid’ would be the best way to do it?
Is there any way to split the data into an array based off the positions of curly brackets?
For example, the first set of curly brackets contains the router_time I’m looking for, so could I separate every bracket’ed string into an array and call it using $output_formatted[0] ?

EDIT: ‘mid’ is useful, but actually won’t work. The character offsets will dynamically change due to the data such as IP address, uptime (can increase the character count by multiple digits), etc…

eibyer · 2018-02-07 22:06:12 UTC

The last time I had to extract info, I had to use indexOf/lastIndexOf to find the positions of my markers in combination with mid().

Edit: I meant to ask earlier, what kind of router is it and what command are you sending.

allrak · 2018-02-07 22:18:20 UTC

I’m running DDWRT on a netgear R7000p

I’m issuing a get request to <IP>://Status_Router.live.asp which returns the data shown in my first post.
Really this is simply a learning exercise for me; I want to get better at webcore, and essentially using functions, arrays, etc…
I see value in being able to pull data from outside of the ST realm and use that data for “stuff”

eibyer · 2018-02-07 22:24:35 UTC

I was going to try the command on a couple of ddwrt routers here at work and I just realized I have no hub here… oops

allrak · 2018-02-07 22:25:32 UTC

Any idea why this doesn’t evaluate?

mid(lastIndexOf((output), “IP”), 10)

$output has the long string of data

lastIndexOf((output), “IP”) <- That works and evaluates to “1354”

eibyer · 2018-02-07 22:31:05 UTC

Try mid(output, lastIndexOf((output), “IP”), 10)

Edit: not sure if you have to pad it by 1, I had to in my test before… mid(output, (lastIndexOf((output), “IP”) + 1), 10)

allrak · 2018-02-07 22:32:35 UTC

This works ^^
Also, the padding works too in order to eliminate the starting "IP:"
mid(output, lastIndexOf((output), “IP” ) +3, 14)

allrak · 2018-02-07 22:33:50 UTC

My problem is just trying to read the nested functions in a logical way that will work in my brain.

Robin · 2018-02-07 23:28:25 UTC

Looks like you have it working, but I wonder if using the replace( ) function might be of help?

replace(output,‘{‘,‘’,’}’,’,’)

This should remove the { and change the } into commas.

You could then use arrayItem()

arrayItem(0,output) should return the Router_Time part.

allrak · 2018-02-08 14:00:07 UTC

I see where you’re going with that, but there’s a comma already in the router_time field that’ll throw it off.
Two questions:

Can I use a different character other than comma to replace the brackets and have arrayItem() recognize it? (I’m assuming arrayItem() separates everything by commas.
Did I miss arrayItem on the wiki somewhere? Don’t see it listed on the functions page.

allrak · 2018-02-08 14:39:42 UTC

Also, it looks like those brackets need to be escaped with a backslash in the expressions:
replace(output, '\{', ',' , '\}', ',')

Robin · 2018-02-08 16:38:06 UTC

It will take comma separated strings or when using a device variable, splits by device. I dont think it splits by anything else… but maybe experiment a little??

You could always replace the existing commas for full stops and then run through again replacing the } for commas and { for null.

Yeah… not on the wiki yet… there were a bunch of functions added mid-development which didn’t find their way there… will look to do some updates when I get time.

infofiend · 2018-03-29 20:38:50 UTC

I understand that updating the wiki would be time consuming - how about just a list of these additional functions?