Cloud vs Local Processing

an39511 · 2019-04-26 13:39:22 UTC

It is nice that WebCore is running local on the HE. However I have noticed a problem when pistons have a mixture of local and cloud issued commands.

This is not necessarily a fault of WebCore but here is the issue;

Using a button to fire a piston that changes modes. (all local)
Same piston sends a message using PushOver that mode changed (cloud)

If HE is not connected to the cloud it obviously cannot complete the PushOver operation.
This is either making HE or possibly WebCore grind to a halt waiting to complete…

Is there some way to test if the HE is connected to the cloud before cloud operations are executed? This could be useful in many situations. Or if a cloud command is issued while not online just skip it and move on?

E_Sch · 2019-04-26 14:15:24 UTC

I think for pushover you are using the pushover device on HE.

I would have thought when you issue the pushover command it would return.

If it is not returning this maybe something to report in regards to the pushover device - it should return in a timely manner when you issue the command.

I think the pushover device is built in, so HE support may need a look.

I expect webcore is just calling the command - there is no easy way to timeout a device handler command from the app.

an39511 · 2019-04-26 15:21:23 UTC

I will do a little more checking to see where it is hanging. As a test I changed the piston to toggle an outlet on/off each time I press a button. If I remove the call to pushover the reaction time between the button push and the outlet turning on/off is nearly instant. If I add a call to pushover when I push the button the outlet will react once but then any further button pushes have no effect. That is why I thought it may be hanging on WebCore. I’ll see if turning on debug will work if HE isn’t connected online. Thanks for your input.

WCmore · 2019-04-26 15:26:33 UTC

I do not have a Hubitat, but I wonder if you can do a quick online query right before sending the external request.

IF $twcweather.conditions.iconCode is between 0 and 50
    Then call to pushover
END IF

E_Sch · 2019-04-26 15:39:59 UTC

twcweather does not work in HE

Would need to do some async http request if you are just testing internet connectivity

an39511 · 2019-04-26 16:20:46 UTC

Here is what the log shows. The first is the log readout of when it is connected to the cloud. The next are two pushes of the button when not connected. I am not that good at reading logs but for whatever reason it looks like the status of the switch state fails to change

LOG FROM BUTTON PUSH WHILE HE CONNECTED TO CLOUD (Turned the outlet on)
4/26/2019, 11:51:34 AM +886ms
+6ms +Received event [Samsung Button].pushed = 1 with a delay of 120ms, canQueue: true, calledMyself: false
+80ms ¦RunTime LockT > 47ms > rtDataT > 9ms > pistonT > 41ms > CE
+98ms ¦Runtime (7465 bytes) successfully initialized in 50ms (v0.3.10a.20190223)
+123ms ¦+Execution stage started
+138ms ¦¦Comparison (integer) 1 is_equal_to (integer) 1 = true (3ms)
+141ms ¦¦Condition #2 evaluated true (10ms)
+144ms ¦¦Condition group #1 evaluated true (state did not change) (13ms)
+148ms ¦¦Cancelling statement #3’s schedules…
+199ms ¦¦Executed physical command [Shop Air Conditioner].on() (31ms)
+202ms ¦¦Executed virtual command [Shop Air Conditioner].toggle (40ms)
+208ms ¦¦Cancelling statement #5’s schedules…
+733ms ¦¦Executed physical command [Pushover].deviceNotification([Outlet toggled]) (512ms)
+736ms ¦¦Executed [Pushover].deviceNotification (517ms)
+743ms ¦+Execution stage complete. (619ms)
+777ms +Event processed successfully (777ms)

All Normal

LOG FROM FIRST BUTTON PUSH WHILE NOT CONNECTED TO CLOUD (Turned the Outlet off)
4/26/2019, 11:53:15 AM +232ms
+11ms +Received event [Samsung Button].pushed = 1 with a delay of 80ms, canQueue: true, calledMyself: false
+81ms ¦RunTime LockT > 50ms > rtDataT > 8ms > pistonT > 39ms > CE
+99ms ¦Runtime (7465 bytes) successfully initialized in 47ms (v0.3.10a.20190223)
+119ms ¦+Execution stage started
+133ms ¦¦Comparison (integer) 1 is_equal_to (integer) 1 = true (3ms)
+136ms ¦¦Condition #2 evaluated true (10ms)
+139ms ¦¦Condition group #1 evaluated true (state did not change) (13ms)
+144ms ¦¦Cancelling statement #3’s schedules…
+179ms ¦¦Executed physical command [Shop Air Conditioner].off() (22ms)
+182ms ¦¦Executed virtual command [Shop Air Conditioner].toggle (31ms)
+189ms ¦¦Cancelling statement #5’s schedules…
+40323ms ¦¦Executed physical command [Pushover].deviceNotification([Outlet toggled]) (40121ms)
+40327ms ¦¦Executed [Pushover].deviceNotification (40126ms)
+40331ms ¦¦ExecuteTask: Execution time exceeded 39329ms, Waiting for 500ms; lastPause: null
+40852ms ¦+Execution stage complete. (40730ms)
+40896ms +Event processed successfully (40895ms)

LOG FROM SECOND BUTTON PUSH WHILE NOT CONNECTED TO CLOUD (Tried to turn the outlet off again??)
4/26/2019, 11:53:56 AM +247ms
+1ms +Received event [Samsung Button].pushed = 1 with a delay of 31044ms, canQueue: false, calledMyself: true
+24ms ¦RunTime LockT > 15ms > rtDataT > 1ms > pistonT > 9ms > CE
+31ms ¦Runtime (7461 bytes) successfully initialized in 10ms (v0.3.10a.20190223)
+49ms ¦+Execution stage started
+63ms ¦¦Comparison (integer) 1 is_equal_to (integer) 1 = true (3ms)
+67ms ¦¦Condition #2 evaluated true (11ms)
+70ms ¦¦Condition group #1 evaluated true (state did not change) (15ms)
+75ms ¦¦Cancelling statement #3’s schedules…
+99ms ¦¦Executed physical command [Shop Air Conditioner].off() (17ms)
+102ms ¦¦Executed virtual command [Shop Air Conditioner].toggle (21ms)
+108ms ¦¦Cancelling statement #5’s schedules…
+168ms ¦¦Executed physical command [Pushover].deviceNotification([Outlet toggled]) (49ms)
+171ms ¦¦Executed [Pushover].deviceNotification (53ms)
+178ms ¦+Execution stage complete. (129ms)
+207ms +Event processed successfully (207ms)

an39511 · 2019-04-26 16:53:42 UTC

Just did the same test using SMS and the long delay/hang was not seen. It is probably as you said that the issue is with the pushover device as it is implemented on HE. Guess I will load up rule machine to see if I can get it to hang there. That way I can present it for support.

an39511 · 2019-04-26 17:58:51 UTC

What does work is making a web request to an external website such as google.com and capturing $httpStatusCode if it comes back 200 I’m golden.

an39511 · 2019-05-17 22:17:09 UTC

Now that I am on Hubitat using webCoRE for all my automation, I have a question.

I know webCoRE runs locally but what I am trying to figure out is when the Hubitat hub IS online does webCore mirror, or send into the cloud any of its actions? In other words, what kind of data volume is shared in the cloud while the hub is online?

The reason I ask is when I was on SmartThings (all cloud based) the data transfer volume between the hub and the internet is about 40 kilobytes per hour. Obviously if many devices are active the data traffic goes up a little, but overall it is very light on data IO.

The Hubitat hub is exchanging far more data while connected to the internet than the SmartThings hub. Does anyone know why that is? Is it webCoRE? Or is it the Hubitat hub itself creating all the data chatter? Looking at the clients in the image posted Amazon is a lot of the data. My guess is Hubitat uses Amazon servers for its cloud connection. Does webCoRE also use that platform? I know that the NTP has nothing to do with webCoRE but does anyone know why such a service would be uploading 16MB of data to NTP? Seems crazy to me.

Anyway to the point of all this. I was running my SmartThings hub on cellular because it was easy on the data. In fact it used on average 250MB per month. Now on Hubitat that data has jumped a lot so I am trying to narrow down why that is.

E_Sch · 2019-05-18 00:21:20 UTC

I think the question you are asking is how much data is the webcore groovy code sharing with the webcore server?

In ST:

webcore groovy code runs in cloud, and webcore server (dashboard) runs in another cloud
- so the communication is cloud to cloud
- what you see locally in this case is if you have a browser open to dashboard.webcore.co; the webcore server -> open browser traffic

In HE

webcore groovy code runs locally
- webcore dashboard/server runs in cloud
- so all communication from webcore groovy code to dashboard to out the internet
- all browser traffic if you have a browser open to dashboard.webcore.co is on your internet

So as you can see there is more data flowing.

Next question - what is it?

if you don’t have a dashboard open locally, it should be less
when you have the dashboard open, regular refreshes of either piston data or piston / tiles listing (if top level is open) is happening.
- the amount of data can vary, depends on
  - what you have open in the dashboard (overall listing of all pistons or an individual piston open)
  - what logging level you have on
  - how often pistons are running/change data / producing traces

I think the refresh is every 10 or so seconds when you have a browser open on a piston or the overall list of pistons.

So you may want to measure with you browser not connected to the webcore dashboard.

Others can fill in items I may have missed.

an39511 · 2019-05-18 01:00:00 UTC

Yes, that makes sense why there is more data going in and out. So if I am not accessing the webCoRE dashboard and pistons are humming along doing their thing, the webCoRE groove code is not sending anything to the cloud? And that’s assuming that pistons are not making use of any cloud based service and only controlling local devices? If so, that is what I would expect. Because webCore is not supported on HE, I am trying to narrow down why the data IO seems higher than what I’d expect. I make use of the HE dashboard but calling it through the local IP and not the hubitat cloud.

What I suspect is webCoRE is not the problem but rather whenever a device checks in, such as a contact open/closes that information is somehow being uploaded to the HE cloud if its connected to the internet. I know HE has been having issues with ISP’s blocking NTP and that may be the reason for the 16MB of upload which is crazy high.