webCORE Dashboard Completely Broken. Please Help [UndeclaredThrowableException]

#26

Here’s a few more facts that may or may not be relevant.

I’ve been using WebCoRE presence sensors since you were required to install them in a separate instance. I have never migrated my sensors to my primary instance so I do have two instances of WebCoRE running. If I try to login to the dashboard for the instance that only has the presence sensors and no pistons in it, I also get the same error behavior, the log show a slightly different error (but it appears to be related to me).

10:27:26 PM: error java.lang.NullPointerException: Cannot invoke method leftShift() on null object @line 1712 (listAvailableDevices)

And there are two instances of the “WebCoRE Presence” in Live Logging. One simply shows a debug log for “Loaded 5 devices” (those are my presence sensors). The other shows “loaded 5 devices” and the error stated above.

For my main WebCoRE instance, there is only one WebCoRE showing in live logging and it shows only this:

92f5df3f-160e-4f68-a76d-663c2ef7b751 10:26:26 PM: debug getChildDevices(false), children=0

92f5df3f-160e-4f68-a76d-663c2ef7b751 10:23:46 PM: error java.lang.reflect.UndeclaredThrowableException @line 1699 (listAvailableDevices)

#27

Left shift error seems to be caused by the thing I fixed in the original storage debug code. Please update to the latest code at my GitHub link for that app and check the All tab in live logging for errors.

#28

No change in behavior after updating the Storage groovy code again.

8:23:39 AM: error java.lang.reflect.UndeclaredThrowableException @line 1699 (listAvailableDevices)

And you can see from the log header, there’s only one log entry for WebCoRE not 2 as you suggested it should be above.

2018-09-15%2008_26_51-Logs
2018-09-15 08_26_51-Logs.png2946×445 65.5 KB

#29

Please bear with me, there is another update to copy and paste over the webCoRE Storage app. Save and publish, then follow a new step just to make sure the device data has to be reloaded.

In the ST app go to Automation > webCoRE > Settings > Available devices > Available devices (again) then tap Done in the upper right. No changes to authorized devices required, this is just a way to ensure that webCoRE refreshes some data for the devices and dashboard.

Then go to the All tab in Live Logging and load up dashboard.webcore.co

#30

Did all this, exact same result. I think I’m doing everything correctly, but is there some way to make sure it’s executing your new code and not some cached version of the old code? I’m grasping at straws here, but maybe some debug comments in the debug Storage piston so we can verify that your new code is actually running?

#31

Just finished up a screen share session with @wjarrettc, I am tremendously appreciative for your time.

Here’s what we learned:

  1. After providing a password for the dashboard, the web request to the webCoRE smart app to load dashboard data is timing out after 20 seconds rather than returning a normal error response
  2. The webCoRE app works normally up until the line where the exception is thrown, which is a call to the storage app to load devices
  3. No part of the storage app is actually executing, I added debug logging and the loadAvailableDevices method where the debugging code has been added is never even called. It seems that the UndeclaredThrowableException is actually stemming from ST attempting to communicate with the storage smart app.
  4. The storage smart app state loads fine in the IDE and its application state does not include any unexpected devices.
  5. All of his automations seem to be operating normally.

The following may be directly or tangentially related:

  1. Jarrett’s Alexa skill is unable to load any devices
  2. Recently a device appeared on his ST account that did not make sense; the device brand and model matched a sensor that he used in the past but the device name included the name of a person that he did not know. This was confirmed unrelated.

The next step for @wjarrettc is to contact SmartThings support. He is going to lead with the Alexa skill issues in the hopes that the problem loading devices there is the same problem affecting webCoRE.

This seems to be a fairly isolated incident. The best recommendation I have at this point for anyone else who is affected by this problem is to contact SmartThings and ask them to review any potential issues loading your devices or timeouts when running child apps like webCoRE Storage. There may be logs or tools on their end that will shed light on the issue.

Tomorrow I hope to find time to look at potential workarounds.

#32

Now when I try to enter “Available devices” page, it shows “something’s wrong we can’t load your screen right now Retry”

Checking if another error report is related to this. Does anyone see an error like the above when trying to add or change available devices in the webCoRE smart app? @wjarrettc @andyhawks @Yairsmail @jasonrwise77

#33

I do not get this error when adding or removing devices in the smarthings old app through the WebCoRE smartapp.

#34

(https://graph-na04-useast2.api.smartthings.com/ide/logs#feecfce2-86cb-440b-8779-5295af886d2f) 7:11:59 AM: error java.lang.reflect.UndeclaredThrowableException @line 1699 (listAvailableDevices)

I do not get any errors while trying to add or remove devices. This issue has been going on for at least a week now.

#35

@ipaterson just to tie off on this one. Upon further review, one of the custom device handlers I had in my IDE apparently had the name of that person in the DTH name. I expect what happened here is that ST found the old motion sensor on the z-wave network when I ran an “add device” routine and I didn’t notice it. If that happened, it would have added it using the devicetype handler name, so I think this is probably not related to this incident. Just clearing that up.

#36

Has anyone contacted SmartThings support yet as recommended in this reply? Jarrett was planning to call during the week but if anyone has had time it would be great to learn more about what is going on here.

#37

Phone support is not available on Sunday, they open at noon EDT / 9am PDT on Monday. I plan to call them Monday afternoon, work schedule allowing.

#38

Oops good to know, thanks

#39

Sorry, I was away this weekend, I do have some logs for you. This error I have seen before on another smartapp, and it was related to too many devices trying to be loaded. When I reduced the # of devices in that app the error went away. I am going to see if that possilby fixes it.
I am curious in my screen shot below it is showing smart app pause, what is actually being paused?

image
image.png1280×984 170 KB

#40

Support update: Called SmartThings support number on their website. Was connected to a tech fairly quickly after going through the automated menu routine. She quickly punted me over to SmartThings Classic support since I’m Hub v2 user on the classic app. Held there for about 20 minutes before putting in my phone number for a call back. Got a call back about 10 minutes later. Spent some time talking to the tech and then he escalated to his senior support tech. After about 10 more minutes on hold, they came back to me and said that they are seeing this issue which is the same as some other users had seen and it “was supposed to have been fixed already” and so they are escalating it to a higher level support team who will work on it now. He told me to expect to hear back later today hopefully on a fix. Fingers crossed.

#41

I tried this for my Alexa issue and it absolutely fixed my problem. Around 135 devices is the maximum that I can have Alexa discover before it times out. The default setting for the Alexa Skill/SmartApp is to make all your devices and routines available. That breaks for me, but if I go in specify specific devices, I can get Alexa to work. Since I’m not really using motion and contact sensors to trigger routines in Alexa, this is a fine workaround for this problem.

I have a 2nd instance of WebCoRE that was created for my Presence Sensors back when the WC sensor needed to be installed in a separate instance. I can break the WebCore Dashboard on that second instance somewhere between 125-165 devices available, depending on server loading. It clearly seems to be a function of number of devices.

#42

I can confirm that by reducing my device count by 15 devices I can now get into my dashboard and pistons. Funny, now I have no live logging in the IDE, can’t ever win.

#43

Sorry @ipaterson that I also was not at home or by a desktop this weekend. Just catching up on this thread now.

I just tried eliminating devices as @jasonrwise77 mentioned just above and that totally worked for me - dashboard even loads on the machine I have been unable to get it to load on for the last couple weeks!

I am only putting one exclamation point on that sentence for now because I’ve experienced that it could load one day but not the next. But fingers crossed that we have a solution. Thanks very much @ipaterson @wjarrettc @jasonrwise77 for the team debugging and patience!

For reference, what’s working for me as of now:

  • Down to 179 devices registered in webCORE from close to 300 when webCORE did not load
  • Down to 1 webCORE instance instead of 2
  • Memory usage of my instance decreased from 32% to 17%
  • About 70 active pistons
  • Every device registered in webCORE is referenced in a piston, no extraneous devices
  • Removed all Hue Scenes loaded from Hue B Smart
  • Removed motion sensors that were only being used in conjunction @bangali’s Rooms Manager app
  • Removed all of my momentary switches created from the SmartThings UI, moved these to routines (note I did not have any routines when webCORE did not load, just webCORE pistons)
  • Removed a few Smart Apps I wasn’t really using, here’s what I still have in place: ActionTiles, Alexa, Ask Alexa, Foscam Manager, GCal Search, Google, Hue B Smart, IFTTT, KuKu Harmony, Life360, Logitech Harmony, MyQ, NST Manager, Plex Manager, Ring, Rooms Manager, webCORE

So my ‘cleanup’ changed too many variables at once to isolate where the timeout may have happened, but hopefully this can help people who encounter this problem see an example of what didn’t work initially and what is working as of now (knock on wood).

IDE Live Logging is also wfm (works for me).

#44

@wjarrettc Just curious, are you having any issues viewing your live logging? Mine seems to hang and it started when my webcore issues started. My live logging does work, but it takes like 20 to 30 tries before it starts to flow.

Cannot login to https://dashboard.webcore.co/
#45

Hmmm … last I heard there was something like a 300-device limit for querying in the ST database … dunno if this was ever changed.