“Backup Bin Updated” msg while trying to save Piston - Does not save to cloud

[Brent_NC]

Hi there. I’m working on a fairly large piston (15 chunks if I’m reading IDE correctly). When I try to save, the banner at the top of the page indicates, “Backup Bin Updated”, but doesn’t show the typically dynamic, “Saving chunk x of y”.

This behavior occurs every time I try to save. If I “Cancel” instead, then go back in to edit the piston, I get the message indicating there is a newer local version and I choose the local option in order to preserve my changes.

I have tried the recommendations here.

https://community.webcore.co/t/webcore-hung-while-saving-piston-cant-access-dashboard-anymore/2802/2?u=brent_nc

When I import the restored piston from the backup code, I’m able to get my latest changes, however when I try to save the restored version, I get the same message “Backup bin updated”, and I’m back in the same loop of despair. Interestingly, there is no “Cancel“ option from within the restored piston.

I’d appreciate any ideas on how to overcome this situation. Thanks!!!

[ipaterson]

Can you check ide.smartthings.com > Live Logging for any error messages corresponding to when you try to save this piston? Seems like a failure inside the smart app.

I’ve seen something like this with one of my pistons, it was very strange… For me it happened on the 24th chunk rather than what seems to be the first chunk for you. A certain sequence of characters in an expression seemed to be causing the chunk to fail. I tried changing the expression around but finally just had to avoid the particular word. Certainly no idea what was going on for me or if it is related to your issue.

[Brent_NC]

Thanks for the quick reply @ipaterson. Nothing of great interest in Live Logging.

I think the entire Piston might be corrupt at this point. I started backing out my changes one by one and now every time I try to delete the below variable setting statement (which is nested within a couple levels of switches) my entire WebCoRE application locks up and the screen goes to unrecoverable “blank” and I have to force close the (iOS) app.

I appreciate the attempt to help. I’ll start over and see how it goes.

[Brent_NC]

Posting a green snapshot of my seemingly doomed-to-fail Phase of Day Setting piston as @eibyer had inquired as to why I was exploring alternate designs, in a separate topic:

General Question: Is it possible to read Google Sheets data into WebCoRE?

1) Give a description of the problem I’m having some challenges getting a large “Phase of Day Setting” Piston to the finish line (something about my choice of design or character sets is causing some sort of corruption…I expect it has something to do with the large arrays I’m loading and then processing through nested switches). I’m completely re-thinking the design and was lamenting not being able to make database calls in place of the arrays. Even if I could just pull keyed records from a sp…

Basically, this is 2/3 of the Piston that gets stuck in the “Backup Bin Updated” unrecoverable loop. Once I add the next switch case and some “log to consul” statements, it starts to fall over and I can’t successfully save (which is why I can only share 2/3).

I accept that these large arrays within multiple layers of cases may just be too much to handle. Looking for more elegant options. I appreciate the interest and ideas.

[ipaterson]

For this complicated of a data structure you may get more mileage out of configuring all of your static data as JSON in a string variable and then using Parse JSON Data when the piston starts. That will allow you to expose hierarchical data on the $json system variable. Note that the expression parsing has special syntax for $json that will not work if you set variables to portions of the json content (e.g. you can do $json.NHT[3][0] but if you assign $json.NHT to a dynamic variable NHT none of that NHT[3][0] syntax will work). You’ve already found arrayItem but you might get away with using $json throughout the piston.

I still can’t say why it is not saving or find the “correct” switch cases and debug lines to add to get it to happen on my end, but that approach could possibly lead to a simplified structure.

[Brent_NC]

OK great. Thank you @ipaterson. Now I have two really excellent ideas for taming this beastly Piston. If I have understood correctly, the recommendation is to take all of my 70 time arrays and structure them as one big JSON file. Then I should theoretically be able to decipher / load specific records into the Piston logic as each scenario requires, using the WebCoRE native parsing functionality. That sounds pretty cool. Almost like a little runtime database load into memory. If that’s the case, then I’m finally starting to understand the power/potential of these JSON functions.

Does anyone have a particular affinity for a spreadsheet-to-JSON converter app?

[Brent_NC]

I’m sure this is drifting out of the “bugs” category of the original post. Just didn’t want to start yet another topic for my issues with this Piston, but happy to move if more appropriate.

Based on my understanding of @ipaterson ‘s recommendation I’ve tried my hand at generating “local” JSON data that can be parsed and then accessed as keyed values in the same Piston. Not having much luck.

Also, in reading some of the other posts related to JSON usage, I got the impression that it may not be possible to generate local, nested JSON datasets. For instance:

Web Request POST to internal server - JSON problems

I think you are out of luck. Simple JSON strings (several field:value pairs) can work just fine, but you’ll never get nested values (your params field). You have to use multiple variables, and include all in the data. You can’t get Webcore to send a literal variable string already formatted as JSON. milhouse

[ipaterson]

Your curly quotes are breaking the JSON data, be sure to use only straight quotes "

[Brent_NC]

Just a bit more info on the original topic (inability to save certain pistons). I started from scratch trying to create a Google Sheets pull per recommendation from Jonathan here:

General Question: Is it possible to read Google Sheets data into WebCoRE?

There is a way to do this using a JSON web request and setting the response as a variable which can be used in your pistons. I’ve been using this for a while to great effect! What you need to do is share your Google sheet by publishing it to the web (n the File menu). Then in WebCoRE make a web request to: https://spreadsheets.google.com/feeds/list/xxxxxxxxxxxxxxxxxxxxxxxx/default/public/values?alt=json But put the spreadsheet ID where the xxxx is. When you open the spreadsheet the link shoul…

While I got the spreadsheet integration working (very cool…Thanks Jonathan!), I ended up with another rather large Piston. Eventually, I got to the point that the piston would not save. WebCoRE issues the same message,”Backup Bin Updated” and never actually saves. Same behavior as originally described. I believe that the failure point is the same number of chunks at 18 and figure that may be important information. I’m starting to wonder if this is a limitation of the iPad app?

I did try closing all applications and ‘rebooting’ the iPad but to no avail. I’ll try @ipaterson ‘s approach above as a next design alternative. After that I’ll try breaking the Piston into several sub-Pistons. May eventually have to look at buying a personal laptop (which I’ve been resisting as have no real need for one otherwise), if this is truly an iPad app issue. THANKS!

[Brent_NC]

BTW… I’ve been able to consistently recreate the failure (presumably going from 18 chunks to 19 chunks) by adding the additional variable setting statement into the ‘Friday_true’ case per below. Actual text for copy/paste is:

concat({$response.feed.entry.content[58].$t},”,”,{$response.feed.entry.content[60].$t},”,”,{$response.feed.entry.content[62].$t},”,”,{$response.feed.entry.content[64].$t},”,”,{$response.feed.entry.content[66].$t},”,”,{$response.feed.entry.content[68].$t},”,”,{$response.feed.entry.content[70].$t})

[ipaterson]

There do seem to be issues specific to iPad here. I am still able to edit and save larger pistons from a desktop but the piston does not save on iPad. For me, the Save button does nothing at all but that may be since I have not saved this piston to a backup bin. I’ll take a closer look on Wednesday.

Your piston really makes me wish that $response array indexes allowed expressions…

offset = @HolidayTomorrow ? 14 : 0;
concat({$response.feed.entry.content[2+offset].$t},”,”,{$response.feed.entry.content[4+offset].$t},”,”,{$response.feed.entry.content[6+offset].$t},”,”,{$response.feed.entry.content[8+offset].$t},”,”,{$response.feed.entry.content[10+offset].$t},”,”,{$response.feed.entry.content[12+offset].$t},”,”,{$response.feed.entry.content[14+offset].$t})

would cut your switch in half but unfortunately that does not work at all.

[ipaterson]

I still haven’t fully traced what this piston will accomplish but I wonder if you could do the date logic with Google sheets formulas… I often have a data sheet like the one feeding this piston and then add a second sheet to summarize the data. Maybe something that you can just fetch today’s data is this piston is ultimately just concerned about the current day.

[Brent_NC]

Yes! Exactly! I built the first iteration of this Piston with the assumption that you could use an expression as the $response index. Such a downer when it didn’t work. That’s how I basically ended up writing the same bloated logic as per the original post, just pulling the data from a spreadsheet vs hard coded arrays. I believe this Piston would be about 1/6 the size with variable $response indexes. Regardless, I see all of this as a great way to learn more about WebCoRE and it’s been a fun way to reconnect with my long-past coding roots. Reporting potential bugs is definitely not a complaint. WebCoRE is an amazing platform with some incredible ‘Minions’ working to make it even better! Thanks again for the help and further ideas. Will definitely consider the spreadsheet simplification option. If I could get Google Sheets API v4 to work (to be picked up in another discussion), that might also help eliminate a lot of code.

[ipaterson]

This looked like it would not be an easy one to overcome:

Maximum call stack size exceeded.

However, that error was after the piston was compiled so the complicated part I assumed to need work was not at fault. Instead, there was a single line of code to prepare the piston chunks for saving that was written in just about the least efficient way possible.

No problem saving the piston in 20 chunks after fixing that up. This will be included in the next release (today or tomorrow).

[ipaterson]

The fix for this was just released in v0.3.108.20180906

webCoRE Update v0.3.108.20180906 - restore pistons from backup file, bug fixes

Please update your smart apps and reload the dashboard for this release which fixes a number of bugs and introduces the ability to restore pistons from a backup file. Restore pistons from a backup file The Backup Piston(s) feature on the dashboard is now accompanied by a way to restore from that backup file. When restoring from a backup you will be guided through the process with the following screen: [screen shot 2018-08-24 at 9 09 48 pm] … [screen shot 2018-08-24 at 9 10 22 pm] The defa…

[Brent_NC]

Very nice work! I was able to successfully recover 2 large pistons on the iPad (both of the “locally saved” versions saved with 19 chunks).

[Brent_NC]

Looks like 30 is the new “chunk limit”? …a significant improvement over 18 on the iPad. Only a slight inconvenience. I suppose if you hit 30 chunks it’s a good indication of inefficient code.

[ipaterson]

Yep I think I hit the same limit (or at least in the ballpark) long ago on desktop with this monstrosity but was able to get it down to 28 chunks.

In this case it’s actually a backend error in the smart app rather that in the browser. You should see an error in the ST logs, probably physicalgraph.exception.StateCharacterLimitExceededException: State cannot be greater than 100000.0 characters since the piston chunks are saved to the application state for later editing.

[ipaterson]

Well, trying to save that piston to see how many chunks it has failed at 26 of the 28… and now my instance is hosed not letting me delete the piston or load the dashboard. This has been quite a week.

[ipaterson]

Adding a new tool to the toolbox here… so when I saved the piston, those 26 chunks first get pushed to the webCoRE app which is the app that went over the state limit. Errors all throughout the dashboard but it was fixed by a clean and rebuild. Something to consider for future improvement to guard against that…