Major Issue - locked out of App - "product engineers investigating issue"

[DeeEllHerzliya]

We have a Co-Authored App in development. Considerable time and effort to get this far. For several days we are locked out - totally, just a message after several minutes. "This page is having a problem" 

To be clear, we cannot reach the Editor, nor restore from Version History. We have no access to any data. Support writes "I'm currently working with our product engineers to investigate this issue. "

(1) For this major disaster the Support Process is horrendously slow. (2) The Chat App doesn't ask for the exact name of the App - nor for the Co-Author's account. So for 2 days they looked in error at a similarly named earlier version.

[DeeEllHerzliya · Accepted solution]

You really are devoted to AppSheet - thanks 

[Steve]

Escalated.

[DeeEllHerzliya · Accepted solution]

You really are devoted to AppSheet - thanks 

[Steve · Accepted solution]

Don't thank me yet. Even though I have an inside channel, I don't always get a response, too. 🙄

[DeeEllHerzliya]

Aleksi - the 5th technical guy wrote "The logs show "Could not fetch specific version requested", most likely this is coming from a virtual column. The app owner made 2 edits on 5/26, so rolling back to version 1.000052 or 1.000051 might help. As of now the stable version is crashing our servers potentially due to an expression. Is it fine for us to restore your 51 or 52 version?"

If it was crashing AppSheet's servers clearly somebody didn't do their job. 🤔

[DeeEllHerzliya]

Are we actually being used as a Beta test site? After just a few hours I have again lost connectivity with AppSheet's server. Just a blank screen and then the same error "This page is having a problem - Error code: STATUS_BREAKPOINT"

On my Android phone the same "parsererror" which if I clear with Cancel opens the App. "Syncing" just returns "parsererror".

[Steve]

Escalated.

[DeeEllHerzliya]

To save you reading all of this, AppSheet can totally lose your App. Suggest you regularly copy to have a series of working Apps.

Support ". . .  it's not showing up now when we search for it, and why sync is failing with a "not found" error. I inspected the database entry as plain text and it looks like the problem is caused by a particular column called "Recent Inspection" which appears to contain an AppFormula that references the column itself - "TOP(LIST([Recent Inspection]),1)". Due to the self-reference and the involvement of lists, the result type is inferred to be an endless list-of-lists-of-lists-etc which produces a structure that bloats the template. "
"I think it may again be necessary to roll back the app version again to break out of this, but I'm not having any luck loading earlier versions myself, including 1.000056 which was supposedly in a good state. It might require manually updating the database entry. Trying to find someone now who can do it."

Started from scratch with a new App and Copies after every update

[DeeEllHerzliya]

June 6

Adam - Support just wrote:

We were able to restore the app by removing the broken virtual column. We'll be looking into why the cycle detection checks missed this, but in the meantime I would recommend avoiding this kind of self-referential app formula.
===============
**CONCLUSION** TAKE A REGULAR COPY - Version History is great if the App exists. This took 10 days to resolve.

[Adam-google]

In general it's good advice. The problem in this case was particularly insidious, and would affect any copies as well. Basically any time the app was loaded in the editor or a new version was saved, the size of the metadata associated with this virtual column would approximately double. So even while everything seemed fine with the app, the template size was gradually increasing until it eventually exceeded some fundamental limit of the system (this is also why rolling back the version only briefly helped). It's not permanently lost even at that point, but required some manual recovery since most of the tools like version history depend on being able to read the latest version.

We've identified a few strategies to detect and prevent this situation going forward and are beginning to implement those now. In the meantime, as quoted above, I would advise caution with self-reference in app formulas, especially if lists are involved.