RESOLVED: Issues with AppSheet Service: June 17 2019

UPDATE: the service is restored and everyone should be seeing normal behavior. (caveat: if you use the REST API, please see below). Details and post-mortem later today.

The service was disrupted for a couple of hours (approx 8AM PST to 10AM PST) except for customers in Europe who had uninterrupted service. The general symptom was very slow load times for the website and timeouts for app syncs. The root cause was server overload caused by an inadvertent runaway customer process that swamped our REST API and effectively denied service to all other traffic.

At the moment, we have disabled the REST API for all customers while we put some more effective throttling in place. We hope to have this fully enabled later today.

Hi community, we are experiencing issues with the AppSheet service impacting applications and our website. The engineering team is investigating, we will keep you posted with updates and resolution.

6 Likes

Some of the servers are back up again. But it may be a while before it catches up with the backlog.

4 Likes

My apps appear to be back to normal.

1 Like

most of the servers should be normal again. more details forthcoming.

We have temporarily disabled the REST API because the source of the problem was a runaway customer process that was hitting the API with so many requests that it led to a complete denial of service.

Other than the REST API, the rest of the service should now be restored.

More details and post-mortem coming later today.

1 Like

Thank you for resolving that so quickly. hAPPy users all around.

is it broke again?

Edit: Maybe not?

There was definitely an issue here in the UK. It’s possible that there is still something not quite right as we’re seeing record changes not being updated even when we get our riders to carry out another sync.

One of my app users is unable to see his apps. He has deleted and re-installed appsheet twice but still has no apps available. Any ideas?

@SteveA this issue is unrelated to our outage this morning. Most likely your user is signing in with a different email address. If you are unable to resolve, please contact support@appsheet.com with details.

@Midland_Freewheelers if you’re seeing record changes not being updated, it is most likely because you have duplicate keys. It is also unrelated to our outage this morning. Please provide details to support@appsheet.com if unable to resolve.

Will this affect migration to mySQL DB?

I am trying to migrate an existing app to mySQL DB. It is giving an error, AppTemplate not present.
Is this related to this?

Hi Yogesh, could yoy please send details to support@appsheet.com Thanks

Hi everyone, we’ve posted a post-mortem of the June 17 incident in the community. Please follow the link below to learn more about the issue and how we plan to mitigate and improve the service in the future.

2 Likes

Done.

Hello,

I can’t log into my AppSheet account, I use google to sign in and when I attempt to log in I receive an error message telling me that I need to contact my administrator of my google account… I am the am the administrator.

Please Help

Screen Shot 2019-06-20 at 4.34.19 PM

Please contact support@appsheet.com directly for help with this.

:+1:t4:

Hi @Santiago,

I’m facing a strange error with REST API.

Error description is
Error:
“Webhook HTTP post request failed with exception {“Message”:“REST API invoke request failed: Error: Value ‘22-06-2019’ in field ‘Date’ in table ‘Attendance’ cannot be converted to type ‘Date’.”} The remote server returned an error: (400) Bad Request.”

In my app all dates are in ddmmyyyy format. But this API is trying to insert date value in mmddyyyy format hence the error is appearing without completing the REST API.

It was working fine last week.
Please check the same.