Site back up: AppSheet service partial outage (10/22/2019)

[pravse]

Many customers are seeing slowdowns or 502 errors for the last 10 minutes. We are investigating and trying to restore the service to normal behavior. We will update status on this post when we know more.

[pravse]

The site should be coming up again. We will post details and resolution here once we know more.

[Stan]

Any update? Still down in the Netherlands

[Admin_IRL]

Still Down in UK too

[11134]

Is that problem for a long perion of time? Or we could have hope to solve it within an hour?

[pravse]

It is still down in the EU, but we expect it to be back up in 10 minutes. Back everywhere else.

[Chris_Jeal]

Brilliant!
Thanks for the update.

[Bertrand_Jame]

Still down in Ivory Coast, is it supposed to be back up in Africa yet?

[CMsDeveloper]

Is my inability to login with Google related to this?
“Sign in with Google temporarily disabled for this app”

[pravse]

@CMsDeveloper, no, please see this post for resoution of your signin issue

If you see a scary warning from Google during the signin process Announcements

Google is in the process of verifying all of its third party apps to ensure valid data access. This is a good thing. However, in the process, they have screwed up AppSheet signin for a few users. If you are one of the unlucky few, you will see a scary screen when you try to sign into AppSheet, telling you there is “danger”, AppSheet is “unverified” and not to proceed. The root cause is that permissions you may have given us in the past are no longer valid, and when we try to sign you in, Google’…

[Daryl_Zernick]

Same issue here. All of our field employees use AppSheet and Google to sign in.

[pravse]

All of EU should be back up.

@Bertrand_Jame could you confirm you are still down? We don’t think you should be.

[Bertrand_Jame]

Sorry for late reply, had to get on a call. It’s back up in Ivory Coast, thanks guys !

[Chris_Jeal]

Seems to be back online in the UK.

Good work team.

[11134]

Appsheet.com is still down in Ukraine. Are there any news?

502 Bad Gateway

Microsoft-Azure-Application-Gateway/v2

[pravse]

Maybe a cached response? Do you mind trying again?
Could you also explicitly try https://eu.appsheet.com to see if that works? Thanks

[GreenFlux]

Most of my apps are back up, but one of them running in another account using Office365 has just started getting this error:

@praveen, is this related? No changes have been made in the editor recently in this app. It just started getting this error today.

[pravse]

Should be unrelated. Do you mind filing a support ticket for this one? Thanks

[1minManager]

Based in UK, 3 accounts (all google auth).

1 working fine.

1 fails with
# 502 Bad Gateway
Microsoft-Azure-Application-Gateway/v2

Other says this when logging in to account (before I even see “My Apps”)
Sign in with Google temporarily disabled for this app
This app has not been verified yet by Google in order to use Google Sign In.

Client reported the app wasnt working at 1600GMT but was fine by 1645GMT. But now (1800GMT) I can’t get into it.

[TDhers]

Hi Simon, Daryl, this is an unrelated issues we have seen for the last 48 hours due to some changes in Google auth. The step to resolve are described here:

If you see a scary warning from Google during the signin process Announcements

Google is in the process of verifying all of its third party apps to ensure valid data access. This is a good thing. However, in the process, they have screwed up AppSheet signin for a few users. If you are one of the unlucky few, you will see a scary screen when you try to sign into AppSheet, telling you there is “danger”, AppSheet is “unverified” and not to proceed. The root cause is that permissions you may have given us in the past are no longer valid, and when we try to sign you in, Google’…

[David_Richey]

Located in Indiana, US. Still down.

[Adam-google]

@GreenFlux I emailed you with some questions about the error you posted.

[Christian_Nere1]

Also down in DK

[CMsDeveloper]

Down in Omaha Nebraska

[Daryl_Zernick]

Down also in Indiana

[Grant_Stead]

Down again

[Chris_Jeal]

Down in UK again.

[Sven_Kutscher]

down in belgium

[Sakis_Koukouvis]

Down in Greece

[Christian_Nere1]

Back up again in DK

[Chris_Jeal]

Up again in UK

[Maximiliano_Cor]

Also in Argentina. does not save changes in the design of the apps and delays in loading

[TDhers]

Closing the loop with everyone on this thread. The sites are now back up and should be operating normally around the world.
They were down again for about 15 minutes around 11:45AM PST. (first time happened at 7:49AM from about 20 minutes)
The entire team has been all hands on deck all morning to identify the root cause of the instability. It had to do with our caching layer getting overloaded and becoming not responsive as it was reaching its size limit.
We reverted to last Saturday’s build to allow us to do some root cause analysis and uncover the source of the cache overload while resuming normal level operation.
All servers are now back to stable level of operations and responsiveness.
We apologize for the disruption and inconvenience our service interruption caused to your business.
It is our responsibility to anticipate situations and insure the service runs reliably to serve your business needs. We clearly have missed the mark here and are working hard to understand the root cause and make sure we learn from it and prevent it to happen ever again.
On behalf of the Appsheet team, I present our sincere apologies to you and appreciation for the business you place in our hands and care.
Thierry

[Jethro]

Appreciate the transparency and service!!