Post Mortem: Office 365 Datasource connectivity outage: 08/20/2019

Tuesday, August 20 around 1AM we started to see a growing number of connectivity errors with applications using Excel sheets stored in Office 365. Since we had not made any changes and deployment, we immediately started to suspect something had changed with one of Office 365 APIs.
Now it appeared that not all sheets connections were broken. Investigations with a few of you revealed it might have had to do with sheet size and possible timeout.
We contacted Microsoft support immediately and escalated gradually through the day to the Microsoft leadership contacts we have. The next day Microsoft acknowledged the problem and issued a fix which allowed us to revert back to our normal codebase. Our own tests as well as all of you who had reported issues shows that we are now back to normal.
We sincerely apologize for the inconvenience this might have created with your users and thank all of you who helped us investigate and narrow down the root cause of the problem so quickly.

1 Like