If you are using Microsoft Office 365 services as a backend data source then you might experience 503s due to an issue on Microsoft O365 side. This issue is with API access point to the service and the regular access to O365 through its UI seems to be fine.
Update July 30th 10:00 am PST: We are following up with Microsoft support team, we will post back once we have any updates to share.
Update July 30th 12:00 noon PST: We are following up with Microsoft team on why some calls to
microsoft.graph.usedRange that returned intermittent errors with response headers:
Update August 2nd 9:00 am PST: Response from Microsoft: Our partner team enabled a change on July 28 which caused usedRange API to fail under some condition. We roll back this change at 4AM Aug 2 (US/Pacific time).
Status Code: ServiceUnavailable
Retry-After: 5
request-id: XXXXXXXX
client-request-id: XXXXXXXXX
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"West US","Slice":"E","Ring":"4","ScaleUnit":"001","RoleInstance":"XXXXXXXXX"}}
Date: Fri, 30 Jul 2021 17:50:07 GMT
The full error and call stack from the graph library was:
Microsoft.Graph.ServiceException: Code: InternalError
Message: Sorry, something went wrong.
Inner error
Code: transientFailure
Message: The request failed due to a transient error. Please try your request again.
Inner error
Code: InternalError
Message: Sorry, something went wrong.
at Microsoft.Graph.HttpProvider.<SendAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Graph.BaseRequest.<SendRequestAsync>d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Graph.BaseRequest.<SendAsync>d__32`1.MoveNext()
Below links have more data:
https://portal.office.com/Adminportal/Home#/servicehealth/history/:/alerts/MO273940
when wil lthis be fixed as my app have stopped working and it is affecting my teams in the field currently
Issue doesnโt seem to be resolved, still getting errors even though MS have stated Issued Resolved. 8am UK time
All my Apps are now down with same error message. No one can do any work !!
My teams are experiencing the exact same error and no one can give a timeline on when it wil lbe operational again
I can get into some apps now, seems to be very random.
@Harsh_Ch, its almost 11am here in UK and users are still having a lot of issues. Has anyone been in touch with Microsoft to report this as it seems like another issue if they are saying previous issue is resolved?
I have had no resolution. Out of desperation i duplicated my filed into GDrive and pointed the system to the new location and i managed to get back into the system. Gdrive seems more stable.
Not possible for me to do this, way too much data, we have a lot of Apps. Where are you located?
Im based in China but my app works for my company in South Africa. I use 15 datasets but the 2 main ones i moved to GDrive to see if that helps a bit. If it does ill be moving the rest over piece by piece. My system runs rought 8000 lines of data through it monhtly.
Same problem since yesterday. My business is paralyzed. Hope that Appsheet ingeneers are really in touche with Microsoft ingeneers, because survival of my business TODAY depends of itโฆ
@Alexis1 @Martina we apologize for the inconvenience. Its the side-effect of the flexibility our platform provides i.e. to let you manage data in your store and power apps/automation from there using APIs.
Microsoft Service seems to have random access issues although the site claims its back to normal. We are trying to follow up with few known contacts. We will post back if we have any success with it.
Hi community:
Do you have some news about this problem?
Just posted the link with new updates.
How could it be explained that only some Microsoft accounts have this problem. And within the same Microsoft account one app works (EvaluacionEmpresas-980586) and another does not (TrazabilidadEmbarques-980586). In this moment, I have reconnected the data source and the error message persists:
Same here. I have some items in my one drive still and its is up and down and affects my users big time
Hello @Martina, we are not sure that the problem has been solved. For now we have migrated the problem apps to Google. At this time we notice that the original apps are working fine, but since the problem is intermittent it is better to wait for AppSheet to confirm that the problem has been solved to migrate the original apps back to users. We are waiting.
@Martina, @Alexis1 and @SmartD as you are seeing the issue is completely random. And we have posted the detailed exceptions we are seeing in accessing MS Graph API for some apps. We are still following up with microsoft engineering and unfortunately, we donโt have a positive news to share yet.
As few users suggested, migrating to Google Drive as a backup in one solution you can employ on your end.
We will update back this thread as soon as we have more information to share.
@Harsh_Ch Today in the morning I noticed that the applications were working. I was just waiting for confirmation from AppSheet to be sure the problem was fixed and was not an intermittence. Thank you very much for the update.
MIcrosoft assures us the issue is fixed now. We are also no longer seeing errors in our logs.
Per Microsoftโs messages to us โฆ
Hi Praveen/Brian, thanks for reporting this issue to us. Sorry for the impact the issue caused.
Thanks to the session info, we think we have found the root cause and made the fix. Could you please check on your side?
Quality is our first priority, you can send email to ecoxlgraph@microsoft.com for Excel Graph related issue in the future.
and
Hi Harsh, our partner team enabled a change on July 28 which caused usedRange API to fail under some condition. We roll back this change at 4AM Aug 2 (US/Pacific time). If you are still hit issue after this time, please let us know.
When issues involve other services outside Google, it is difficult for us to be predictive or transparent, and we also have only so much we can do, frustrating though it is. In this case, hereโs a timeline of what we did to pursue the issue:
Overall, I have found Microsoft a good and reliable partner in fixing issues with services when we escalate. We can only escalate occasionally however, and when we do, we need to have fully ensured that the problem is not on our side, and that there is no other way to resolve it. And once we escalate it, it is difficult to effectively convey the same sense of urgency that we feel when our own customer is blocked. We donโt know if they are working on it or not, and we can ask them about progress but we cannot badger them. All this adds friction and delay. In this case, the intervening weekend probably did not help with time to resolution.
@Martina and others, I know this doesnโt change the fact that you have been badly disrupted in your business, but at least you know that the most senior members of the AppSheet team were doing what we could to unblock you.
@praveen thank you for your detailed response and all your work.
Its the end of the working day here. I will get users to test out in the morning and let you know.
Its not feasible for us to transfer data over to Google when there are issues. We have too much data, Apps and users to keep it all in sync then to move it back.
Indeed. Moving data between data sources is a very significant undertaking. You cannot do it easily โ has to be an intentional thoughtful thing.