AppSheet Sync Performance Improvements

We are pleased to announce significant improvements to AppSheet sync performance. We’ve reduced sync times by 20% to 30% or even more in some cases.

Over the last few years, in addition to improvements like the recently announced Quick Sync, we’ve been making investments in improving and modernizing our backend service. This has resulted in a new backend that is faster and more efficient. We’ve been shifting parts of AppSheet to the new backend gradually to ensure we do not disrupt ongoing usage across the product. Recently, we shifted traffic for sync, and we’ve seen exciting performance improvements we’d like to share with the community.

During peak traffic times, we’ve seen a reduction of more than 30% in sync times at the 95th percentile. The top 5% of the slowest app syncs took less than 70% as long to complete. Here’s a graph comparing sync times at the 95th percentile over 2 days.

95p sync cropped.png

Notice that the differences between the old and new are greatest at the times when peak load caused the old backend to slow down whereas the new backend handled the traffic gracefully. Even for syncs not in the 95th percentile, we are seeing an over 20% improvement in sync times.

What do I need to do to enable this feature?

Nothing, this change is rolled out to all users already. If you haven’t already, you should consider enabling Quick Sync for your existing apps. More information on improving the sync performance of your apps can be found here.

We hope you are all as excited as we are as we reap the fruits of our investments in the AppSheet service. We will continue updating you in the future about additional improvements in the user experience and performance of AppSheet.

Thank you!

The AppSheet infrastructure team

36 23 3,342
23 REPLIES 23

This is fantastic news! Well done, infra team 👏

I truly believe improvements in that area (quick sync) will extend the longevity and breadth of effectiveness of AppSheet as a whole. 

Thank you to the development team. Perhaps this is what all users have been eagerly anticipating - an improvement in synchronization performance.

However, some applications may not be able to enable quick sync due to the use of advanced functions in security filters.

Clarification: all apps benefit from the current sync improvements, not just those using Quick Sync.

Were there any changes made to AppSheet functionality (e.g. how AppSheet interacts with data, calculates/recalculates virtual columns, how data actions or automations run etc.), or was this achieved strictly from making the backend infrastructure more robust? It does feel like syncing is occurring faster.

There were no functional changes. It was strictly making the backend more robust and efficient.

I don't know if these changes affect to our app or not, however we increasingly encountered time out errors recently, nearly happened to all users, so we must wait for a long time for synchronization to complete

Could someone give me soluiton for this solution, thank you!

p/s: our app use Google Sheets as main source

khiemtg_1-1695708286273.png

 

WhatsApp Image 2023-10-02 at 16.51.58.jpegWhatsApp Image 2023-10-02 at 16.52.08.jpegMe too is a couple of day i have the same problem. Is so so strange .
must of my problem is when i use a BOT. do you use too a bot? The strange things is if you look this same event have so different result. One take 1517 second and the same later only 4second. 

yes, I also use bot. I still don't know solution for this issue, very frustrating

My app is syncing about 1 - 2 minutes.

So it is very bad for me, i can not use Scale Partitioned tables, because it is maybe the only way to speed it up.
I have Public app, without user sign in (it's too hard for my senior users create and use google account and login), and in some month about 3000 users can use the application...

But i am a non-profit organization, so we cannot afford to pay such huge sums of money as your price list says... 😞

Please consider using Scale Partitioning for all users, that would be the only real help.

So far only Enterprice plan users can use it, but the price is not written, and the sales department is not responding, maybe it is too expensive and only for big enterprises :-(((

WOW, it is big jump. So much thank the team for this @hugheshilton .

I have a question regard to the perfomance matter.

Let's assume that I have two App those are A and B.

  1. A and B have the same functions, same UI/views.
  2. A use Google Sheets as Database
  3. B use Appsheet Database. 

What are the difference between A and B in term of perfomance?

It would be greate If we can see a chart or to know the analysis as what you've done with your update. 

Thanks so much for your time. 

AppSheet Databases are significantly faster than Google Sheets. How much faster they will be for your particular case is going to depend on your app and your data. Generally speaking, the more data you have, the greater the performance difference should be between Sheets and Databases.

This update wouldn't really affect the difference in performance between Sheets and Databases.

When you say


@hugheshilton wrote:

AppSheet Databases are significantly faster than Google Sheets. How much faster they will be for your particular case is going to depend on your app and your data. Generally speaking, the more data you have, the greater the performance difference should be between Sheets and Databases.

This update wouldn't really affect the difference in performance between Sheets and Databases.


Do you mean cloud or on-premise databases would not see an increase in performance? This update really is only for appsheet databases? 


@Luke_H wrote:

Do you mean cloud or on-premise databases would not see an increase in performance? This update really is only for appsheet databases? 


Still waiting for someone from appsheet team to answer this question. 

 

Well synctime is really one of the most importaint issues for our users. Unfortunatly we have since this release experienced the oposite. And not due to our app changes. Could that be related? @hugheshilton 

 


@hugheshilton wrote:

This update wouldn't really affect the difference in performance between Sheets and Databases.


I've noticed that when I use the same app with identical data, and I copy it to a database (Supabase) using the core plan, it loads much faster than when I use Google Sheets. However, the speed difference between the database and Google Sheets is quite significant. Is there a specific reason for this? Does the current upgrade only improve speed when a database is used and not with Google Sheets? I've observed a notable increase in speed when using a degree of parallelism set to '3' with the AppSheet core plan. I'm hopeful that with a degree of parallelism set to '10', as in the enterprise plan, the sync speed can be reduced to nearly 1 second from the previous 3 seconds. Can you please confirm whether this update actually affects the performance of Google Sheets as a database?

Here is the comparison: 

Screenshot 2023-11-07 at 3.27.43 PM.png

Screenshot 2023-11-07 at 3.33.13 PM.png

 

On average, this is at least four times faster than Google Sheets. 🤔 @hugheshilton 

Hi @Rifad 

AppSheet has always performed better with a SQL type database as the datasource. This is more about the query performance of a SQL DB vs. the Sheets API. SQL will always be faster than a SaaS API on similar data. 

Additionally, AppSheet can convert well-constructed Security Filters to limited SQL queries which makes sync times on tables larger than 10K rows much more performant.

In my experience, AppSheet apps will never consistently sync below 2 sec. My goal for sync time with any app is to get it in the 3-5 sec range. Trying to get anything lower than that gives diminishing returns on development efforts.  

If you have a database attached to your app, turning on Delta Sync and Quick Sync are going to give you the best Syncing Experience. 

Hello @Stefan_QREW 

I've created a significantly large application, putting in my utmost effort to minimize the use of virtual columns. Instead, I've employed a combination of slice and dashboard views to effectively depict related rows without excessive computational overhead. The app typically loads within 10 seconds, which is quite satisfactory considering the number of tables and views in use. Here are the statistics for my tables in Google Sheets. (Still under development)

Screenshot 2023-11-08 at 1.19.35 PM.png

 

Regarding the improvements mentioned in the post, it appears that the 30% reduction in sync time has a significant impact and is well-optimized for the database and not google sheets.

 


@Stefan_QREW wrote:

In my experience, AppSheet apps will never consistently sync below 2 sec. My goal for sync time with any app is to get it in the 3-5 sec range. Trying to get anything lower than that gives diminishing returns on development efforts.  


As you're aware, the Degree of Parallelism for the Enterprise plan is set at '10,' which is approximately three times that of the Core plan. If the database can currently load in less than 3 seconds, would this mean that with the Enterprise plan, it would be around 1 second to load the app? I'm curious about this, as it could be a compelling point to present to my client in favor of upgrading to the Enterprise plan.

 


@Stefan_QREW wrote:

If you have a database attached to your app, turning on Delta Sync and Quick Sync are going to give you the best Syncing Experience. 


The performance is outstanding when compared to Google Sheets, which leads me to consider whether, over the course of a couple of months as the data in the sheet increases, this could potentially impact the synchronization performance. 

Another notable improvement I observed is the speed at which it takes to edit a row, which is just 0.25 seconds, whereas Google Sheets takes approximately 1 second for the same task. This efficiency becomes particularly evident when carrying out looping actions and automations, as the database consistently demonstrates a performance that is 3 times faster than Google Sheets.


@Rifad wrote:

As you're aware, the Degree of Parallelism for the Enterprise plan is set at '10,' which is approximately three times that of the Core plan. If the database can currently load in less than 3 seconds, would this mean that with the Enterprise plan, it would be around 1 second to load the app?


Hi @Rifad this depends on you largest table. If your largest table already needs 3 sec to load, it will also take 3 sec to load with a degree of parallelism = 10. A higher degree of parallelism only helps if you have more than 3 large tables.


@Rifad wrote:

Here are the statistics for my tables in Google Sheets.


@Rifad Thanks for sharing those details. Your app seems to have a VERY large number of tables, 115. Are those all connected to one app? I assume you're using Data Partitioning?

My suggestion is that no single AppSheet app should have more than 30 data tables. There are various strategies I use to achieve this number, but consolidating pick-list tables into one larger master list table is one of the easy wins. You don't have to do this with traditional web apps, but with AppSheet, the system can only do so many table queries in parallel (3 -10). So more than 3 times to fetch data begins to really slow down performance. 


@Rifad wrote:

putting in my utmost effort to minimize the use of virtual columns


Minimizing Virtual Columns that perform Cross-table calculations is a good strategy to improve performance. If you have SQL as a datasource, you can leverage SQL Views to achieve this. This moves the calculation to the database instead of AppSheet's server.

If SQL is not an option, you can also uses Static Columns to perform the Cross-table calculations in their App Formula. Then the calculation only runs when that row is updated, instead of for every row in the table during sync

 



 

@fanfan @Stefan_QREW Everything makes sense now, and I appreciate your clear explanation. Thank you!

Here's another issue related to the update action, involving a simple action in two different scenarios. Due to this, the synchronization process takes a significant amount of time to complete. While it's understandable that syncing can take time, does the writing process also impact the overall size of the app?

Screenshot 2024-01-16 at 11.55.50 AM.pngScreenshot 2024-01-16 at 11.58.46 AM.png

I think it depends on whats type of edit you do. When you add an image or a file, the edit time will take longer, because it has to upload the image / file.

Its just a simple column update action

@pravse @Steve If you are both still available, could you please provide guidance on this matter?