Re: Introducing new AppSheet database feature in p... - Page 2

ShirleyN · 10-03-2022 12:00 PM

Please see updated GA announcement --> here

Hi everyone!

Over the past year, we’ve been conducting user research with app creators, end users and some of you to better understand the app creation experience. Through this research we’ve uncovered challenges with the existing external data sources and noticed a gap in datastore options for citizen developers. With Sheets being the most common connected source for AppSheet apps, we noticed frustrations around:

Formatting a spreadsheet for app creation can be time consuming
Changes to the data schema after initial app creation are challenging and can easily break apps
Unstructured data leaves room for human error in data entry
Scalability and sync speed starts to deteriorate as Sheet size increases

With these challenges in mind, we set out to build a native database for citizen developers to easily and securely manage their data. We believe this will also improve the experience for app creators.

Today, I’m happy to announce the release of the new AppSheet database feature for public preview! During public preview, access to AppSheet databases is enabled by default for everyone but it will not affect existing apps unless you explicitly add a blank table or connect an AppSheet database inside the AppSheet editor. Use of this public preview feature will be free to everyone but limited to 10k rows per table and 20 tables per database. Note that these limits will change for our public launch.

To get started, you can create a blank database from the My Apps page. If you’d like to start building an app from scratch, there is also a new Blank app option. This blank app will create a new AppSheet database to use as the data source.

Within the database editor, you can set the same column types as in the AppSheet app editor for your data.

After that, you can create an app directly from the database. This will create an app on the current table only. If you have references to other tables, you will need to add them using the AppSheet app editor. This is something we’re working on improving.

Since this feature is in public preview, we’re still making improvements to it and appreciate your patience with issues you may face. Some features that are coming include a smoother import/export of data, database recovery, and email notifications after sharing a database.

We will also be updating our support documentation to include this new feature. For those who would like to disable access to this feature, see the Disable AppSheet databases policy.

Thank you and happy app building,

Shirley

Dec 8 Update: Sheets import is now enabled on the My Apps page! More details on import and other ways to create a database here.

Feb 1 Update: Column types set inside the editor were getting reset every users regenerated schema twice. This has been fixed.

Feb 14 Update: Virtual columns can now be set as labels inside the AppSheet app editor.

Apr 26 Update: Deleted tables can now be restored through the new history experience inside the database. Please see support article here.

May 3 Update: You can now share a database with an entire AppSheet team as a shared datasource. AppSheet teams are only available to AppSheet enterprise users.

May 24 Update: Audit logs for AppSheet databases are now available for admins and database editors. You can also restore deleted databases. More info here.

dugmanstar

Hi, when is export coming now? It looks like this was an old message but i cant see a feature to export the database from within appsheets.

neovinu

error while adding new row to Appsheet Database

ShirleyN

Hi @neovinu do you remember if this was for an unauthenticated app where "Require sign in " is switched off? There is a known issue here that we're investigating.

Ratatosk

Looks promising! 😀

Before I try out this feature, I have core plan, will I be able to use these features within that plan?

ShirleyN

@Ratatosk Yes the feature will be available for all plans but limits will differ. We're actively making improvements in this area and unfortunately can't disclose what the final limits will be just yet but considering the comments in this post we know this is top of mind for everyone.

Ratatosk

Also. Is there a way to import existing data?

andrew_caff

You could use the copy data to a new source under the tables storage section like @Fabian_Weller suggests.

In theory you could brute force this with an action. The one where it is add a row to another table using values from this row. And then just use the execute action behavior where your key column is true. It might take a while to sync all of that, but it would be a brute force solution

Fabian_Weller

Here it says:
At this time, you can't import data directly from another data source (such as Google Sheets) on the My Apps page. As a workaround, you can copy existing app data to an AppSheet database in the AppSheet app editor.

I used the Copy Data to New Source button to copy a table to AppSheet Database.

I got this Error:

Error while copying table to google_tables (Google Tables): {"method":"POST","url":"/api/template/copyAppTable","source":{"src":"jquery","jqXHR":{"readyState":4,"responseText":"{\"Message\":\"An error has occurred.\"}","responseJSON":{"Message":"An error has occurred."},"status":500,"statusText":"error"},"exception":{}},"status":500,"statusText":"error","responseText":"{\"Message\":\"An error has occurred.\"}","responseJSON":{"Message":"An error has occurred."},"exception":{},"name":"n"}

In fact, the Database was created with the correct headers, but it it empty.

pawann

Exactly the same behavior here (error message + empty database table created).

Copy&pasting 500 rows max from the spreadsheet to the automatically created database did the trick, but is cumbersome with many rows...

ShirleyN

Hey @pawann and @Fabian_Weller sorry for the late response, I've created a bug for this and will investigate.

hartable

Is there another work around for this? I get the same error but only a blank database with no table at all is created instead of the table with headers like the people above.

Koichi_Tsuji

To be perfectly honest, I can't find the reasons why this new features will solve the listed problem on this original post.

Yes, setting the spreadsheet is sometime troublesome. however, dealing with new Appsheet database will involve bunch of new things. One intention is to remove the process "Regenerate structure" in case we change the column scheme on the spreadsheet. However, on new AppSheet database, we need to do the case to let Appsheet editor to recognize the changes in schema.

Leaning new things is requiring new "cost" in terms of time we spent on. Once again, I do know know what is the real benefit to learn the new AppSheet database, as its required setting is more complicated than Google Sheet.

I will stay with Google Sheet to be honest, by avoiding using new AppSheet database, unless I persuade myself to find the benefit to use this new feature.

Therfore, I wont tough.

If this new feature gets popular, the comparision analysis should be presented. A) Using Google Sheet B) Using new AppSheet DataBase. It is always pros/cons when we compare something else. However, there is no clear indication what problems we have on Sheet with AppSheet will be solved by the new AppSheet database. The "limitation" is only highlighed here. Google Sheet have limiation 100k cell, while new data base has 10K records and 20 tables. Takinng into account that, I would go with google sheet, as limitation is lower, while it is said this project is going to increase the scalabiliy, which I dont think true.

In terms of performance issue, I do not understand why new AppSheet database will improve the performance issue. Yes, in case of cloud SQL, it will, as we can apply "Index" to the SQL, which is significantly improve the performance of the app, as App will not read the whole table on sync..... AppSheet database coul be based on non-SQL concept, alike Firebase or other non-SQL data bases. So not sure why new AppSheet database will bring the better performance, comparing Google sheet. However, the performace issue is usually taking in place when the table is large enough. If our table is actually large, then we will hit the limit of AppSheet database. No other choice to move to cloud SQL.

All in all, I could not find what the real benefit to go with new data source.

@takuya_miyai

SkrOYC

Same, more so from a non-Google user that do not rely on GSheets. Moving to AppSheet Databases doesn't sound like a wise move considering the closed-source nature of the platform and, consequently, our data.

We are with tied hands IMHO. Go SQL paying a lot or go AppSheet Database where you don't own your data but will get better performance and less cumbersome. That's the idea they are projecting.

Finally, we are not touching the ease we get from current datasources to manipulate our data/structure.

In the end, I just hope we will not be pushed to use this in the future. I think that this AppSheet Databases is just another feature to appeal new users instead of existing users that have experience with the platform already

Koichi_Tsuji

Very true.

As far as we see, the one of the scope of the project is to improve the scalability, however, Google is putting a limit (number of rows and tables), which could be less than what we can manage with Google Sheets. Then this objective never going to be fulfilled.

With Google sheet, we can set the format of the cells as they are, roughly speaking. Setttin data type for AppSheet database in addition to Appsheet editor. Thank you god, new repetitive works to app creator, which will reduce the efficiency, while we have less storages. Again as I mention, learning new database setting, it will surely increase the cost to the app creators to learn new things.

New appsheet database seems to be another new "app" which we need to learn.

Dealing with Appsheet need a cost to learn.　New AppSheet database need more learning and settings.　This will detrioate the efficiency, which most of appsheet creator expects.

When this dicusssion was started, I questioned if this integration will achieve the real-time integration, but we found out it will not. At that point of time I know it, I lost the interst, then decided to stay with sheet and cloud SQL.

I admit it is fancy to deal with Google Tables, as it will provide the modern and userfriendly UX alike Air table, but actually I dont care. we need a pure and simple data source.

For now, it looks like we need to deal with two differnent apps...

I m years-running pro-appsheet creators, so I can manage to deal with those new features, but bar for beginners, it is getting more difficult to deal with.

Koichi_Tsuji

Surprising stuffs is this new featue is said to be Preview release.

For the existing preview features,we need to explicitely turn on the preview features explicitly. However, this new feature is actually avaible for all the users by default.... It looks almost identical as GA features now. So we are trying to deploy our policy not to let the app creators touch this new features upon, as it will definitely cause the massive disruptions.

However, sadly the team policy to control to ban this is actually not really working. we are at a loss.

zito

FWIW, one of the reasons for building this feature is specifically for customers who would rather not simply host their data in Sheets, either because they are not heavy Workspace users, or because they want something more closely aligned with AppSheet directly. It also will (in the future) make it easier to apply consistent governance policies, by having one policy engine that will apply across both the apps and the data store for those apps. I know in the past you've expressed concern about the potential for AppSheet becoming too closely aligned with Workspace, and this is very much oriented towards providing people with an alternative data store to a Sheet if that is relevant to their environment.

In terms of this:

@SkrOYC wrote:

go AppSheet Database where you don't own your data but will get better performance and less cumbersome. That's the idea they are projecting.

I understand the perspective around closed-source - if that's a concern, yes, AppSheet and the AppSheet database are both closed source. In terms of data ownership, though, for Cloud, Workspace, and AppSheet specifically, the terms of service are quite clear in that you, the customer, own your data, and Google commits to only access your data in the process of delivering and improving the services they are stored in. Speaking for myself, from my time working in Cloud, Workspace, and AppSheet, this is a commitment that is taken very very seriously.

All that being said, we don't want to push you to use this in the future - we just want whatever will help users build the best apps, and we want to make sure that people have a variety of options based on their preferences and needs. I completely agree with the comments that if you are an experienced AppSheet user, the usability of the AppSheet database becomes less of a factor.

However, I think there's a gap between "I'm a very experienced user who just wants someplace to park my data that meets my needs" and "I'm a complete newbie to app building and need help getting started". Certainly we see that users, even those who have built an app successfully in the past, can struggle to get their sheets set up the way that they want, or need something in between a Sheet and a SQL database, and we hear from customers at decent-sized deployments of AppSheet that they want all their users to be using something like AppSheet database for their day-to-day needs because the manageability can be significantly improved.

Again, this is just the first step, and there's a lot of work yet to be done, but hopefully that adds some additional context. Thanks as always for the feedback.

SkrOYC

Thanks @zito for the comments. We all have to appreciate the fact that you take your time to give a proper response, we like it or not.

In this case I think that it's good to hear that it will be just another option and that AppSheet can be a service on it's own without depending on third-party services like MSExcel, GSheets, etc. That makes sense.

I'll consider this as just another feature, even if I use it or not

Again, thanks for your time

Fabian_Weller

@Koichi_Tsuji wrote:
Google Sheet have limiation 100k cell

The maximum number of cells in Google Sheet is 10 million.

https://workspaceupdates.googleblog.com/2022/03/ten-million-cells-google-sheets.html

zito

As always, thanks for the feedback, appreciate it. A couple of points that I want to flag:

- The size/scale limitation for AppSheet databases is the *current* size/scale limitation. We've done internal performance scale and testing to significantly higher data volumes, but because this is a public preview available to all users we want to make sure that unexpectedly high adoption doesn't negatively impact the experience. I expect the GA limits will be higher, and will continue to scale in the future.

- Agree that the regenerate structure is not ideal - we're looking at ways to address that, but didn't want to block the public preview on this, given the technical complexity. Stay tuned.

- From a performance perspective, I'm not quite sure I follow the logic - one of the reasons Sheets isn't as performant as a SQL database is because it doesn't have the notion of indexes, and its API is row-and-range based. There's all sorts of tricks we can implement to improve the read and search performance, but it's just not built to be a full data store. The Sheets team strategy for scaling sheets to much higher data volumes is Connected Sheets, where the data is stored in BigQuery and analyzed in Sheets - not using larger and larger sheets.

AppSheet databases are using a structured data store under the covers that is closer to a CloudSQL database, and we can apply indexes and performance optimizations automatically that are simply not available in a Sheet. It will likely never scale as large as a standalone CloudSQL database, and there are many advanced database features that are unlikely to find their way into AppSheet databases, but we can continue to advance and optimize this specifically for the use case of helping to build AppSheet apps.

SkrOYC

@Koichi_Tsuji wrote:
When this dicusssion was started, I questioned if this integration will achieve the real-time integration, but we found out it will not. At that point of time I know it, I lost the interst, then decided to stay with sheet and cloud SQL

Same. I though this AppSheet Databases was the response for a real-time AppSheet, considering that the datasource will be on AppSheet servers. But this is just another datasource that it's an AirTable lookalike.

I guess for some this will be useful.

@Koichi_Tsuji wrote:
With Google sheet, we can set the format of the cells as they are, roughly speaking. Setttin data type for AppSheet database in addition to Appsheet editor. Thank you god, new repetitive works to app creator, which will reduce the efficiency, while we have less storages. Again as I mention, learning new database setting, it will surely increase the cost to the app creators to learn new things

And we are not even talking about how modular can be the tables creation when using a worksheet with notes on headers, something that has never been mentioned in the docs but that is inmensively useful for creators. Now with this I'm stuck on mouse-land, something that's clearly not efficient

SkrOYC

I still think this is a good addition, but for Google and new users, not AppSheet creators that want the hundreds of Feature Ideas to be considered instead of new features that to be honest are here because of burocracy

Koichi_Tsuji

@SkrOYC

You should not be one not to test this new featue further... On the initial private release, we tested, but stopped to test as we found out this new feature will not bring a real beneft. So we ceased our hands.

I know the many community members are get excited with this new feature, but I personelly never disturb.

I just say Good luck.

Lidio_Pereira

I swear I believed this feature would be the holy grail of the APP's quick response.

I also did the test, but unfortunately I didn't see a significant performance change.

Should I still dream that one day a new update with significant performance changes? or maybe I will have to look for another solution for data growth?

SkrOYC

@Koichi_Tsuji wrote:
You should not be one not to test this new featue further... On the initial private release, we tested, but stopped to test as we found out this new feature will not bring a real beneft. So we ceased our hands.

Same as well, I was on the alpha and ditched the feature considering the fact that it provided almost no value for us. At the same time, I moved to help the Desktop Mode team with the bugs because that's a real needed and globally available feature.
Providing feedback for this AppSheet Databases felt like helping the development of a feature that I'd never use and worst will be another payed feature with limits of usage.

Koichi_Tsuji

Yeah, exactly same.

I (we) wont touch this new feature at all, and I close my eyes, and focus on others.

With the original post for this thread, the bunch of "user problems" are listed. However none of them will be solved by the introduction of this new features. That s what we would like to say.

Then I move away.

Koichi_Tsuji

I spoke too much... I zip my mouse. I wont say anything more about this project.

Good luck everyone those who have interest in this project.

lizlynch

@Koichi_Tsuji - Your feedback is always appreciated!

Koichi_Tsuji

We are explaining to our clients (major appsheet users), just ban this feature through the policy control, as it could be badly affecting to all their activities. sorry. We are never gonna touch this new features.

Atlas

Looks promising, I hope it does not have a row or cell limit in the future. But for now, I will keep using sheets and I will keep watching this thread.

Good luck team!

Henrique_Vieir2

Do we already have the limit per AppSheet subscription plan available? I just found it for the preview. This feature is amazing!

PabloDemichelis

Is there an estimate of how much time the synchronization time is reduced compared to using GSheets?
In my case I have 6 tables with many references and it takes 5 seconds to synchronize each time... What proportion would it improve with the DB?

SkrOYC

@PabloDemichelis wrote:
it takes 5 seconds to synchronize each time

That's on the highest performance category

Saddam

I really Appreciate it!!!!

Saddam

Dear Concern

I would like to know about new feature of Appsheet Build-In Database!
And I have some question about this feature!

Before start, I want to say that I'm very happy to got it. Because I feeled absents of this feature before. then I requested to higher level of Appsheet for this feature. finally I got it.

Thank you Appsheet👍.

=== Questions for Appsheet Build-In Database ====

Can I convert my all Google Sheet to Appsheet Build-In Database?
There are any restriction or limitation in Appsheet Build-In Database?
How many rows and columns will support in Appsheet Build-In Database?
Image and Videos where will be store?
Date and Time format is available according to locality? or Can I select the format?
How will work related table?
Can I Export or Import excel or any other format?
Should I need to pay for it?
How much Storage will give me?
Should I need to buy Storage?
After transfer ownership of Application then database will be transfer automatically?
Appsheet will work fast using this Build-In Database compare to other?
Is it possible to create column by auto request?

I'm waiting for your's valuable feedback...

Steve

I myself see no reason to move my own apps to Databases, and will stick with Google Sheets. While tinkering with it, I found it more difficult to use than Sheets. I like the flexibility of the less-rigorously-structured spreadsheet.

Of course, Databases is in preview, so that alone is a reason not to use it for anything important.

Consequently, I will not be supporting Databases in the community in the foreseeable future.

thierry6

🚀 awesome !

takuya_miyai

Hi @ShirleyN @zito

As new features are released, new ways to use AppSheet will increase, so I'm looking forward to AppSheet Database as well.

I am sure some of the current issues will be improved, but what I am most excited about at this stage was the performance improvements.

I have compared 10,000 rows of data between Sheet and AppSheet Database, and it appears that Sheet still has the advantage at this stage.

Reading the same data in Sheet and AD (AppSheet Database) required about 2.5 to 3 times the time.

Of course, since this is still a Preview function at present, I expect this difference to improve in the future.
I expect AppSheet Database to become a more powerful tool.🤗

zito

Thanks for doing this testing, and I recognize that you were pretty clear that you felt this feature was not useful for you, so I am doubly grateful for your time.

That being said, performance on individual reads and writes was not something we prioritized for the preview - not to say that is unimportant, but that the usability and user journeys were the primary focus.
I'll talk with the team and @ShirleyN about how we can look at performance through the preview process.

Ricky_Chua

Hi Zito, will there be any performance reveals for sync times between Sheets and Database?

As someone who has made reducing "sync times" for my organization a top priority, I am definitely homing in on the performance improvements should we migrate over to Database.

takuya_miyai

Also, 10,000 rows seems to be a soft limit.
In checking performance, 120,000 rows of data could be registered.
It was very difficult to add the data one by one.

Just kidding.😁
I simply copied the data with Add new row Action and Automation. I think the performance of AppSheet Database was better than Sheet for performing this copying.

Introducing new AppSheet database feature in public preview

Please see updated GA announcement --> here