Transitioning your Document Tables from Preview to GA

Why is there a change?

In the early release of document tables the state of the document extraction was exposed as application visible data in the application schema. In particular there were columns for IsEdited, NeedsAttention, StatusCode and AttentionDetails. This coupled the business logic of the document content with the processing state details. It was decided this was not desired.

What has changed?

With the latest release, the status data is not available in the AppSheet tables. In addition, only documents that are considered finalized will be available in your application. If you have previously created tables with the previous release of document support, these applications will continue to work with a few caveats:

  • All columns that contained state information will no longer return data to your application.
  • All documents still in a not-final state will no longer appear in your application
  • Any application logic that depended on the state columns will not function properly (e.g. slices, automation, etc)

Instead, a new mechanism has been introduced to manage the not-final extractions. These will be logged to your application’s audit log, and those logs will contain links to help you resolve the issues (in this initial release, the resolution will need to occur in Sheets directly).

How do I align my old application with the new solution?

To bring your application into alignment with the future state of this feature, you have to take two actions:

  • Remove all dependencies on the state schema fields (e.g. slices, automations, etc).
  • Remove the schema fields from your table.

Once those two things have been completed, your application will have the same behavior and structure as if it were built with the updated release of this feature. As always, if you encounter unexpected issues with this migration, please reach out to AppSheet support.

What in the world is this about?

4 Likes

The document processing solution was released in February as part of the automation preview. We have made changes to the implementation details as part of the GA launch this week. This document explains to users who have built apps on the preview implementation, how to migrate their app definition to be aligned with the official GA implementation.

1 Like

I’m with Steve… LoL

What are document tables?

2 Likes

I believe he’s referring to the “use a GDrive folder as a table” preview feature.

1 Like

What is the “state” meaning in this context? Sorry English is not my language so not understand what it does mean here.

In addition , all in all , I don’t understand at all about this announcement what it does mean. … due to my lack in skill and knowledge should be blamed after 4 years with appsheet…

2 Likes

How would we both miss it? Hehehe

2 Likes

Hello, sorry for the confusion. I should have started with:

TL;DR: If you have not used document based tables, you can stop reading.

Background to hopefully clear up the confusion in this thread.

In February, as part of the Automation Preview launch, we also introduced a feature to allow users to define tables that will automatically import structured data from documents (initial launch is invoices, receipts and w9s). As part of the primer released with the announcement included several examples of building applications with the folder based data sources.

During the Preview period, we determined that it was preferable to move the error handling of document extraction from the app definition itself into a separate process. This rolled out yesterday to users, and the original post was to help those users understand why their apps have changed behavior, and what they need to do moving forward.

For those interested to learn more, there is a summary with links to more details about document based tables.

2 Likes

Probably I have been missing some important posts or announcemente made before, that s probably the reason, I m not able to catch up with you and what you said in this post.
I remember I did a quick test for invoice scanning feature, which was super buggy, and I gave the feedback. As well as invoice scanning, i tested folder reading as table, which was also buggy, I gave the bunch of feedback to your team.
Thats all what i remember.

1 Like

I m not sure if this post/thread is something to do with our awaiting feature of read folder as table. I just re-tested, but “path” fields still returning just file name instead of relative path value, which I reported to Appsheet Team, it is not making sense. We have file field where we have file name values so the two different fields are returnig the duplicated values.

1 Like

Sorry i need to correct my words.
extension was not displayed…

1 Like

If the file type is PDF, excel, word etc, then we can see the extension now.
But still we are not sure if they are google files, docs, slide, sheet, as no indicaition which file type they are.

Tsuji,

Thanks for these recommendations. I will prioritize making these changes.

1 Like

We need “relative path” for sure to read files faster than calling those files using google file IDs as far as our testing was concerned.
Yes, it could be technical challenges to read and display the file types for google files (doc, sheet, slide etc) but it really add more of fuels to Appsheet.
We have existing webhook for google docs to manipulate bunch of stuffs. Download docs in other file format etc. To make those to happen, we get to know what file type they are.

Thanks for listening and working hard to improve the platform.

1 Like

I just let my appsheet to read a folder with bunch of imiags files ( presumably 100,000 image files) but it takes forever.
Once the app grow as the time goes by as a result of users interaction with the app, the numbe of files naturally increase.
I m not testing properly, but server caching mechanism for folder oriented table could be useful, otherwise it is possible that app will generate this virtual table even for the huge (number of files) folder upon every single syncing, which is causing the massive amount of time for pure sync.

1 Like

I tested my idea.
Upload image, files from appsheet app (files are saved to google cloud)
Once the sync is finished, I got a URL to download the file (regardless of the file type)
Awesome.
I just utilized the IDs for the file wihch is returned by folder originated table.
We don’t need any PHP or any other backend stuffs, just manipulate appsheet expression only . Once again awesome thanks.

1 Like

Great. I am glad it is working for you. I will add the relative path fix in. Do you still think the mimetype info would be helpful? We can provide it as a text mimetype pretty easily, interpreting it into docs/sheets/etc can get more complicated

Agree with the caching for large folder. That is on our roadmap.

1 Like

Yes for sure.
I mentioned it to Scott before that we could be able to capture any of meta data as much as possible. Like Exif data as well.
Imagine, people upload file to the folder NOT through the appsheet, but wish to consume the file on appsheet.
Typically, image files.
Once we could get to read exit, then we can display those file on appsheet map view. Awesome, isnt it? We could read when those image and file are generated.

We do have some app like that ,but we are running GAS to read meta data, metatype for Exif and other info to display on appsheet map view, which is working fine, but my point is to make this happen without coding, as appsheet is purely no-code platform to do bits and pieces.

1 Like

Excellent points. We will include this in our roadmap. Thanks again for all your insights!

2 Likes

Thanks .

One more thing in our wish list
Once we direct read folder then we read all the files INC sub folders.
Even appsheet app folder structures are always nested . We don’t want to read each different folders one by one. Appsheet is lacking in capability to union tables.reading folders as different tables is reducing the capability.
Once again we use GAS to read all the sub folders

1 Like