Enhance CSV Export to Support User Specified 'Search Filters'

Yeah the view is from a filtered slice but it can be further filtered by selecting rows from the parent view in the dashboard.

Hi Jordan,

To be honest, I am not sure I understand what you are describing.

Can you do the following:

  1. Create a simple test app that illustrates the problem.
  2. Tell me what account id the test app is running under so I can include that account in the rollout. You could create the app in a new free test account that you create if you don’t want to include your production account in the rollout.
  3. If the Export does not work despite being in the rollout, then submit a bug report.
  4. In the bug report include:
    a. The account id
    b. The app name
    c. The exact steps for me to follow when running the app to reproduce the problem. Please be as explicit as possible when describing each step because I will be following your steps explicitly.
1 Like

I believe this is referring to an Interactive Dashboard?

1 Like

Hey Phil,

Unfortunately, I don’t have time to create a new app to demonstrate this. I wish appsheet had a Wingdings font (or some way of hiding the text) so I could make quick video screenshots without having to worry about exposing sensitive data. In any case:

I have a Parent table and a Child table in a Dashboard view which has Interactive Mode enabled. Accordingly, when i click on a row in the Parent view, it filters the Child view (basic stuff).

Separately, the parent also happens to be from a filtered slice. I can’t imagine that this has any effect on the overlay action behavior but, @SkrOYC asked which is why I mentioned it.

Hi Jordan,

I completely understand. I am really busy too.

Let’s just ignore it for now. If there is a problem, someone else will probably trip over it who has time to submit a bug report.

1 Like

Just additional feedback (not for you only, but for your UI team basically)

Once we hit export action, there is no indication currenlty available if the app is really processing job or not… if the data set is relatively large, then after a min, we see the exported csv file at the end. During the process is ongoing, we dont now if the process is really doing something , as there is nil indication. At the end, this gives a risk, the user hit the export csv button quite few time, resultiing in number of downloaded files.

@Takuya_Miyai

4 Likes

Hi Koichi san,

Early this week, a customer reported that a CSV Export was taking a very long time and producing a truncated CSV file. In some instances, the CSV file contained only half of the expected 32,800 records. The number of records contained in the CSV file varied from execution to execution. No error message was displayed.

When I investigated, I learned the following:

Previously method ‘Evaluator.RowEntries’ ignored Export Time Limit exceptions thrown by the Expression System. We caught the Time Out Exception but ignored it.

Unfortunately, ignoring the error and continuing had three bad consequences.

  1. It made it impossible to determine why Export was failing.
  2. When the exception was caused by a time out, we ignored the time out and continued processing rows. This lead to subsequent time outs on all subsequent rows. All of which were also ignored. As a result, the Export could greatly exceed the time out limit.
  3. When a time out occurred, we simply dropped the rows that exceeded the time out from the ‘RowDetailsReply’. As a result, the Exported CSV file was truncated at some arbitrary point with no explanation.

We now report the time out exception, provide diagnostic information, and immediately terminate the Export.

The customer’s export was taking a very long time because 23 of the 40 fields in the table being exported had complex Editable_If expressions. These were being evaluated for each row being exported. This resulted in the over 750,000 expensive calls to the expression system. Those calls were causing the time out. I was able to help the customer rewrite the expression to make it more efficient. We also moved the Editable_If expression into a virtual column and had the Editable_If expressions in the other 23 fields refer to the value in the newly added virtual column. This was possible because the Editable_If expressions in all 23 columns were identical.

I mention this because once my change is released later this week, the Export should terminate once the Export time limit of 2 minutes is reached. That should eliminate very long running CSV Exports that yield bad results. I have also improved the error message that is displayed when a time out occurs. Hopefully, it will make it easier to diagnose such problems in the future.

2 Likes

thanks, really thanks

HI @Phil san,

Thanks for your explanation.
What I really suggested is not only in terms of dealing with error, but also ‘success’ as as well.

When the export CSV process is kicked in by hitting action icon, it stays silent. When the set of thed data is releatively large, it often take sometime, min or mins. During the exporting csv process is ongoing, there is no indication on client side. On importing CSV file, it is not perfect but at least the modal pops us, saying “it make take some time to complete job”

All in all, once the exporting to CSV action is fired, better to give some sort of indicaiton to the app users to tell them that AppSheet is doing a job, which take a time. It is just purely UX/UI issue.

In the meantime, if we try to create Excel file, using Automation/BOT, for 32,800 number of the rows, I expect it will fail due to time out (2 min wall). But yes, Exporting to CSV with the same number of the records, it want fail.

I scratched my head, why exporting to CSV is relatively powerful (at least do a job within 2 mins time for large set of records) while the technically opposite operation of creating Excel file operation under BOT for the same number of records are often easily fail due to time out.

My assumption is exporting number of API calls are differenct between those two different operation due to difference in backend codes to do the jobs.

Cheers.

@Takuya_Miyai

2 Likes

Hi Koichi san,

Please submit a bug with clear directions on how to reproduce the Excel file problem.

1 Like

@Phil san,
Thank you sir. Will do and submit support ticket later today, along with sample app where you can dig into.
Cheers.

1 Like

Hamlet’s Export View problem was occurring when he specified a Search Filter condition on a Ref field that referred to a table with a Numeric key. Many of the Ref field values in the exported table were blank. When the Search Filter expression was evaluated for rows containing a blank Ref field value, those rows were incorrectly being included in the exported CVS file.

This problem can be avoid by following the guidance contain in topic “Unexpected rows appear in the Exported View when using a Search Filter conditions” in this article Troubleshooting CSV Export | AppSheet Help Center

1 Like

Thank you for this! Last week was the first time I tried a filtered view with the export feature and was confused why it exported all records.

Koichi san wrote:

Export CSV will now display a dialog box say “Export CSV Started” when the export action begins.
When the export action completes, it will display a dialog box saying “Export CSV Succeeded” if export is successful or a dialog box containing the error message if export fails.

This change will be released shortly after the Thanksgiving holiday here in the United States.
We are observing a code freeze on new features until then.

3 Likes

Hamlet has uncovered another Export CSV bug.

The problem occurs when using a Search Term in the Search box of the view being exported.
If the table being exported contains a Ref to another table, the Search Term should look for a matching value in the label field of the reference row. The client does this correctly; the server does not. The server simply ignores such matching rows and does not export them.

I am working on a fix for this problem.

3 Likes

@Phil san, This is perfect💯 Thank you for listening to us.

Could you please update this on Feature Release Note once the new code is deployed so that other user get to know through the note? Thank you again.

@Takuya_Miyai

3 Likes

In a few days ago, I submitted a request to support desk asking to pin you for the support, did you get it?

@Phil

I did but I am buried in bugs.

2 Likes

@Phil san, thank you for your confirmation. As far as my ticket was delivered to you, it is fine for now. Please take your time.
I created sample app, which hopefully make your life little easier to debug the issue. Cheers.

1 Like

Thanks for being so understand Koichi san.

1 Like