Enhance CSV Export to Support User Specified 'Search Filters'

The CSV Export feature is described in this article https://help.appsheet.com/en/articles/5337071-export-this-view-to-a-csv-file
As described in topic “Row Filtering” in this article there are four ways to filter the rows of the exported view.

  1. The app creator can base the View on a Slice having a Row Filter Condition. Only rows that match the Row Filter Condition are exported.
  2. The app creator can specify a LinkToFilteredView with a filter expression. Only rows that match the filter expression are exported.
  3. The app user can specify a Search Term in the Search box for the view while running the app. The app user can search any field for which Search? is true. Only rows that contain the Search Term are exported.
  4. The app user can specify one or more Search Filter conditions using the Filter dropdown in the Search box while running the app. The app user can can apply Search Filter conditions for any combination of fields. Only rows that match all of the Search Filter conditions are exported.

Prior to this change, the server honored the first three forms of row filtering but not the fourth when doing CSV Export.
The server simply ignored this fourth form of filtering when performing CSV Export.

This problem has been corrected. The server now honors all four forms of row filtering when performing CSV Export.

This change will be rolled out over the next week or two. If you are having this problem and want to be included in the rollout, please respond and provide the account id that owns the app.

16 39 2,060
39 REPLIES 39

Awesome!!

SkrOYC
Participant V

I need this on acc 2445010 and 2365694
Thanks!

Done. Please let me know if it is now working for you.

Nope, not yet.

Hamlet,

Please submit a bug and ask support to assign it directly to me.

Please include:

  1. Your account id
  2. Your app name
  3. The exact steps to reproduce the problem. Please tell me the exact steps to follow.
  4. Please ensure you have granted support access.

Report submitted.

SkrOYC,

Done. Please let me know if it is now working for you.

It’s working great for me.
Thanks!

Hamlet
Participant V

372242 - acct id

Thanks!

Thanks for the update SkrOYC.
Glad to hear that it is working.

Hi Hamlet,

I used your app to confirm that the new CSV Export code is working when you are in the rollout.
For some reason, it appears that the server is not recognizing that your account is in the dynamic rollout. I went out to dynamic rollout admin page and confirmed that your account it of 372242 is listed there. Frankly, I am stumped. Can you give it a day and then try again. I am not entirely sure how quickly dynamic rollout updates reach the servers. I thought it was immediate, but I may be mistaken.

Please let me know what happens when you try it again tomorrow.

I have a child view in a dashboard being filtered by a parent view. The child view has an export button (overlaid actions enabled). The export I get when I export from the dashboard view isn’t filtered. I have to full screen the child view in order for it to export the filtered results. Is this expected behavior or is this a bug?

Is your view based on a Slice that is filtered by a Row Filter condition?

Yeah the view is from a filtered slice but it can be further filtered by selecting rows from the parent view in the dashboard.

I believe this is referring to an Interactive Dashboard?

Filter by interactive dashboard is not mentioned in the most latest update from team when the export CSV is capable of dumping data based on view the user is seeing on the app.
So my understanding is it is not possible for the moment.

Probably need to employ deeplink action to bring users from dashboard to table view using link to filtered expression before exporting data.

This would mean that this new “View aware” export csv is breaking the previous way were if we have a dataset filtered by a Slice it should respect what’s inside the slice:

No . Filter on interactive dashboard is not really a slice .

Maybe we are talking about to different topics. But I’ve created dashboards with a table view based on a slice and a detail view with quick edits. The slice listens (the row filter condition) to the detail view and changes it’s dataset accordingly, making the table view change at the same time.
This is what I though you were refering to as “Interactive Dashboard”.
Now, if you are talking about the “Interactive mode” were using ref between views each view can change a little bit, I’m with you about the fact that is not a feature available at the moment

Derick_Paula
Participant III

Nice!

Hi Jordan,

To be honest, I am not sure I understand what you are describing.

Can you do the following:

  1. Create a simple test app that illustrates the problem.
  2. Tell me what account id the test app is running under so I can include that account in the rollout. You could create the app in a new free test account that you create if you don’t want to include your production account in the rollout.
  3. If the Export does not work despite being in the rollout, then submit a bug report.
  4. In the bug report include:
    a. The account id
    b. The app name
    c. The exact steps for me to follow when running the app to reproduce the problem. Please be as explicit as possible when describing each step because I will be following your steps explicitly.

Hey Phil,

Unfortunately, I don’t have time to create a new app to demonstrate this. I wish appsheet had a Wingdings font (or some way of hiding the text) so I could make quick video screenshots without having to worry about exposing sensitive data. In any case:

I have a Parent table and a Child table in a Dashboard view which has Interactive Mode enabled. Accordingly, when i click on a row in the Parent view, it filters the Child view (basic stuff).

Separately, the parent also happens to be from a filtered slice. I can’t imagine that this has any effect on the overlay action behavior but, @SkrOYC asked which is why I mentioned it.

Hi Jordan,

I completely understand. I am really busy too.

Let’s just ignore it for now. If there is a problem, someone else will probably trip over it who has time to submit a bug report.

Just additional feedback (not for you only, but for your UI team basically)

Once we hit export action, there is no indication currenlty available if the app is really processing job or not… if the data set is relatively large, then after a min, we see the exported csv file at the end. During the process is ongoing, we dont now if the process is really doing something , as there is nil indication. At the end, this gives a risk, the user hit the export csv button quite few time, resultiing in number of downloaded files.

@Takuya_Miyai

Hi Koichi san,

Early this week, a customer reported that a CSV Export was taking a very long time and producing a truncated CSV file. In some instances, the CSV file contained only half of the expected 32,800 records. The number of records contained in the CSV file varied from execution to execution. No error message was displayed.

When I investigated, I learned the following:

Previously method ‘Evaluator.RowEntries’ ignored Export Time Limit exceptions thrown by the Expression System. We caught the Time Out Exception but ignored it.

Unfortunately, ignoring the error and continuing had three bad consequences.

  1. It made it impossible to determine why Export was failing.
  2. When the exception was caused by a time out, we ignored the time out and continued processing rows. This lead to subsequent time outs on all subsequent rows. All of which were also ignored. As a result, the Export could greatly exceed the time out limit.
  3. When a time out occurred, we simply dropped the rows that exceeded the time out from the ‘RowDetailsReply’. As a result, the Exported CSV file was truncated at some arbitrary point with no explanation.

We now report the time out exception, provide diagnostic information, and immediately terminate the Export.

The customer’s export was taking a very long time because 23 of the 40 fields in the table being exported had complex Editable_If expressions. These were being evaluated for each row being exported. This resulted in the over 750,000 expensive calls to the expression system. Those calls were causing the time out. I was able to help the customer rewrite the expression to make it more efficient. We also moved the Editable_If expression into a virtual column and had the Editable_If expressions in the other 23 fields refer to the value in the newly added virtual column. This was possible because the Editable_If expressions in all 23 columns were identical.

I mention this because once my change is released later this week, the Export should terminate once the Export time limit of 2 minutes is reached. That should eliminate very long running CSV Exports that yield bad results. I have also improved the error message that is displayed when a time out occurs. Hopefully, it will make it easier to diagnose such problems in the future.

HI @Phil san,

Thanks for your explanation.
What I really suggested is not only in terms of dealing with error, but also ‘success’ as as well.

When the export CSV process is kicked in by hitting action icon, it stays silent. When the set of thed data is releatively large, it often take sometime, min or mins. During the exporting csv process is ongoing, there is no indication on client side. On importing CSV file, it is not perfect but at least the modal pops us, saying “it make take some time to complete job”

All in all, once the exporting to CSV action is fired, better to give some sort of indicaiton to the app users to tell them that AppSheet is doing a job, which take a time. It is just purely UX/UI issue.

In the meantime, if we try to create Excel file, using Automation/BOT, for 32,800 number of the rows, I expect it will fail due to time out (2 min wall). But yes, Exporting to CSV with the same number of the records, it want fail.

I scratched my head, why exporting to CSV is relatively powerful (at least do a job within 2 mins time for large set of records) while the technically opposite operation of creating Excel file operation under BOT for the same number of records are often easily fail due to time out.

My assumption is exporting number of API calls are differenct between those two different operation due to difference in backend codes to do the jobs.

Cheers.

@Takuya_Miyai

Manupunk
Participant I

thanks, really thanks

Hi Koichi san,

Please submit a bug with clear directions on how to reproduce the Excel file problem.

@Phil san,
Thank you sir. Will do and submit support ticket later today, along with sample app where you can dig into.
Cheers.

In a few days ago, I submitted a request to support desk asking to pin you for the support, did you get it?

@Phil

Hamlet’s Export View problem was occurring when he specified a Search Filter condition on a Ref field that referred to a table with a Numeric key. Many of the Ref field values in the exported table were blank. When the Search Filter expression was evaluated for rows containing a blank Ref field value, those rows were incorrectly being included in the exported CVS file.

This problem can be avoid by following the guidance contain in topic “Unexpected rows appear in the Exported View when using a Search Filter conditions” in this article Troubleshooting CSV Export | AppSheet Help Center

Thank you for this! Last week was the first time I tried a filtered view with the export feature and was confused why it exported all records.

Koichi san wrote:

Export CSV will now display a dialog box say “Export CSV Started” when the export action begins.
When the export action completes, it will display a dialog box saying “Export CSV Succeeded” if export is successful or a dialog box containing the error message if export fails.

This change will be released shortly after the Thanksgiving holiday here in the United States.
We are observing a code freeze on new features until then.

@Phil san, This is perfect💯 Thank you for listening to us.

Could you please update this on Feature Release Note once the new code is deployed so that other user get to know through the note? Thank you again.

@Takuya_Miyai

Hamlet has uncovered another Export CSV bug.

The problem occurs when using a Search Term in the Search box of the view being exported.
If the table being exported contains a Ref to another table, the Search Term should look for a matching value in the label field of the reference row. The client does this correctly; the server does not. The server simply ignores such matching rows and does not export them.

I am working on a fix for this problem.

I did but I am buried in bugs.

@Phil san, thank you for your confirmation. As far as my ticket was delivered to you, it is fine for now. Please take your time.
I created sample app, which hopefully make your life little easier to debug the issue. Cheers.

Thanks for being so understand Koichi san.