Import CSV action is broken currently.

Koichi_Tsuji · 02-09-2023 12:45 AM

See the gif below.

Obviously broken. Import operation should be UPSERT. However, when csv has the duplicated IDs, AppSheet is currently uploading all of the records. Everyone must be careful now if your app has import CSV action which is currently affected.

@takuya_miyai

Steve

I know you aren't inclined to do so, but please contact AppSheet Support for help with this.

Koichi_Tsuji

Yeah, as you know, I always hesitated to do so, as the new support desk does not understand what the problem is. However, to contribute to the community, I have already reported to support desk and they acknowledged for your guidance.

Koichi_Tsuji

No update on this critical issues.....(x.x)

Some of our PROD app functionaities were lost due to this unfixed problems. It is obvously touching upon SLA I believe, as far as the working app is broken.

Last 10 days, no update from the support desk. I just sent reminders to get the update.

This is one of the example out of millions support problems.

Please dont excuse "we are on the long holiday" , which can not be an excuse..... (x.x)

@devingu

Koichi_Tsuji

Still no news from support desk even waiting for weeks, or almost a month.

In the meantime, the action type of "App: export this view to a CSV file (not a row-level action)" is also NOT working as before.

As its action name suggests, the only columns which can be seen on the list type view from where the actions are initiated used to be downloaded to CSV file. This was original spec when this export csv function was introduced.

But now it exports all the columns which exists on the table/slice the view is made out of.

If you want to hide some column, do not wish to let the users to view a set of column , then you hide from the table view. But export action will dump all of your secret to the users.

This is a huge risk for all the app creators.

@Rifad

I just say. Be careful.

Rifad

I don’t know what to do honestly at this kind of situations.

Koichi_Tsuji

Me neither .

Rifad

This can be scary. I am just panicking seeing this right now. I have few applications that use import feature. I don’t what and how I will find errors now.

@Koichi_Tsuji from when is this started?

Koichi_Tsuji

Hi @Rifad

We firstly noticed this when we get the questions from our clients, and we managed to repro the issue with our environment. This happened few weeks back. So it is possible this problem should have happened even before.

Anyway, the CSV import function is completely out of work, but Google never announced from their side proactively to the users .... That is also another problem currently Google has.

@takuya_miyai

Koichi_Tsuji

FYG, still not "proper" answers...from support desk.

We said our apps are broken (un-use because of that).

The support desk said it take some time, but who cares......

They said there will be workaround, then we ask what it is.

They gone quiet on me.

This is almost daily seeing story when it comes to "app breaking stories happens". I m surprisedd no higher Google management personnels get involved while the users are not able to user the app wihch have been up and running and all the sudden, the app fails due to lack of the QA inside Google.

Yes, the same stories had happened with AppSheet (before G), but such a problem should have been fixed straighaway after we report such.

EvelynM

Recibiste respuesta, cual fue la solución?

Koichi_Tsuji

Not sure if you read my post, but probably not. I asked support desk to see what their workaround in their mind, but they never ever come back.

Usual Google/AppSheet desk silly story

Koichi_Tsuji

This single problem imply bunch of Google Problem.

This is critical issue, which makes apps un-use while they have been working fine.

Not sure what causes the new problems for those apps, but factually the apps are broken.

Then asked suport desk for help.

No response. If you are lucky, then they start to ask silly questions. The typical silly question that is always coming "what is your app ID"?

Silly.

I always ask AppSheet support desk guys, to get your hands dirty. It takes probably 2 min to repro the reported problem, then you will realize what the issue is.

Once we share the ID of the probleming app, then it will give an excuse to the support desk guys.

"We tested your app by coping , no problem. App should work properyl We will close the case if ther is no response back from you"

No hope.

Google is testing the problematic app with their own environmennt, not testing users app in the same conditions.

Koichi_Tsuji

Phil san (i know who orignally introduced tihs new feature with us .., but retired), can you come back to fix this problems.

Afer the weeks, it is obvious to me there is nodoby in Google to fix the problems...... our claim never taken while our apps are not working at all. Using export CSV, then the apps are broken as the source table s having the rows with the duplicated keys...

This is only one of 1000 issues the new (after Google) is preserving. But most of them (nearly 99%) ARE NOT SOLVED.

We only here from the support desk, the issues are addresed after sending the renders. But thei anwer is silly. Without our reminder , they never the update the issue to the posters.

I claim over this poor services , which have no exist before Google aquisition, then all the guys inc my ex AppSheet friends they say this is how Google goes, which I never case. It is not my own problem......... The Appsheet is likely to be complely destroyed by the Google. The percentable of this lielyhood is getting higher honestly. One in Google office, please let me know once you decide to sale AppSheet out to someone. I will get in touch with them accordingly, before Google kills appsheet.

Steve

@Koichi_Tsuji wrote:

Still no news from support desk even waiting for weeks, or almost a month.

Attn @devingu, @isdal

isdal

Thanks for escalating this one Steve. @Dan_Bahir for FYI

@Koichi_Tsuji From our investigation it looks like this happens when the CSV being imported has duplicate keys, it is possible that this is a regression but we have no recent changes (for many months) in this area of the system.

In our tests CSV import works as expected for files with unique keys, in those cases any existing rows will be updated and not duplicated. A workaround is to ensure that the imported CSV is well formatted and without duplicate keys.

It would help us to understand a bit more about your use case, for example, if you import a csv file with duplicate keys, which row should "win"? In the normal system the most recent write wins but in a csv file we don't have that information. The top most one, bottom most one and random are all possibilities.

Koichi_Tsuji

This is coming from the user cases of our clients.

They have external system which export CSV which always contains duplicated rows. They have no control to avoid such as it is the third party system. They need to import this CSV to AppSheet app. The absolute starting point is to start to NIL records on the table. Then as you see in my screenshot (GIF) I shared here, AppSheet is jsut import all the duplicated row. It is not a question if or not the behavior is changed or not, but it is critical issues for all the AppSheet user as AppSheet allow the dulicated rows entry through the Import CSV action.

Koichi_Tsuji

The problem is if there is no existing ID on the table.

If the csv files contains the duplicated row value like "A0001" while there is no existing records with key of "A0001" value. Then we export. The result is duplicated rows are generated And the app is going to be broken.

It is always possible the App user wrongly import the CSV to the target table while CSV file contains the duplicated rows. They are realistically not able to check if the existing rows are saved on the table/apps already , before they fire import actions.

All in all, this is a glitch/bug, risking the security of our apps.

Koichi_Tsuji

From the statement of Praveen (ex. CEO) of AppSheet, I understand Google does not want to take any aciton on the existing codes (features given to the users) as far as possible. This single example is demonstrating what Praveen said is right.

Without hearing the problem out of the actual users and take such a voice seriously, they stick with "We never made changes", which we never care and we are not sure if it is right or not. That is pointless.

They forgot to hear the voice from the users to identify what the hidden problem is to improve the services.

Frastrating overall.

Koichi_Tsuji

New support desk is not helping as I repeated 100s time or more.They are just focusing on extinguish fire (claim from users) rather than understanding what the real problem behind.

Thats why they can not deal with the users voice in the proper manner, but just pushig their thought all the time.

Claiming the errors and problem will never be listened and taken properly. How can we develop our business app? Business app need seamlessly working while AppSheet app pose problems almost daily. Report such to here or Google (I dont want to post a request for help to proper support desk, as it is silly), then no response which is average response. THen after weeks, they come back it is not a problem... Which god we have to pray on this circumstances. I m always at a loss.

Most of my old friend from Appsheet are left now, so I have nobody to speak to even privately. Probably, this is a fact things are "Googlized"

momoi

@Koichi_Tsuji

I experienced this phenomenon at least a year or more ago.

AppSheet checks the KEY values of the records that already exist in the table and the KEY values of the imported records to detect duplicates. Duplicate judgment of records in the dataset being imported is not performed.

I believe this is a specification of AppSheet. Such specifications are not unusual and could be other databases as well.

I think the imported dataset is considered a single chunk. I'm guessing that the AppSheet backend system can compare records being imported with records already in the table, but not between records being imported.

As I said, this phenomenon is not that unusual. If you expect to have records with duplicate KEY values in your CSV, I think your only option is to do data cleansing before importing.😀

Koichi_Tsuji

@isdal

See my additional comments. You dont know what I have been claiming from the beginning.

IF or not the functionality changes or not is not an issue......

IF you agree with what I claim is a problem, please take action internally in Google.

If you disagree, then it is fine. However, all the community member should know how Google cheaply deal with the users voices, even though we detected the false in Appsheet.

d_bahir

What is the behavior that your would expect ?
I see several options on how to handle this situation:
1. If there are multiple occurrences with the same key, insert the first and update if there are subsequent rows.
2. Notify the user that that there are rows with duplicate keys in the import file and that is not supported.
3. Insert just the first row and discard the others.

My expectations is that most users have files with clean data, without duplicates and errors and therefore would appreciate an error if there are rows with duplicate keys.
Could you please provide more information on your expectations ?

Koichi_Tsuji

@Steve

@devingu

After a month wait, I got following from support desk. Honestly, the context of their response is not surprising, but just "dissapointing". I m at a loss and no more power to continue to talk with them, as it will not help us at all.

Firstly, my point by raising this issue/ticket is NOT to discuss about "workaround". Once the users upload CSV with duplicated rows, then App is going to be breaking up. This is a risk, and basically we have no "workaround" to completely remove this risk to 0 percent. They always (recently) insist "there is no changes made for this particular functionalities), but that is not our point. Regardless of such an issue is inheritaed from the day 1 after the feature is GAed or not, any hidden defects will be found later.

It is obvious that Google does not want to take such a voice from us in positive way, but it appears to me the support desk is just aiming at "closing ticket", rather than seriously looking into the core of the problem. In this case, they should consider any possible tequnical meatures to avoid the case where app is completely broken down by the users interaction.

There should be some options with them.

1) Review the code related with Import CSV (I know Google is not willing to touch and change the exisitng code though... but it should be possible)

2) Implement some new mechanism to throw errors (fail the import csv operation) when the app users try to import the csv file where the key values is not exsting in the target table, but there is duplicated inside CSV.

--

But unfortunately, such a suggestion will not get through...... Very much annoying.

Anyway, everyone, we have to be careful when it come to Import CSV action if it is in place with your app. It could be possible it will break your app anytime. What I can do is just share my experiences and give precaution to the community.

==

Hi Koichi,

Thank you for your patience.

Our specialists have provided several options on how to handle this situation:

1. If there are multiple occurrences with the same key, insert the rows first and update if there are subsequent rows.

2. Notify the user that there are rows with duplicate keys in the import file and that is not supported.

3. Insert just the first row and discard the others.

Our expectation is that you have files with clean data, without duplicates and errors and therefore would appreciate an error if there are rows with duplicate keys.

Could you please provide more information on your expectations so we can provide those details to our specialists in order to provide you a better resolution to the case.

In case of any further issue you can feel free to contact us.

Koichi_Tsuji

After struggling with support desk over a month, they finally sent a relatively positive response.

They seemingly admit (finally) "This is a problem".......

They must react on day 1 in that way, but I sincerely hope AppSheet devs team get more serious to address this type of issue, as far as any single operation breaks up as a part of serurity loop hole.

Just update.

Also hope they will not close the ticket with me without me knowing about it to contribute their own KPI (how much the support desk close the tickets..)

@Rifad

Koichi_Tsuji

In the meantime, another CSV action issue I reported (dump ALL the column values rather than column on the view) is never addressed.

The level of the seriousness is equal with this issue, but I hope someday and sometime AppSheet support desk get aware of it. I fed up with strugging with them , so I will not push them no more. Why?

I m tired with them.

Rifad

Yeah these days I ask them to connect me directly to a developer. Still they send me a copy paste message. And after explaining for an hour they admit its a bug and then tell it will be transferred to specialist.

Many of my apps are broken and Inventory everything is messed up because of their irresponsible behaviour.

I am not sure what words should I put to make them understand the severity of issues sometimes.

Koichi_Tsuji

"Specialist" ??????????

I m not sure who they are. Obviously no skill and no knowledge as far as we assess viewing their idiotical response.

Koichi_Tsuji

What a joke.