Track changes after csv import

[machadolopes]

Hi there,

I need to import a csv set everyday with a list of employees leave and absence date ranges (about 4000 records). I have managed to create a bot to import the csv to the table but every import i make, the table adds all the records to the previous day (so, the second day i have 8000 rows in the table). Then i need to check it there was any change in this set comparing with the actual set (previous day). 

My concerns are.

1) Is it right to keep adding records to a table considering the row limit of  384,615 rows that would equal to only 96 days of importing data, not to mention the other existing tables in the app.

2) How to properly track changes (adds, deletions and changes) in the rows of each daily csv import?

my table have this headers:
<td>Name</td>
<td>Register#</td>
<td>Absence Reason</td>
<td>Status</td>
<td>Begin Date</td>
<td>End Date</td>

Thanks in advance.

[Aurelien]

Hi @machadolopes 

---

@machadolopes wrote:

1) Is it right to keep adding records to a table considering the row limit of  384,615 rows that would equal to only 96 days of importing data, not to mention the other existing tables in the app.

---

Not a good idea 🙂

AppSheet can face a limited amount of data, rumor has it that it's around 100k rows maximum.

You may want to run your csv import on another table, then use a bot to clean it up and save only the necessary rows.

---

@machadolopes wrote:

2) How to properly track changes (adds, deletions and changes) in the rows of each daily csv import?

---

You may want to use a bot, as stated above. Something like a "per row" process:

- if in the "reference table" and everything similar ==> use a status "unchanged"

- if in the "reference table" and one change ==> use a status "update"

and so on 🙂

For reference: 

Build automations - AppSheet Help

Automation limits - AppSheet Help

[Markus_Malessa]

What would be considered a unique identifier on your imported data? Could you concatenate multiple columns of your dataset and say that is the unique key column of your data? Then if you use the built in upload a CSV any key value it does not find already in the dataset it will add, but when the same key value is found it will update the row instead. I'm thinking could your key column be constructed using [Name]&[Register#]&[Begin Date] for example.