Hi Guys, what is the best practice to handle large google sheets tables in excess of 100 000 rows? I’m wanting to pull in a sheet of transactional sales data that is extremely long, but having all of the data is key to some of the machine learning practices I am wanting to try and achieve…
I have been playing around with some ideas in this area.
First, for basic management of the transactions, you likely do not need all of the history. I would recommend using Security Filters to trim down the rows for more recent history only.
I’m curious what kind machine learning you are applying that you would need 100000 rows?
One of my concerns is charting against history, without pulling in all the history rows. The charts deal mainly with summarized information for daily, weekly, monthly, quarterly, etc. My thought here is capture the summarized details in their own table that remains mostly static. A challenge is dealing with situations such as corrections to the history.
I am then toying with the idea of placing history in a separate sheet and building separate apps, one each for history and reporting/charts.
Scripting services would likely come into play heavily to manage the behind the scenes movement of the data between sheets.