Hello, I have a webhook moving records from o...

[Daisy_Ramirez]

Hello, I have a webhook moving records from one table to the next and its taking about 5 minutes per record to post. The user will often select multiple records so 5 records is currently taking 25 minutes to post. User would still need to sync to have records reflected in the view. Not functional for the user - how do I significantly improve performance ?

[Daisy_Ramirez]

@Philip_Garrett_Appsh Hi Philip - I tested 3 records using my developers login (WH_copy_favs_to_tasks REST API INVOKE: 4:04) and the total time to process was 66 seconds - we can function with this number.

When I login as a standard user (not the developers login) the total time to process was 706 seconds.I received the following errors in the process as well:

“Webhook HTTP post request failed with exception The underlying connection was closed: An unexpected error occurred on a receive. Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”.

Why would the process be dramatically slower for a non-developer login? Are there permission differences? I’d hate to start the redesign process if there’s just a simple adjustment needed. Thanks

[Phil]

@Daisy_Ramirez

I have discovered where the time is going. One of your virtual columns is the culprit. You can confirm this for yourself by checking the Performance Profile.

The API call is taking a total time of 05:06.156 (a little over five minutes and 6 seconds). Adding the new row is only taking 00.03.457 (a little over 3 seconds). The vast majority of the time is being spent calculating virtual columns.

That is taking 04:52.238 (a little over 4 minutes and 52 seconds). The worst of the virtual columns “last_task_id” is taking 04:47.535 (a little over 4 minutes and 48 seconds). If you reduce the time spent computing virtual columns your performance should be fine.

[Daisy_Ramirez]

@Philip_Garrett_Appsh Hi Philip it sounds like I’d have to redesign some of the fields in the main app. It doesn’t explain why the developer’s login (my login) processes a record almost immediately. Are there permissions the developer has over another user that make this so? I’ll do some more testing on both logins. Thanks Philip

[Daisy_Ramirez]

@Philip_Garrett_Appsh Hi Philip, I’ve been testing scenarios and regardless of whether I have the columns fully calculating or not, the developer login processes 3 records in right around 60 seconds every time. This is the case whether I’m on the computer or on the phone.

Unfortunately I cannot get a standard user to process 3 records anywhere close to 60 seconds when columns are being calculated only when column calculations have been removed. This leads me to believe there is another issue outside of the column calculation time. Please do let me know how I can escalate this one. Thanks Philip

[Phil]

@Daisy_Ramirez

What do you see when you look at the Performance Profile?

It will tell you exactly where the time is going. Take a close look at the time being taken to compute virtual columns. That was the problem the last time I looked.

[Daisy_Ramirez]

@Philip_Garrett_Appsh Hi Philip Levent was able to describe the reason the developer login generates a successful AND immediate posting vs. a secondary user - reconciliation of credentials, security etc.

I removed all initial values thought to be an issue from your previous post and also removed the LASTTASK_ID calculating field from the table and unfortunately there is no change in performance.

It still took over 45 minutes for 5 records to post in my last test scenario vs. 45 seconds on the developer login.

I could really use some help on this issue and am open to approaching in a different way if at the end of the day records post in a timely manner etc. Thanks.

[Phil]

@Daisy_Ramirez

Are you still working on app “3rdArm-506404”?

In that app the table “tasks” has a column called “last_task_id” with the AppFormula:

=MAXROW(tasks_user_slice_ALL, createdAt, AND([name]=[_THISROW].[name],[createdAt]<[_THISROW].[createdAt]))

The Performance Monitor says that AppFormula is taking too long to compute.

In order to protect other users from having one application monopolize the server, we limit how much time a single application request can take. Your API calls are exceeding that limit, so they are being cancelling once they hit that time limit.

[Daisy_Ramirez]

@Philip_Garrett_Appsh Yes, I’ve removed it and tested without it in place. I recently entered it back in for another test. I’ve placed a completely clean table in another app (no validations, initial values etc) and directed the webhook to post to that table instead. The time is over 45 minutes as well for 5 records.

Is there something I’m not considering in this case? Is it possible to re-import the table I’m posting to so I can keep the table in the same app?

Is there another way to seamlessly move data from one table to the next in bulk within the app so I don’t have to use an API?

[Phil]

@Daisy_Ramirez

I think you might want to work with Levent and see if he can help you accomplish this with Google Script.