Chained Derefs

Bahbus
New Member

So… uh… How long have we been able to do this?

7 25 1,074
25 REPLIES 25

I’m able to use this after they featured this in Office Hours

For about a month. We tried to roll this feature out four months ago but it caused some unexpected side-effects with references, so we had to proceed more cautiously.

This is a very valuable capability and much requested by the more advanced app creators. Good work by
@natalie in our dev team.

Ok, sweet. I thought I had saw it was possible, but then as I tried searching around couldn’t find the proof again, so I also thought I was going crazy.

However, I would like to note for you and & @natalie (without me knowing if this is just still not ready/possible) that this doesn’t seem to work for me:
[Ref].[Related List][Text Col]

Just in case it’s supposed to.

That would be cool!

Yeah! I really want something like that to just evaluate to a List of Text.

Or, in the case of some saucy like [Ref].[Related List][Another List] (and any use of [List][Another List]), I would like it to flatten to just List of [Another List] base type instead of, List of List.

@praveen for us formula aficionados, may I ask what exactly is happening to make this magic happen? Is there an equivalent SELECT() that would explain it?
3X_d_5_d51363a862e7ab883241c312ac5d7f271579cdd3.gif


What I’m also really wondering is what sort of performance effects this would have on an app with lots of data?

Will using chained derefes instead of adding that extra virtual column bridge increase the performance of an app?

Is it worth going back to completed apps and implementing this? I have some that contain a ton of in-between virtual columns to get the extra layer of depth.

I am only because otherwise I have a VERY expensive VC, where as now I can traverse already calculated paths and lists.

Um.
Yes.

um
Ok
Thanks

My expensive VC just went from an average sync of 30 sec to 5 for just itself - which is still my most expensive. So, that’s a major win.
3X_9_d_9da834179525dcb7a705f1938417cfc0619f8d1e.png

@Steve it looks like no one knew we could chain derefs, and the docs need updating.

Done.

Beautiful.
Thanks @Steve for always turning out!

I didn’t know :(, wish i did ha. I listened to the office hours but pretty sure they said it was something they were working on, not that it had been rolled out.

I’m part of the people that didn’t know…

Nobody told me.

Seriously? ok.

Guys, 2021 WAS GOING TO BRING MORE TRANSPARENCY ETC. Lets not Bitch and hope it happens

I’m most excited about this for workflow report documents…

I think it’s fascinating that no one ever wants to take public credit for new features and bug fixes… This community would celebrate and high five the developer(s) responsible so hard!

So True, I would be all over that lol

I guess it’s still 2020 so we’re still releasing features and telling nobody :[

This just needs a process and it completely fell between the cracks — it’ll be one of the first things we do in January.

Wrt how it’s implemented, the problem with chained/nested derefs was one of parsing — [A].[B].[C] doesn’t match any expresion syntax supported by an Excel/sheets formula parser. In the past, I’d tried changing the parser we use, but that ended up being very messy and I abandoned it. @Natalie took another crack at it, and she used a smart pre-processing technique that let us parse these expressions. Once that was done, then the rest of it was relatively within our grasp (it was code we are familiar with in our expression system).
You can think of a dereference expression as doing a “lookup” on the fly. A two-step nested deref has two lookups one after the other. Each lookup is efficient because it is looking something up based on its key and that is an equality comparison on a unique value.
Until we had this chained deref expression, you’d have to break this up by pre-computing virtual columns. That is essentially pre-computing all but the first hop of the chained deref lookup, and storing their results as virtual columns. So this pre-computation happens for all the rows, not just for the particular row you might care about. The precomputation happens in our server at sync time.

So which is preferable? If you’re in any case going to have to compute the chained deref for all N rows of your table, then maybe you’re better off with the VC approach. You’re using our server to do it and you need more space for the data, but then it’s a little less work in your app on your device. However, if you only need this chained deref occasionally (let’s say you use it in a form to fill in a value), then it’s going to be better without the VC. In general though, I’d be surprised if a sync time was increased from 5s to 20s because of a VC that just precomputed and cached a deref expression. So that seems odd.

Thank you @praveen very good explanation

@praveen and @natalie, thanks for investing in this feature and explaining how it works. I love learning the how the platform works!!!

This feature has been on my wish list for a long time and whether or not it’s more performant than carrying over virtual columns it’s certainly more intuitive/user friendly! That’s the primary benefit in my opinion.

Thanks for tucking this feature under the tree for us this year!

Top Labels in this Space