I'm working with a virtual column whose type ...

Iโ€™m working with a virtual column whose type is โ€œNumberโ€ and have been having trouble getting decimals to calculate properly.

If I use โ€œ1/3โ€ in an expression the result of the calculation is โ€œ0โ€.

If, however, I type the expression as โ€œ1/3.00โ€, which should produce the same result, I get โ€œ0.33โ€.

Can anyone tell me what is going on?

I want โ€œ0.33,โ€ so Iโ€™m happy with the result now, but I wonder if I have to type something like โ€œ1/3.00โ€ to get the calculation to work properly or if thereโ€™s some other issue of which Iโ€™m not aware. Also, Is this issue unique to virtual columns?

0 4 343
4 REPLIES 4

Now I see that the truncation of integers has been discussed before:

โ€œLooks like youโ€™re getting โ€œinteger divisionโ€ - because both inputs are integers the expression type system infers the output should be an integer, and 55/60 truncated to the nearest integer is 0.โ€ โ€“ Adam Stone

https://plus.google.com/+MultitechVisions/posts/2JkMZC86LcC

I guess the solution I happened to find is a known workaround.

Nonetheless, itโ€™s not at all intuitive and I couldnโ€™t find anything about it in the โ€œMath Expressionsโ€ documentation.

I would suggest that the issue be fixed in some way or that documentation be improved.

@Kirk_Masden

In general programming interface there isnโ€™t an integer and/or decimal concept. Instead there are number and float concepts matching them. Therefore, for example in Java or JS, when you log (1/3) you get a 0. But if you use float(1/3) then you get a 0.33 value. The concept in AppSheet is pretty much the same.

Thank you, @Levent_KULACOGLU!

This has been a good learning experience for me.

For AppSheet, however, I think more/better documentation is needed because I think the whole idea is for the platform to be easy to use for people who are not programmers.

Thanks to you, though, my own issue has been resolved.

@Kirk_Masden We should explain this better, thatโ€™s true. +Steve Coile FYI

Top Labels in this Space