Monday, January 31, 2011

LS: Currency data type seems to return proper results in arithmetics

I had a problem the other day with JS' inaccurate arithmetics. One can also stumble onto the same problem with LS:
Dim first As Double, sec As Double, third As Double
first = 0.1
sec = 0.2
third = 0.3

Print ( first + sec ) = third
'// -> False

I got a little bit curious, and lurked around in the documentation for other number data types in LS. Lo and behold, there's Currency. It seems to have some inbuilt functionality that corrects the inaccuracy in data types like Double.
Dim first As Currency, sec As Currency, third As Currency
first = 0.1
sec = 0.2
third = 0.3

Print ( first + sec ) = third
'// -> True
Nice to know if you need accurate results when doing arithmetics in LS.

1 comments:

Anonymous said...

but that has always been the case for floating-point arithmetics in computers. The currency model multiplies the values by 100, truncates them, then stores them internally as integers.

Andrew Magerman