Computers work with binary numbers, not decimals, 0.1 is infinite fraction

[Maxis]

Hello, while looking for solutions for rounding, i came up to this post.
So as Tomasz mentioned in his first post of the thread, the 3 functions can do any kind of rounding, just by properly using divide and multiply on those 3 functions as they are basically operating on the integer part of the number.

Q1. Any DIRECT function to do the rounding on the decimal part of a number?

Q2. In the AddCloumn function, the format parameter says "By default all variables are displayed with 2 decimal digits". If that's the case, then 1.2 & 1.5 should work the same as Prec(X,2) & Prec(X,5) respectively. But i noticed the format parameter to be working as rounding to the decimal value of the input. So a 1.2 format value for 125.1294 should return 125.12. Rather it is returning 125.13. Kindly someone shed some light.

Q3. Just to get my head around a peculiar problem am facing, i wrote a simple code to see what it returns.
a=45239.451236587; ab = a*100; abc = int(ab); abcd = abc/100; Filter = 1; AddColumn(a,"a",1.9); AddColumn(ab,"ab",1.9); AddColumn(abc,"abc",1.9); AddColumn(abcd,"abcd",1.9);

Except for the abc, the rest 3 columns are returning unexpected values.
a ab abc abcd
45239.453125000 4523945.500000000 4523945.000000000 45239.44922

am starting to wonder if i am losing it.

Please guide. Thank You.

[nsm51]

32 bit floating point has been mentioned many times. Your "a" starts out of the valid number range.

AmiBroker Knowledge Base » About floating point arithmetic

[Tomasz]

@Maxis - The original thread was about INTEGERS, you are asking about decimal fractions.

You are making several mistakes.

To avoid mistakes you really need to read the manual.
Prec() function TRUNCATES (not rounds) numbers. Truncation is sometimes called rounding DOWN (although this term is not correct).

AddColumn() on the other hand performs formatting. Formatting is a printf-like formatting (%.2f) and it uses IEEE754 rounding. It is "nearest, ties to even", see: https://en.wikipedia.org/wiki/IEEE_754#Roundings_to_nearest

So Prec( 0.1294, 2) will truncate last 2 digits (and you will get 0.12) while formatting AddColumn( 0.1294, "text", 1.2 ) will round last 2 digits (and you will get 0.13).

Second important thing is that computers do NOT work with decimal numbers. They work with binary.
I wish they taught binary arithmetic in schools but they don't. And the result is that 99.999% of people completely don't understand how math works in computers.

Each function including Prec() works with binary numbers.

What Prec() does it truncates (rounds down) to nearest representable binary of desired decimal fraction.

Even if you round them to "decimals" they are still binary numbers and subject to binary (IEEE standard) representation that is used by ALL computers. Certain fractions (such as 0.1 and 0.01) are infinite in binary representation, like 1/3 is infinite in decimal representation. Other fractions like (0.5, 0.25, 0.125, 0.0625... and combinations of them) have finite binary representation because they are negative powers of 2.

Infinite fractions (such as 1/3 in decimal system) can not be represented accurately.

What you see on screen is result of conversion (formatting) of binary number to decimal string. What you see is not actual number (which is binary) but result of formatting.
In IEEE754 worldwide standard such formatting makes sense only if you display significant digits. There are 7 significant digits offered by IEEE754 single precision.
So formatting 1.9 that you use makes no sense (exceeds significant digits)

You have to READ this slowly and carefully:

AmiBroker Knowledge Base » About floating point arithmetic

Here is something for you. Take Microsoft Excel and do the exercise as shown in the video:

1. In cell A1 enter -1
2. In cell A2 enter formula =A1+0.1
3. Copy-paste cell A2 to subsequent rows
4. You expect to get -0.9, -0.8, -0.7, -0.6, ...., -0.2, -0.1, 0, ...
5. Look at the surprising value that you get in A11. It should be zero, but it isn't. It is 1.3878E-16

0.1 and zillions of other fractions are infinite (and as such inaccurate) in binary floating point regardless of precision and you would end up always with issues like that.

(Video shows Microsoft Excel in action)

[Maxis]

OK. Thanks for the inputs people.
sure i will read and try to understand both the given references.

@Tomasz you are right Sir. First, maths is not something i am very good at, and second, "I wish they taught binary arithmetic in schools but they don't. And the result is that 99.999% of people completely don't understand how math works in computers."
i will read up on this whole subject, as u say "slowly & carefuly" multiple times and try to get my head around it.

"Thank You" once again for reassuring that all is not lost with me,

Regards,