No rounding issue?

rangeHigh = 0.67672;
rangeLow = 0.575675;
//part_001 = rangeHigh - rangeLow;
part_001 = 0.101045;
part_002 = 0.0002526125;
part_003 = part_001 / part_002;
part_004 = ceil( part_003 );

( using AmiBroker 6.39.1 x64 ) Hi everyone, I am a little confused here because when I execute this code with part_001 assigned explicitly with 0.101045, part_003 is equal to 400 exactly, but when I execute the code with part_001 assigned as the result of rangeHigh - rangeLow ( which is also == 0.101045 and is not a rounded value of any kind ) and I divide part_001 by part_002, part_003 results in 400.0001. This is a problem because ceil( part_003 ) then results in 401 and not 400. This causes some problems in my program. Am I missing something here ? I recalculated it on different calculators and the result should be 400 exactly and not 400.0001 Thanks a lot for a quick solution and explanation

@Neltum, please see this article: AmiBroker Knowledge Base » About floating point arithmetic

rangeHigh = 0.67672;
rangeLow = 0.575675;
part_000 = rangeHigh - rangeLow;
part_001 = 0.101045;
part_002 = 0.0002526125;
part_003 = part_000 / part_002;
part_004 = part_001 / part_002;
_TRACEF("0: %1.9f", part_000);
_TRACEF("1: %1.9f", part_001);
_TRACEF("2: %1.9f", part_002);
_TRACEF("3: %1.9f", part_003);
_TRACEF("4: %1.9f", part_004);

If you print/trace more decimals you'll see that the (part_000 and part_001) values are NOT the same.

Thanks for the quick response ((; I will read the article

Ceil is going to always give you integer LARGER than number (if any fractional part is present).
You can use round() instead.
Computers use binary, not decimal, you can't look at decimal representation and make conclusions, because such assumptions fail.

Yes thank you, much better understanding after having read the article ((:

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.