An observation - Difference between two array values not consistent


Need your help in finding why the computed results (please refer the values circled in Red and Green under Column titled xPdiMdiDiff, from the attached .JPG file) are coming up inconsistently in the Exploration.

What I have done ?
I have used the index NIFTY with IEOD Periodicity of One hour for this purpose. I have used "exp(array)" function of AFL and simply tried to put up the difference between two array values. I have attached a .JPG file with my comments, AFL code snippet and the results from the Exploration.

Additional Information
FYI - I have also tried with RSI(13) vs RSI(21) and could see the results calculated and coming up in Exploration, properly and as expected.

Sorry, but despite your attempts to describe the problem you are STILL not saying:
a) what do you expect?
b) what you get instead (what difference you observe)
c) why you think it is "wrong"

For detailed instructions see: How to ask a good question

What I GUESS (based on apparent attempt to display results with 20+ digits) is you are simply victim of false impression that computer hardware has infinite precision. That is not the case.

You need to carefully read the Knowledge Base about hardware limits of your computer:

I would like to reply as below.

a) I expect the value as a result of performing subtraction to come up right in Exploration.

b) I did not get the subtracted value as expected (as at now noticed it only for PDI(9) and MDI(9) arrays). As per the attached image, the subtracted value shows up good till some digits only, it did appear to me that was the limitation.

c) As a result of digits limitation encountered, without assuming things further, I also did try with RSI(13) vs RSI(21) array with exp(array) function and performed the subtraction. The results came up as expected for this use case. I have pasted below the AFL code snippet. Please also refer the attachment for this use case.

Filter = 1; 
xRsi13 = exp(RSI(13));
xRsi21 = exp(RSI(21));
xRsiDiff = xRsi13 - xRsi21;
AddColumn(xRsi13, "xRsi13");
AddColumn(xRsi21, "xRsi21");
AddColumn(xRsiDiff, "xRsiDiff");

Hence posted this query seeking to get assistance as to why the subtracted value is appearing inconsistently for the use case with PDI(9) vs MDI(9) arrays.

I understand your point that relying on computer hardware for precision values in digits is to be avoided. But programming languages (especially Functional programming languages like R and Python, Java 8 and above) have come a long way to prove otherwise that now exploit the underlying hardware more efficiently.

Let me know if you still have any questions.


The values are 100% as expected given precision defined by IEEE floating point standard and hardware floating point unit that all computers use. It has absolutely nothing to do with programming language.

You did not read the article carefully enough (in fact you did not even click on link - I know because clicks on links are counted). Read it again slowly and carefully and repeat until you understand it (especially parts about precision and significant digits). It is all described there in great deal of detail and there is nothing more to add.

You need to LEARN a lot first. Among other things learn that precision (number of significant digits) of floating point representation on ALL computers is limited. Attempting to format numbers with 30+ digits as you do is absurd because they are meaningless. 32-bit IEEE standard precision gives guarantee of 7 significant decimal digits. They are accurate. Anything beyond that is subject to finite representation and rounding/truncation on binary level. READ THE ARTICLE.

There is also forum search that gives you relevant info:

1 Like