Strange SetForeign() behaviour

Hello,

I am noticing some strange behaviour in my coding.
I try to calculate the 20 period atr value of a list off 28 forex symbols from a higher timeframe.
The strange thing is, it is caluated correctly only for 10 symbols, the other 18 symbols get really high values.
This is the code i use, it is run on a 5 minute chart:

for( n = 0; ( Ticker = StrExtract( TickerList, n ) ) != ""; n++ )
{
SetForeign( Ticker );
TimeFrameSet(inDaily);
atr20[n] = LastValue(ATR(20));
TimeframeRestore();

atr20exp[n] = LastValue(TimeFrameExpand(atr20[n],inDaily));
}

I have checked the realtime quote window and all quotes for all forex symbols are there.
I really don't get it why only 10 symbols are calculated correctly and the other 18 symbols give strange high values ( around 29 million).
I use Amibroker Pro 6.20.1 64 bit version.
Hope you could shed some light and point me in the right direction.

strange results do often occur due to User Error.

Somebody has to call this function after the work is done with the Foreign symbols RestorePriceArrays()
https://www.amibroker.com/guide/afl/restorepricearrays.html

Hi travick,

Thanks for your reaction.
As i read in the bottom of the link you send:
TimeFrameRestore and RestorePriceArrays
is essentially the same function. So please note that calling RestorePriceArrays also resets the time interval set by eventual previous call to TimeFrameSet

Also in this link they describe exactly as i did it:
http://www.amibroker.com/kb/2014/10/20/foreign-timeframeset/

Either way, it does not work either when i use restorepricearrays()

That's what happens when one "forgets" to mention what they've already tried in their opening post.

Since you didn't, some one will ask. By convention, I prefer to use Setforeign() and RestorePriceArrays() just to be sure.

atr20[n] = …

This is a single element of an array.
How do you pass that to a function that expects an Array?

TimeFrameExpand(atr20[n],inDaily)

Did you check this part?
I'd rather

atr20 = ATR(20);
…  TimeFrameExpand( atr20, inDaily ) ...

No point calling LastValue() twice anyway, right?

Sorry, you are right about that, i tried to be complete but i guess there was some info missing.
I have to get out now, but will be back soon and comment on the second part.

Also, having holes or having unaligned timestamps at times will cause unexpected behaviour behaviour.

Are you sure that all symbols are aligned correctly ?

Can You post a TRACE() for the last few bars of the base symbol and the Foreign one where you have problems.
Run it from a loop with / without TimeFrameSet().
For eg. If the Foreign symbol has future data relative to the base symbol, its a problem.

Are all the symbols from the same Market timings ?

Below is a piece of a trace of a ticker that does calculate the atr20

|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:54.59|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:54.62|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:54.65|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:54.69|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:54.72|
|0.007890581|Formulas\Custom\Price 53310.afl|217|21|19:50:54.76|
|0.007890581|Formulas\Custom\Price 53310.afl|217|21|19:50:54.80|
|0.007890581|Formulas\Custom\Price 53310.afl|217|21|19:50:54.84|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:55.05|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:55.08|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:50:55.13|

and below a trace of another ticker that does wrong calculation (very high value, this time 15000 not millions)

|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:08.87|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:08.91|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:08.94|
|15026.65|Formulas\Custom\Price 53310.afl|217|21|19:57:08.97|
|15026.65|Formulas\Custom\Price 53310.afl|217|21|19:57:09.01|
|15026.65|Formulas\Custom\Price 53310.afl|217|21|19:57:09.06|
|15026.65|Formulas\Custom\Price 53310.afl|217|21|19:57:09.10|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:09.29|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:09.33|
|{EMPTY}|Formulas\Custom\Price 53310.afl|217|21|19:57:09.36|

This is a reaction to the post travick made above about not being able to pass single element to array.

I realize there is another piece of information missing.
The code posted above maybe is more clear if i add this line on top of it:

Tickerlist = CategoryGetSymbols(categoryWatchlist, 2);

Now you can see that it itterates through a watchlist off tickers.
Each [n] is another ticker, that's why i coded atr20[n] = ...
How to pass this to a function that expects an array is a good question.
Maybe it cannot, i'm not sure, I thought it would fill the array with all equal numbers (lastvalue(atr(20))).
If i did like you then how would i store the values for each ticker? use staticvars?

Tickerlist = CategoryGetSymbols(categoryWatchlist, 0);
for( n = 0; ( Ticker = StrExtract( TickerList, n ) ) != ""; n++ )
{
SetForeign( Ticker );
// TimeFrameSet(inDaily);
atr20 = ATR(5);
RestorePriceArrays();

atr20exp[n] = LastValue( TimeFrameExpand( atr20, inDaily) );

_TRACEF("%s=%.2f", Ticker, atr20exp[n] );	
}

Above is fine for my current TF.
uncommenting this // TimeFrameSet(inDaily);

is also fine but some erratic values appear where there are data holes.

2 Likes

Thx travick, crystalclear, will try this out tommorow when the market is open again.

Hi travick,

here's the trace without Timframeset:

|AUD.CAD-IDEALPRO-CASH=0.00033|Formulas\Custom\Price 53310.afl|219|10|20:10:02.65|
|AUD.CHF-IDEALPRO-CASH=0.00025|Formulas\Custom\Price 53310.afl|219|10|20:10:02.68|
|AUD.JPY-IDEALPRO-CASH=0.03321|Formulas\Custom\Price 53310.afl|219|10|20:10:02.72|
|AUD.NZD-IDEALPRO-CASH=0.00027|Formulas\Custom\Price 53310.afl|219|10|20:10:02.75|
|AUD.USD-IDEALPRO-CASH=0.00025|Formulas\Custom\Price 53310.afl|219|10|20:10:02.79|
|CAD.CHF-IDEALPRO-CASH=0.00029|Formulas\Custom\Price 53310.afl|219|10|20:10:02.82|
|CAD.JPY-IDEALPRO-CASH=0.03516|Formulas\Custom\Price 53310.afl|219|10|20:10:02.85|
|CHF.JPY-IDEALPRO-CASH=0.03483|Formulas\Custom\Price 53310.afl|219|10|20:10:02.88|
|EUR.AUD-IDEALPRO-CASH=0.00054|Formulas\Custom\Price 53310.afl|219|10|20:10:02.92|
|EUR.CAD-IDEALPRO-CASH=0.00056|Formulas\Custom\Price 53310.afl|219|10|20:10:02.95|
|EUR.CHF-IDEALPRO-CASH=0.00025|Formulas\Custom\Price 53310.afl|219|10|20:10:02.98|
|EUR.GBP-IDEALPRO-CASH=0.00029|Formulas\Custom\Price 53310.afl|219|10|20:10:03.01|
|EUR.JPY-IDEALPRO-CASH=0.04582|Formulas\Custom\Price 53310.afl|219|10|20:10:03.05|
|EUR.NZD-IDEALPRO-CASH=0.00056|Formulas\Custom\Price 53310.afl|219|10|20:10:03.07|
|EUR.USD-IDEALPRO-CASH=0.00037|Formulas\Custom\Price 53310.afl|219|10|20:10:03.09|
|GBP.AUD-IDEALPRO-CASH=0.00075|Formulas\Custom\Price 53310.afl|219|10|20:10:03.12|
|GBP.CAD-IDEALPRO-CASH=0.00076|Formulas\Custom\Price 53310.afl|219|10|20:10:03.14|
|GBP.CHF-IDEALPRO-CASH=0.00048|Formulas\Custom\Price 53310.afl|219|10|20:10:03.17|
|GBP.JPY-IDEALPRO-CASH=0.05981|Formulas\Custom\Price 53310.afl|219|10|20:10:03.21|
|GBP.NZD-IDEALPRO-CASH=0.00074|Formulas\Custom\Price 53310.afl|219|10|20:10:03.24|
|GBP.USD-IDEALPRO-CASH=0.00050|Formulas\Custom\Price 53310.afl|219|10|20:10:03.27|
|NZD.CAD-IDEALPRO-CASH=0.00029|Formulas\Custom\Price 53310.afl|219|10|20:10:03.30|
|NZD.CHF-IDEALPRO-CASH=0.00021|Formulas\Custom\Price 53310.afl|219|10|20:10:03.33|
|NZD.JPY-IDEALPRO-CASH=4.27385|Formulas\Custom\Price 53310.afl|219|10|20:10:03.36|
|NZD.USD-IDEALPRO-CASH=0.00019|Formulas\Custom\Price 53310.afl|219|10|20:10:03.39|
|USD.CAD-IDEALPRO-CASH=0.00041|Formulas\Custom\Price 53310.afl|219|10|20:10:03.42|
|USD.CHF-IDEALPRO-CASH=0.00027|Formulas\Custom\Price 53310.afl|219|10|20:10:03.46|
|USD.JPY-IDEALPRO-CASH=0.02540|Formulas\Custom\Price 53310.afl|219|10|20:10:03.49|

and here it is with Timeframeset daily:

|AUD.CAD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:21.99|
|AUD.CHF-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.02|
|AUD.JPY-IDEALPRO-CASH=0.81482|Formulas\Custom\Price 53310.afl|219|10|20:12:22.06|
|AUD.NZD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.09|
|AUD.USD-IDEALPRO-CASH=0.00607|Formulas\Custom\Price 53310.afl|219|10|20:12:22.13|
|CAD.CHF-IDEALPRO-CASH=34141.65234|Formulas\Custom\Price 53310.afl|219|10|20:12:22.17|
|CAD.JPY-IDEALPRO-CASH=30813.60938|Formulas\Custom\Price 53310.afl|219|10|20:12:22.20|
|CHF.JPY-IDEALPRO-CASH=30813.56641|Formulas\Custom\Price 53310.afl|219|10|20:12:22.23|
|EUR.AUD-IDEALPRO-CASH=0.01104|Formulas\Custom\Price 53310.afl|219|10|20:12:22.27|
|EUR.CAD-IDEALPRO-CASH=22.28209|Formulas\Custom\Price 53310.afl|219|10|20:12:22.31|
|EUR.CHF-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.35|
|EUR.GBP-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.38|
|EUR.JPY-IDEALPRO-CASH=1.05341|Formulas\Custom\Price 53310.afl|219|10|20:12:22.42|
|EUR.NZD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.46|
|EUR.USD-IDEALPRO-CASH=0.00782|Formulas\Custom\Price 53310.afl|219|10|20:12:22.48|
|GBP.AUD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.51|
|GBP.CAD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.54|
|GBP.CHF-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.58|
|GBP.JPY-IDEALPRO-CASH=1.40026|Formulas\Custom\Price 53310.afl|219|10|20:12:22.62|
|GBP.NZD-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.65|
|GBP.USD-IDEALPRO-CASH=0.01109|Formulas\Custom\Price 53310.afl|219|10|20:12:22.69|
|NZD.CAD-IDEALPRO-CASH=34141.65234|Formulas\Custom\Price 53310.afl|219|10|20:12:22.74|
|NZD.CHF-IDEALPRO-CASH=30812.83984|Formulas\Custom\Price 53310.afl|219|10|20:12:22.77|
|NZD.JPY-IDEALPRO-CASH=15818.28613|Formulas\Custom\Price 53310.afl|219|10|20:12:22.80|
|NZD.USD-IDEALPRO-CASH=15817.52637|Formulas\Custom\Price 53310.afl|219|10|20:12:22.84|
|USD.CAD-IDEALPRO-CASH=0.00789|Formulas\Custom\Price 53310.afl|219|10|20:12:22.88|
|USD.CHF-IDEALPRO-CASH=0.00573|Formulas\Custom\Price 53310.afl|219|10|20:12:22.92|
|USD.JPY-IDEALPRO-CASH=0.67711|Formulas\Custom\Price 53310.afl|219|10|20:12:22.96|

There are some differences...
Other then the 5 minute - daily values offcourse.
I can't explain it.

Firstly, the codes in this thread make no sense to me:

  1. There is no point in using LastValue twice.
  2. There is no point in using TimeFrameExpand when you do LastValue on result

Secondly, debug your formula using techniques presented here: How do I debug my formula?

If you run codes like this from indicator (you are not saying that so guessing games here)
you must have data cache large enough to keep those 28 symbols (Tools->Preferences->Data). Also when compressing from 5 minute to daily you need at least 5760 (24 * 60 / 5 ) 5-minute bars to get 20 days worth of data required for ATR. So you might need to make sure that BarCount is actually higher, otherwise you won't be able to calculate ATR. Again, you should debug your formula as explained here: How do I debug my formula?
It is your code and your data. No one can debug your setup except yourself.

1 Like

Hi Tomasz, thanks for helping.

I know, the code has some weird stuff in it, i'm not a professional coder so all this is a lot off trail and error really.
I've got rid off the lastvalue errors, and trying hard to understand why I see these traces my code is producing.
Right now i don't get it, that's why i reached out to this forum, maybe somebody could point me in the right direction.
I've checked the data cache it says: 10000 max. symbols, 4096 max. MB and no limit on saved quotations.
Looks ok to me.
Also barcount exceeds 25000 5-minute bars on all symbols, should be ok too.

Offcourse I know that I need to do the debugging and i'm trying to do that, but on this one i'm a bit stuck.
I think i've reached the ceiling of my programming/analyzing skills :sob:.
If there are no further ideas what to do or to test i'll leave it here and stop developing this idea, that is no problem.

As explained in linked document, get the insight into what your formula is doing by displaying intermediate values, for example, put _TRACE or debug breakpoint on line

....
atr20 = ATR( 20 ); // put breakpoint in debugger or...
_TRACE("atr20 for " + Name() + " is " + LastValue( atr20 ) + "\n");
...

and use Watch window (or Log window) to see the values in atr20 variable (prior to any subsequent transformations)

1 Like

Post snapshots of data, i've mentioned it twice now.

Those erratic values are reproducible with Data holes or unaligned Trading times, you need to the entire range that is in computation.

1 Like

Ok, I have some work to do, I will get back when it is done, wich could take a while. (busy scedule lately)
Thx so far Tomasz.

travick I though I did post snapshots when i posted my trace output to this thread.

I hear you regarding these unaligned timestamps, but thing is I first have to figure out how to check this.
I will get back later.

In quote editor (Menu bar -> Symbol -> Quote Editor) check whether some of your symbols (resulting in millions values of ATR) have invalid DateTime entries with OHLC zero. (Me thinks you are using DDE plugin)

See example picture below

Delete those ones if they are there

019

If having some of those symbols in your watchlist being iterated then you would get exactly that.... million values in ATR.

129

If those invalid entries are not there then

150

1 Like