SetForeign vs Foreign behaviour with different time frames

[LinusVanPelt]

I've spent two days pulling my hair out trying to track down this issue, and finally put together the below test code.

The hair pulling part is in the "D3" section, which was what my actual code was doing.

Can you explain why the D3 section doesn't work, and what the hell is ROCShortD3 and ROCLongD3 actually returning?

To be clear, I've got this working (just don't code "D3", use one of the other approaches), but I'd like to get a better understanding.

I'm particularly interested in rows where ROCShort and ROCLong are both negative. See 2018-12-31 or 2020-03-31.

I have read this KB article: https://www.amibroker.com/kb/2014/10/20/foreign-timeframeset/ I admit I was using Foreign() instead of SetForeign() in my code, but the KB article did not explain the different behaviour between the two, i.e. why SetForeign() must be before TimeFrameSet(), while Foreign() must be after TimeFrameSet().

// See https://www.amibroker.com/kb/2014/10/20/foreign-timeframeset/
ShortPeriod	= 1;
LongPeriod	= 5;

// Current chart symbol should be SPY, Daily periodicity
// Either walk this through the debugger or run an Explore with *Current SPY

// Monthly ROC (Works) (Remember current symbol is SPY)
TimeFrameSet(inMonthly);
	ROCShortM				= ROC(C,ShortPeriod);
	ROCLongM				= ROC(C,LongPeriod);
TimeFrameRestore();

// Convert to Daily
ROCShortD1 = TimeFrameExpand(ROCShortM,inMonthly,expandLast);
ROCLongD1  = TimeFrameExpand(ROCLongM,inMonthly,expandLast);

// Use the SetForeign function (Works)
SetForeign("SPY"); 

TimeFrameSet(inMonthly);
	ROCShortM				= ROC(C,ShortPeriod);
	ROCLongM				= ROC(C,LongPeriod);
TimeFrameRestore();

ROCShortD2 = TimeFrameExpand(ROCShortM,inMonthly,expandLast);
ROCLongD2  = TimeFrameExpand(ROCLongM,inMonthly,expandLast);

// Use the Foreign function (DOES NOT WORK)
cSymbol = Foreign("SPY","C");

TimeFrameSet(inMonthly);
	ROCShortM				= ROC(cSymbol,ShortPeriod);
	ROCLongM				= ROC(cSymbol,LongPeriod);
TimeFrameRestore();

ROCShortD3 = TimeFrameExpand(ROCShortM,inMonthly,expandLast);
ROCLongD3  = TimeFrameExpand(ROCLongM,inMonthly,expandLast);

// Use the Foreign function *inside* the TimeFrameSet (Works)
TimeFrameSet(inMonthly);
	cSymbol 				= Foreign("SPY","C");
	ROCShortM				= ROC(cSymbol,ShortPeriod);
	ROCLongM				= ROC(cSymbol,LongPeriod);
TimeFrameRestore();

ROCShortD4 = TimeFrameExpand(ROCShortM,inMonthly,expandLast);
ROCLongD4  = TimeFrameExpand(ROCLongM,inMonthly,expandLast);

IsEOM = TimeFrameExpand(1,inMonthly,expandPoint);

if (Status("action") == actionExplore) 
{ 

	//Filter = 1;
	Filter = IsEOM;
    //Filter = ROCShortD < 0 AND ROCLongD < 0;
    //Filter = ROCShortM < 0 AND ROCLongM < 0;

    AddColumn(Close,"Close",1.2);
    AddColumn(ROCShortM,"ROCShortM",1.2);
    AddColumn(ROCLongM,"ROCLongM",1.2);
    AddColumn(ROCShortD1,"ROCShortD1",1.2);
    AddColumn(ROCLongD1,"ROCLongD1",1.2);
    AddColumn(ROCShortD2,"ROCShortD2",1.2);
    AddColumn(ROCLongD2,"ROCLongD2",1.2);
    AddColumn(ROCShortD3,"ROCShortD3",1.2);
    AddColumn(ROCLongD3,"ROCLongD3",1.2);
    AddColumn(ROCShortD4,"ROCShortD4",1.2);
    AddColumn(ROCLongD4,"ROCLongD4",1.2);

    //SetSortColumns(4,5);
} 

[fxshrat]

@LinusVanPelt

// Use the Foreign function *inside* the TimeFrameSet (Works)
TimeFrameSet(inMonthly);
	cSymbol 				= Foreign("SPY","C");
	ROCShortM				= ROC(cSymbol,ShortPeriod);
	ROCLongM				= ROC(cSymbol,LongPeriod);
TimeFrameRestore();

ROCShortD4 = TimeFrameExpand(ROCShortM,inMonthly,expandLast);
ROCLongD4  = TimeFrameExpand(ROCLongM,inMonthly,expandLast);

No, that one does not work the same as using SetForeign.
Both of your ways of using Foreign function with TimeFrameSet are not correct (meaning you won't get same results as with using SetForeign as shown in KB article).

Foreign function internally restores price arrays.
And as been written in KB restoring price arrays is equivalent to using TimeFrameRestore.
Also if using TimeFrameSet then the variables you want to compress have to be declared after TimeFrameSet.

The only way to compress Foreign is to use TimeFrameCompress as shown in second example of the KB article you have linked to. The only two ways are the ones of the KB artivle. So why do you want to do it differently?

Also see this example where compressed index data (of ^GSPC) is calculated with compressed data of selected symbol.

Timeframe setup

This has been posted several times in forum already. Do not use Foreign within Timeframeset. First and formost... there is also Knowledgebase Article https://www.amibroker.com/kb/2014/10/20/foreign-timeframeset/ As newbie entire AmiBroker KB is mandatory read before coming to forum. Compress Foreign symbol before and then use it in TimeFrameSet with selected symbol's array for further calculation(s) _SECTION_BEGIN("RSC Mansfield FIJO EXPANDED"); period=52; spx = Foreign("^GSPC","C"); spx_…

So there you can use spx_weekly inside TimeFrameset also because it has already been compressed to same interval inWeekly before (and there is no restore applied after spx_weekly declaration and before TimeFrameset).

[LinusVanPelt]

@fxshrat

No, that one does not work the same as using SetForeign.

Ok, perhaps that does not work. It merely returns the exact same values as D1 and D2.

@fxshrat

Foreign function internally restores price arrays.

AFL Function Reference - FOREIGN. No mention of Foreign internally restoring price arrays.

The KB article states that TimeFrameRestore() and RestorePriceArrays() restore price arrays.

@fxshrat

And as been written in KB restoring price arrays is equivalent to using TimeFrameRestore.

Nope that's not what it says. It says that TimeFrameRestore() and RestorePriceArrays() restore price arrays. No mention that Foreign() restores price arrays. Until your post I have not read that the Foreign() function restores price arrays, so thanks for that.

@fxshrat

Also if using TimeFrameSet then the variables you want to compress have to be declared after TimeFrameSet.

Isn't that what I'm doing? I want the ROC calculated on monthly periodicity, which are derived after TimeFrameSet(). I then need that expanded to the original daily periodicity, which again is what I'm doing. I need you to elaborate on this, or are you merely explaining the process rather than correcting an error?

@fxshrat

The only way to compress Foreign is to use TimeFrameCompress as shown in second example of the KB article you have linked to.

Yep, you're right. Ok so:

SetForeign()
TimeFrameSet()

TimeFrameRestore() (or RestorePriceArrays() )
TimeFrameExpand(,period,expandLast) to expand the stuff to daily (assuming daily periodicity)

OR...

x = Foreign(Symbol,"C")
y = TimeFrameCompress(x, periodicity)
using y
z = TimeFrameExpand(y, periodicity) to expand the stuff to daily

don't assume that SetForeign() == Foreign(), and don't deviate from these examples. Got it.

@fxshrat

So why do you want to do it differently?

Because the KB did not make it clear (to me anyway) that Foreign(Symbol,"C") could not be used where SetForeign() is used in the first example.

https://www.amibroker.com/guide/afl/foreign.html
https://www.amibroker.com/guide/afl/setforeign.html

Now I know. Thanks for your post.

[Tomasz]

The only supported way to mix timeframe functions with foreign data access is described in the Knowledge Base:

Time compression of data retrieved from another symbol

The article lists TWO methods:

1. SetForeign/TimeFrameSet
2. Foreign/TimeFrameCompress

NOTHING ELSE is supported.

Manual and KB for obvious reasons shows only ALLOWED ways to use the program. The number of ways any program and any function can be abused is infinite and one can not list them all. Example:

// WRONG WAYS to use printf() - small sample of infinite set
printf("%a"); // wrong
printf("%b"); // wrong
printf("%c"); // wrong
printf("\z"); // wrong
printf("%g", "test"); // wrong
printf("%s", 12); // wrong
// endless number of examples of wrong usage can be given