Questions about multiple timeframes and futures data

PeterD · February 3, 2022, 3:19am

I have a one-second IQFeed DB, and am using multiple timeframes in my charts, to pull some EMAs from higher timeframes for plotting. I am probably doing something wrong, or missed an important sentence in the docs, or have an incorrect setting.

In the charts below, the blue and red lines are EMAs in the timeframe of that chart, while the light blue and pink dashed lines are EMAs from higher timeframes (light blue is one TF higher and pink is two TFs higher). When I say one or two TFs higher, I have a function that returns the number of seconds in the higher TF according to this way of looking at TFs.

I should be able to verify that the code is pulling the correct values just by looking at the prices labels, however I can see that incorrect values are being returned. For example, in the 5-second chart (top left):

the light blue number 4534.39 (TF1EMA21) should agree with the 15-second chart's blue number of 4534.33 (EMA21)
the pink number 4534.38 (TF2EMA5) should agree with the 1-minute chart's red number of 4534.17 (EMA5)

Another issue is that the 4-hour chart does not have any value for the pink number (TF2EMA5).

Here is the chart code:

// Higher TF test

function NextTF( Seconds )
{
	// Returns the interval one timeframe higher than the one passed to the function, in seconds.
	
	switch( Seconds )
	{
		case 1:				Output = 5; break;
		case 5:				Output = 15; break;
		case 15:			Output = 60; break;
		case 60:			Output = 300; break;
		case 300:			Output = 900; break;
		case 900:			Output = 3600; break;
		case 3600:			Output = 14400; break;
		case 14400:			Output = 86400; break;
		case inDaily:		Output = inWeekly; break;
		case inWeekly:		Output = inMonthly; break;
		case inMonthly:		Output = inQuarterly; break;
		case inQuarterly:	Output = inYearly; break;
		default:			Output = Null; break;
	}

	return Output;
}

TF0		= Interval(0);
TF1 	= NextTF( TF0 );
TF2 	= NextTF( TF1 );
EMA5 	= EMA( C, 5 );
EMA21 	= EMA( C, 21 );

if( TF1 > TF0 )
{
	TimeFrameSet( TF1 );
	TF1EMA21 = EMA( C, 21 );
	TimeFrameRestore();
	TF1EMA21 = TimeFrameExpand( TF1EMA21, TF1 );
}
else TF1EMA21 = Null;

if( TF2 > TF0 )
{
	TimeFrameSet( TF2 );
	TF2EMA5	= EMA( C, 5 );
	TimeFrameRestore();
	TF2EMA5	= TimeFrameExpand( TF2EMA5, TF2 );
}
else TF2EMA5 = Null;

// CHARTING
SetChartOptions( 2, chartShowArrows | chartShowDates | chartWrapTitle );
LowerStyle 	= styleLine | styleThick;
HigherStyle	= styleStaircase | styleDashed | styleThick;
//Plot( C, "Close", colorDefault, styleCandle | styleNoTitle, Null, Null, Null, 0 ); 
Plot( EMA5, "EMA5", colorRed, LowerStyle, Null, Null, Null, 1 );
Plot( EMA21, "EMA21", colorBlue, LowerStyle, Null, Null, Null, 1 );
Plot( TF1EMA21, "TF1EMA21", ColorBlend( colorBlue, colorWhite, 0.5 ), HigherStyle, Null, Null, Null, 1 );
Plot( TF2EMA5, "TF2EMA5", ColorBlend( colorRed, colorWhite, 0.5 ), HigherStyle, Null, Null, Null, 1 );

And here are some relevant settings:

I must have a blind spot here since I haven't found my error, and I appreciate any help you all can provide.

Peter

fxshrat · February 3, 2022, 8:15am

It is pulling correct numbers!
It has been mentioned several times in forum that there are three expansion modes.
Have you read documentation of all functions completely?

https://www.amibroker.com/guide/afl/timeframeexpand.html

Available modes:

expandLast - the compressed value is expanded starting from last bar within given period (so for example weekly close/high/low is available on Friday's bar)

expandFirst - the compressed value is expanded starting from first bar within given period (so for example weekly open is available from Monday's bar)

expandPoint - the resulting array gets not empty values only for the last bar within given period (all remaining bars are Null (empty)).

Disable QuickAFL and load more bars.

PeterD · February 3, 2022, 12:14pm

Hi fxshrat,

Thank you for your help.

I am now using expandFirst (default was expandLast). I was surprised to see that expandFirst provided the values that I am looking for on most of the intraday and some of the higher TF charts, so I must not yet fully understand the TF compression and expansion. I don't understand why the first value of the close of the higher TF would be the one that matches the last value in the chart, so I will think about that more.

However,

the 4-hour chart still has no value for TF2EMA5 (weekly TF), although I added SetBarsRequired( sbrAll, sbrAll ) at the top of the code to disable QuickAFL
the 4-hour chart value for TF1EMA21 doesn't match the daily TF
the weekly chart value for TF1EMA21 doesn't match the monthly TF
the weekly chart value for TF2EMA5 doesn't match the quarterly TF

There must be something else. Perhaps something related to the mixed EOD/intraday data (this is how IQFeed provides the data)? Or perhaps the number of seconds for the inWeekly and inMonthly constants are not appropriate for futures data since futures trade 6 days/wk?

There is just still something not right here.

Here's revised code and screenshot (which is probably too small to read).

// Higher TF test

SetBarsRequired( sbrAll, sbrAll );

function NextTF( Seconds )
{
	// Returns the interval one timeframe higher than the one passed to the function, in seconds.
	
	switch( Seconds )
	{
		case 1:				Output = 5; break;
		case 5:				Output = 15; break;
		case 15:			Output = 60; break;
		case 60:			Output = 300; break;
		case 300:			Output = 900; break;
		case 900:			Output = 3600; break;
		case 3600:			Output = 14400; break;
		case 14400:			Output = 86400; break;
		case inDaily:		Output = inWeekly; break;
		case inWeekly:		Output = inMonthly; break;
		case inMonthly:		Output = inQuarterly; break;
		case inQuarterly:	Output = inYearly; break;
		default:			Output = Null; break;
	}

	return Output;
}

TF0		= Interval(0);
TF1 	= NextTF( TF0 );
TF2 	= NextTF( TF1 );
EMA5 	= EMA( C, 5 );
EMA21 	= EMA( C, 21 );

if( TF1 > TF0 )
{
	TimeFrameSet( TF1 );
	TF1EMA21 = EMA( C, 21 );
	TimeFrameRestore();
	TF1EMA21 = TimeFrameExpand( TF1EMA21, TF1, expandFirst );
}
else TF1EMA21 = Null;

if( TF2 > TF0 )
{
	TimeFrameSet( TF2 );
	TF2EMA5	= EMA( C, 5 );
	TimeFrameRestore();
	TF2EMA5	= TimeFrameExpand( TF2EMA5, TF2, expandFirst );
}
else TF2EMA5 = Null;

// CHARTING
SetChartOptions( 2, chartShowArrows | chartShowDates | chartWrapTitle );
LowerStyle 	= styleLine | styleThick;
HigherStyle	= styleStaircase | styleDashed | styleThick;
//Plot( C, "Close", colorDefault, styleCandle | styleNoTitle, Null, Null, Null, 0 ); 
Plot( EMA5, "EMA5", colorRed, LowerStyle, Null, Null, Null, 1 );
Plot( EMA21, "EMA21", colorBlue, LowerStyle, Null, Null, Null, 1 );
Plot( TF1EMA21, "TF1EMA21", ColorBlend( colorBlue, colorWhite, 0.5 ), HigherStyle, Null, Null, Null, 1 );
Plot( TF2EMA5, "TF2EMA5", ColorBlend( colorRed, colorWhite, 0.5 ), HigherStyle, Null, Null, Null, 1 );

Tomasz · February 3, 2022, 12:23pm

To get better understanding of what is happening in your code and how functions work, use advice given here: How do I debug my formula?

fxshrat · February 3, 2022, 1:17pm

Take a look at the scroll bar on 4 hour chart and at the scroll bar of daily chart.
There is clearly data missing where 4-Hour interval is chosen. So there is not enough data to plot weekly EMA on 4-Hour data length. That's why it remains NULL.

As for difference of daily EMA on 4 Hour.
Set start time of Data base settings - Intraday settings to even full hour such as 08:00.

As for weekly to monthly, quarterly... see here

Tomasz · February 3, 2022, 2:39pm

The thing is that in "mixed" mode, EOD data records are only used if selected chart interval is daily or higher. If selected chart interval is intraday, then INTRADAY data are used, including all eventual compression to higher time frames, see:

If you want WEEKLY chart, you have to open another window with Weekly interval selected. Or it can be daily and compressed to weekly.

PeterD · February 3, 2022, 3:08pm

@fxshrat, your explanation of why data from weekly and monthly TFs may seem strange makes sense now, thank you for sharing that post!

And you nailed it on the 4-hour chart: not enough data! @Tomasz explained it perfectly, and this is new information for me (very helpful):

The intraday data only goes back to Jan 7th. Although IQFeed allows 6 months of tick data (which is used for 1-second bars), I didn't realize that my DB does not go back far enough for the 5-week EMA to calculate, because I assumed that just pressing "Force backfill" would be enough. Bad assumption on my part.

In DB settings, I previously used 1000000 bars which is about 34 days due to the warning that pops up saying to use an integer between 1 and 1000000. But to calculate a 5-week EMA, I need 35 days.

So I just changed it to 2000000 and after dismissing the warning, pressed Cancel.

I was surprised to see that the 2M number seems to be in use in spite of the warning (this may be a bad idea; the warning is there for a reason).

Anyway here are the quotes in my DB, along with the performance monitor:

Before using 2M bars, I was watching old bars disappear from the history. Now they are being retained as new ones are added.

I'll have to test this more when the market closes (IQFeed restricts backfill history to 7 or 8 days during regular NYSE market hours).

Many thanks for all your help!

system · May 14, 2022, 3:08pm

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.