Receiving misaligned price data when using SetForeign()

If I execute the following AFL code with symbol VIX.XO loaded in the chart window, I get what I expect which is the close for today and the close for yesterday. However, if I load any other symbol to the chart window than VIX.XO, say SPY for example, then it returns yesterday’s close as the close for today and the day before yesterday’s close as the close for Ref(close, -1).

The last 3 day’s of VIX.XO daily closes is:

4/23 $17.10
4/22 $18.63
4/21 $17.20

The data displayed in the chart for VIX.XO is correct, see the attached file.

This is the AFL code I am executing:

SetForeign("VIX.XO");
_TRACE("!CLEAR!");
_TRACEF("VIX.XO Close = %g", C) );
_TRACEF("VIX.XO REF(Close, -1) = %g", Ref(C, -1) );
RestorePriceArrays();

This is the trace output when VIX.XO is loaded in the chart window, which is correct, see the attached file.

VIX.XO Close = 17.10
VIX.XO REF(Close, -1) = 18.63

This is the trace output when SPY is loaded in the chart window, which is incorrect, see the attached file.

VIX.XO Close = 18.63
VIX.XO REF(Close, -1) = 17.20

VIX problem

It isn't incorrect.

SPY has to have actual data for April 23rd to show data for April 23rd and 22nd.
Take a look at title of chart or take a look in quote editor to see what is last timestamp for SPY.
If it doesn't have data for April 23rd then there won't be output of price data of VIX for that date if SPY is selected symbol. So update price data of SPY then.

If SPY has data for all listed dates of your post then you have selected different bar other than last bar of chart. Trace outputs data of selected bar. Click on blank right margin area or on price axis to get data of last bar.

The SPY data is/was up to date. Additionally, I did not have a bar selected at the time. When I do select the bar for today in the SPY chart, the trace data for VIX.XO is still incorrect, as described before. Please see the attached files. Which show the SPY data from the Quote Editor and the chart of SPY with the last bar selected (today's bar) and the trace data for VIX.XO showing the wrong data.

SPY price data
SPY chart with VIX trace data

For completeness, see the quote editor data for symbol VIX.XO attached.

VIX quote editor data

You did not say that you output daily TF of intraday data.

So take a look at the time it shows for SPY in the chart title and take look at the time it shows for VIX.XO in chart title (if that latter one is selected symbol).

If datetimes do not fit (not showing same date and same time) then if SPY being selected symbol _Trace* will not show same output as if VIX is selected symbol.

It is case of missing data.

As written in manual:

  • 1 - default value - missing data bar OHLC fields are all filled using previous bar Close and volume is set to zero.

https://www.amibroker.com/guide/afl/setforeign.html


Set to intraday time frame and try again

SetForeign("VIX.XO");
_TRACE("!CLEAR!");
dailyClose = TimeFrameGetPrice("C", inDaily);
prev_dailyClose = TimeFrameGetPrice("C", inDaily, -1);
_TRACEF("VIX.XO Close = %g", SelectedValue(dailyClose));
_TRACEF("VIX.XO REF(Close, -1) = %g", SelectedValue(prev_dailyClose));
RestorePriceArrays();

Thank you for pointing out the different start time for the intraday day data for the VIX.

THERE IS NO DATA MISSING FOR THE VIX.XO (VIX) , since the VIX and VIX related securities do not have high frequency data like stocks. The data on the VIX index is updated every 15 seconds by the CBOE, 4 beats per minute, and is behaving exactly as it should.

This means that the first daily 1-minute bar will always start at 9:31 AM EST for the VIX and VIX related issues.

It appears that there is a deficiency with the SetForeign/Foreign commands in handling this special case for the CBOE VIX and VIX related securities data.

For this special situation, it doesn’t make sense to backfill the data since all the intraday data provided by the CBOE exchange exists for this symbol.

I did try the suggested fix you provided. It did not resolve the problem. I still get the same result, prior day close instead of current/selected day close even when selecting the current bar in the chart.

There is missing data....
Because as you wrote yourself

But SPY starts at 09:30 If you carefully look at your chart title!
-> Does it show 09:31 AM? I don't think so:


On selected intraday chart timeframe (as I wrote)? Did you?
Note: Intraday TF (1min, 5min,....Hourly,..) is not equal to daily TF.

I need this to work for daily timeframe, not intraday.

All the data provided by the CBOE for this symbol exists in the database, so there is not data missing. I don't want to get into a semantic argument about this.

It appears the problem is due to the fact that VIX data is not high frequency data like stocks and the first 1-minute data provide by the CBOE/IQfeed starts at 9:31. Is it possible that having the database settings base time interval set to 1-minute is part of the problem? Should I set it to tick or 1-second for real time data?

In comparison to SPY data the 09:30 bar is missing in your VIX.TO symbol. It is a technical fact. I was not talking about what CBOE is sending.

What you may do is setting to "End bar of interval" in Tools - Preferences - Intraday. Then the time component will disappear in daily TF and it should work then when daily TF is selected interval.

23

Nevertheless keep in mind that show "Start time of interval" is recommended setting.

Setting the End time interval as you suggested did not fix the issue. It did change the behavior, now both the close and prior close are equal to 18.63, using the code you provided below:

SetForeign("VIX.XO");
_TRACE("!CLEAR!");
dailyClose = TimeFrameGetPrice("C", inDaily);
prev_dailyClose = TimeFrameGetPrice("C", inDaily, -1);
_TRACEF("VIX.XO Close = %g", SelectedValue(dailyClose));
_TRACEF("VIX.XO REF(Close, -1) = %g", SelectedValue(prev_dailyClose));
RestorePriceArrays();

The time at the top of the chart for VIX.XO remains 9:31:00 after changing to END time interval in Preferences.

On my end time component disappears if setting to "End time of interval" and being on daily time frame.
I checked it on local DB.

Reason for showing time on your end may depend on selected data plugin.


The code I posed was supposed to run in intraday TF. Not daily.

I tried your suggestion to change the time stamp of compressed intraday bars to “END of time of interval,” and that changes the behavior, but it does not resolve the problem.

I reached out to IQfeed technical support and they reached out to the CBOE and they confirmed that VIX data disseminates at 8:31 CT, here is their direct quote:

"The VIX index disseminates at 8:31 CT. This allows time for the SPX options that are used for it's calculation to quote and calculate the VIX index value."

The way AmiBroker Foreign commands deal with symbols whose data disseminates from the exchange after the open and not at the open has some flaws. One remedy would be to back fill the 1-minute bar with the closing 1-minute bar from the prior day, but I won’t presume to know the implications of taking this approach.

It would be great if this could fixed, so that I may use the value of the $VIX in indicators.

You completely mix up things again.
I did not say to use intraday code with END time of interval setting.

Also don't use that posted intraday code. It is suppose to retrieve daily bar data of VIX of same day when SPY is selected symbol and if intraday interval is selected. But you do not want to use intraday time frame so do not use that time frame functions code.

Also do not set to End time of interval anymore. I was saying that it was tested on local DB. And there time component is removed with that setting if daily interval is selected. But since you use data plugin don't use it that way.


Anyway to bring this to an end.

Create composite of VIX via Analysis-Scan.
(I will not explain how to use analysis and scan as you just have to read manual and articles and forum.)

So I just tell the main part.

Save this code as AFL

if ( Status("action") == actionScan ) {
	if ( Status( "stocknum" ) == 0 )	{
		StaticVarRemove( "~VIX_daily" );
	} 
	StaticVarAdd( "~VIX_daily", C );
	Buy = 0; 
}

// Used in other environments in addition to Scan
VIX = StaticVarGet("~VIX_daily");

_TRACE("!CLEAR!");
_TRACEF("VIX.XO Close = %g", VIX);
_TRACEF("VIX.XO REF(Close, -1) = %g", Ref(VIX, -1));

Then apply that code to Analysis and setting to Daily periodicity as well as enable Pad&align and insert SPY as reference symbol. Click OK. Select VIX.XO as analysis symbol. Set Apply to: Current, Range: All quotes and run Scan.

22

Then after Scan being finished apply that same AFL to chart.
If SPY is selected symbol it should show VIX data of same day as of SPY.

As aside: Analysis can be saved as project file. Project file can be run via batch.

23


BTW, There does not need to be fixed anything in regards to Foreign functions.


As aside posted time frame functions code was not quite correct. Rather this way.

SetForeign("VIX.XO");

TimeFrameSet(inDaily);
dailyClose = C;
prev_dailyClose = Ref(Close, -1);
TimeFrameRestore();
expandmode = expandFirst;
dailyClose = TimeFrameExpand(dailyClose,inDaily,expandmode );
prev_dailyClose = TimeFrameExpand(prev_dailyClose,inDaily,expandmode);

_TRACE("!CLEAR!");
_TRACEF("VIX.XO Close = %g", SelectedValue(dailyClose));
_TRACEF("VIX.XO REF(Close, -1) = %g", SelectedValue(prev_dailyClose));

But don't use it as you want to apply Daily Chart Interval.

3 Likes

You misunderstood me, I did not use intraday code with END time of interval setting. Please read more carefully. I used that setting with daily time frame, as you said, and it did not behave the way you said it would in that mode.

Please understand that I want to use the daily VIX data in a custom indicator for all symbols, not just SPY. If you go to the original comment, I made it clear that SPY was just being used as an example, but the problem occurs with all symbols.

While I appreciate the cleverness of the solution you presented, it is a work around that does not do what I need. If the Foreign commands could handle this situation, then we would not be having this discussion.

The problem is simple: All the data for the VIX that was sent by the CBOE exists for each day, but the Foreign command will not give the current day's close for the VIX because a single 1-minute bar that is not disseminated by CBOE is interpreted as a hole by the Foreign command. Instead of the Foreign commands fixup process replacing the 1-minute bar with the previous 1-minute bar, it replaces the entire daily bar with the previous daily bar. This is a flaw, because the data for the current day is complete, accurate and an Open, High, Low, Close exists and should be accessible.

As @fxshrat explained you many times, it works as documented and it is correct way of doing things. Past data of selected interval are used to fix data holes, as documented. Plain and simple.

The software that operates as documented is flawless.

Using future data would be an error (future leak).

It completely does not matter what you think "it should" be doing. These are just your assumptions that are incorrect. What you are having is your DATA problem. Properly designed program works the same, documented way for all symbols. Program must not work differently "just this time" only because one particular user has opinion and one particular symbol has missing data.

You have been given a number of solutions already. In addition to that you can define day session starting from 9:31 in the settings and turn on filtering for day session (instead of 24 hours trading):

And by the way: if you change Preferences setting, you have to APPLY changes (press the button). If timestamp did not change, it means you did not apply the changes.

1 Like