IQFeed 6.2 Level 1 corrections & data spikes

IQFeed's unfiltered data has many spikes in all symbols, all day long. I'm challenged by these and would prefer a data feed that doesn't have the spikes.

For example, here is SPY from unfiltered IQFeed, before backfilling with corrected data (Force Backfill makes the spikes go away):

Here is SPY from Interactive Brokers, without backfilling:

The IQFeed spikes are so frequent that the data is just not usable for live trading because I have to Force Backfill on all symbols every few minutes, manually. It's a full-time job just keeping the data clean.

However, IQFeed's API has a feature that started in v6.2:

IQFeed will now send Trade Correction messages on the Level 1 port when watching a symbol that can be used for correcting historical data.

Is it possible that the data plugin could use these corrections to compensate for the unfiltered spikes that occur, so that I don't have to manually Force Backfill 50 times a day? Or is there another solution?

Bad ticks are a filtering problem and fixed by excluding certain execution sources and specific encoded or timestamped messages (see below). I'm guessing when the IQFeed says "Unfiltered", they mean completely unfiltered. I've also noticed, that even with a clean feed, there are numerous bad ticks during pre and post sessions. I don't think you can apply your own custom filtering unless you were getting the RAW exchange messages. Too bad quote providers can't take an xml file from us to then apply our specific filters we would like from their quote and set for our account etc. No doubt not a feasible idea, technically speaking.

TAQ (Trade and Quote) specifications snip-it...

Here's what I mean. AB shows bad spikes but IQFeed's chart app does not. That's what I want to see in AB.

Here's AB:

Here's IQFeed's Chart app:

Somehow the IQFeed Chart app knows how to remove the spikes. I would like the AB plugin to do the same. Am I the only one?

First backfill (what you see on "IQFeed Chart") is NOT the same as streaming data.
Easily when you "force backfill" you would get what you see from "IQFeed Chart".

Data for backfill and streaming are coming from DIFFERENT end-points, different API calls and everything about them is different.

Second thing is that plugin ALREADY FILTERS out bad ticks from BACKFILLED data (it is in the "Config" page ATR based filter). The filter however is not applied to streaming data because for it to work it needs both past AND future data (to detect spike it needs data surrounding spike).

Third thing is that IQFeed API docs are essentially NON-EXISTING and all that IQFeed really says their feed is "unfiltered" and that's it ... deal with that.

So plugin TRANSPARENTLY passes everything it receives from IQFeed. In the absence of docs I am not really keen to play guessing games about which tick is good and which it isn't.

You pay to IQFEED for the data, not to me. So DEMAND from IQFeed to deliver clean data.

It is obligation of DATA VENDOR who charges you money to deliver quality data.
It is NOT obligation of me to figure out which data bits are wrong.

IQFeed API should have an option "give me the clear data, I don't want your unfiltered bad ticks".

For some reason eSignal DOES NOT SEND those spikes, neither do Interactive Brokers, so there are vendors who can send clean stream.

2 Likes

Hi Tomasz,

Thank you so much for your insights. I don't know anything about IQFeed's documentation so I'll reach out to them. I agree, there should be a way to get a filtered feed like eSignal and IB.