I am currently exploring using the IB data plug in to receive forex quotes. Last night before bed, I backfillled the data via the plugin for these guys....
Before going to bed, my charts looked like this...
This morning when I woke up, all my charts look like this....
I'm trying to understand what has happened and how I can find out. Is backfilling something that has to be done on a continuous basis, or, is there a way to keep the charts continuously updating data?
Is there an advantage to using an RT plug in over, say, making periodic REST API batch calls for data and repeatedly importing them via ASCII every hour?
@SpandexMan what happens is that IB, when the market is closed, sends a -1 quote.
Check the offending symbols in the "Quote Editor":
You should avoid backfilling when the market is closed and/or delete incorrect quotes before entering new live data.
Alternatively you should write some code to "fix" your data before using/plotting it.
In this case it's pretty straightforward since you know what "bad" values to look for.
For a more general approach to dealing with outliers, search the forum (a similar issue has been discussed here, by the way).
Ah, perfect. That's great @beppe, I'll check that out. Thanks.
This was a useful insight into the issue I was having. Since that time, I have tried various solutions and workaround with varing degrees of success, but if I leave the system unattended for periods of time, thje IB plug in seems to spew out -1 in the OHLCV fields which, as you can imagine in real time, is a barrier to being able to trade effectively intraday.
I tried my hand at writing a .dll last night to get a better approach (as the data issue is the major sticking point I am finding in the algo trading) but I failed spectacularly and I can see that I'd have to spend another 200 hours learning C++ to implement that properly, having learned 'conversational Python' (actually, I probably know enough Python to order a beer and ask about the weather, but its enough to get by). I feel like Sysiphus sometimes.
Anyway, I'm really thinking of pivoting into using periodic rest API calls (or even continuously updating a csv using websockets and running local database and doing periodic ASCII imports. Of course, I don't get RT quotes then but I think I can probably live with that since my current approach is based on close price in any case. Is approaching things this way ineffective? It seems straightforward enough which is making me wonder why I'm investing so much into RT data.
It would be easier to just use IQFeed instead. Or Polygon.io if you don't care about real time quotes.
@SpandexMan let me check if I can provide option to filter out those -1 inside the plugin. I hope to get you the answer in few days.
Having said that, I fully agree with @PeterD. IQFeed is so much better than IB that it is absolutely worth trying.
Thanks Tomasz, that's legendary.