Interactive Brokers and Norgate

Would one use Norgate in addition to Amibroker and Interactive Brokers or is Interactive Brokers similar to Norgate? And are these recommended. Or which are recommended services?

Having a little trouble because I'm reading the "Introduction to Amibroker book, which I believe came out in 2012 (I don't think there's a newer book), while looking at information today so if you help clear my confusion that would be awesome.

Thanks,

@Mi6secret,

Norgate is a Data supplier (End of Day only as far as I understand).
IB is a Broker. As a broker they allow you to get data (snapshot - not quite real time) of a different timeframe - like 1 hour or 1 minute.
AmiBroker is (The BEST) Analysis software. It allows you to use the data (from either Norgate or IB - or both) for your own analysis. The analysis can be charts, scans, explorations, or back tests.

Hope this helps

1 Like

@Mi6secret,

I use all three, in separate database files.

I view Norgate as the "golden record" of daily prices. Also, Norgate has done an outstanding job at categorization of symbols which is not available anywhere else. I use these categorizations in my assessment of market condition.

Norgate has what I consider to be the most accurate prices. Definitely superior to Yahoo, I may add. It's not even close in my opinion. Here's an example from 2018, where Norgate is on the left (good data) and Yahoo is on the right (bad data because Yahoo didn't account for a split in December 2018 correctly; they have fixed it since then I think).

image

Norgate also populates the most recent split dates and split ratio fields in AB, which I will get back to in a moment.

I use IB for intraday data. Backfill is very slow. But since IB is my broker, the data is cheap. Unless you pay extra, you are limited to 100 symbols concurrently for streaming data.

IB does not always adjust backfilled data for splits correctly. They generally get it right but not always. I run a scan in Norgate to identify which symbols split recently, so I can go and verify that IB correctly handled the split. If they didn't I already have the split ratio and can use AB to adjust for it.

I hope this helps.

1 Like

Thank you @PeterD
Can you provide more detail concerning how you work with Norgate and IB through automation? I'm trying to wrap my head around if IB gives you intraday and Norgate gives you daily and the data at the end of the day could differ between the two...and I'm automating my trades through IB...How does this work? For example, do the two reconcile automatically and choose who's right and adjust the trades or do you, manually end of day, go in and reconcile and manually open/close/short/cover.

I would also agree with Peter wholeheartedly about the quality and usefulness of Norgate data. Additionally, as I'm not sure it was mentioned, you can choose different levels of corporate adjustments and the charts will immediately update (by a simple toggle) by going into database settings and selecting split adj, split and Dividend Adjusted etc. A small script should also allow to plot those at the bottom of the chart like most charting packages support.

An optional subscription is the delisted securities which in addition to allowing proper survivorship testing, also allows you to do cool things like this...

After I read a book (older books like Darvas or Jesse Livermore) you can follow along by pulling up the chart of the security he's talking about. I've even marked up a chart he was explaining etc. Here is an example.

Be safe out there.
Sean

DarvasExample

Hi @Mi6Secret,

It would be great if there was some automatic reconciling between the databases, but this would be a manual effort.

I just go and look at the charts in my IB database for bad data that stick out like a sore thumb.

If you're going to just use daily data for your trading decisions, then you can just use a Norgate database and send orders to IB, and I think it's reasonable to assume that Norgate has accurate data.

But if you're going to trade intraday, then you would use your IB database and you will have to keep it maintained and do your quality checking yourself.

In a perfect world, someone would develop software that connects to multiple data sources, e.g. IB, IQFeed, TDA, IEX, etc., performs some magical analysis to determine what is the "golden record" of price data, and then publishes this data feed for AmiBroker (or any application, like Excel) to use in real-time.

But I don't have the budget or skills to do this and don't know of anyone who has created something like it. And it would be very complicated to manage.

So to keep it simple I just use IB for intraday data and look for obvious problems, while assuming the data is dirty. And I use Norgate for the "golden record" of daily data.

Anyone have better suggestions?

If one trades futures one may consider CSI data. They used to have very reliable EOD data (combining several sources and correcting by hand) and their software/data was compatible with Amibroker. I can't comment on recent versions though.
http://www.csidata.com/?page_id=1229

There is one problem with CSI data exporter. They export data in YYMMDD format (TWO digit year) and that is not really forward looking and causes problems for dates >2020