How to save IBKR real time data feed to a local database?

Hi all,

I am quite new here so I apologize in advance if my question is ignorant or irrelevant.
For the past several days I went through numerous forum topics, demo videos, FAQ, descriptions and so on but was not able to find any solution myself so far and therefore I decided to write in the forum for help.

I want to save every day 1 second chart data for 3-4 US stock symbols per day (they are different every day, not one and the same) for the time period from 7AM-9:30AM(pre-market session only). I want to create local database with this data for backtesting purposes later on.

I was thinking to get the charts/data from the Interactive Brokers plugin however Interactive brokers allow up to 2000 bars for the 1 second chart backfill, so basically I cannot backfill the whole pre-market session.
I am also not sure how to save the real time data from IB plugin even for the 2000 bars that I can backfill. Any help with that will be appreciated.

So my questions are:

  • is there a way to save locally real-time data feed from IB plugin for a specified symbol and time period?
  • if it is not possible to do that with IB data plugin, is it is possible to do the same thing with DTN IQfeed plugin? IQfeed allow up to 180 days 1 second backfill as per my chat with their sales.

Any links to similar topics, advises and help will be highly appreciated.
I am also looking for a paid service for strategy automation, so any recommendations on that will be truly appreciated.

Thank you!

Your user account isn't marked with License verified badge. Did you purchase AmiBroker license?

Only users with License verified badge are allowed to post on this forum.

So before posting, make sure you verify your license as explained here.

As to the answer: Trial / demo does NOT save data. Once you purchase the license, you don't need to do anything to save - AmIBroker automatically saves data it received from plugin when you exit the program if Local data storage is enabled in the database settings (that is the default setting)
Explained as always in the manual:

Thanks for reminding me, I have Pro version license and I just verified my forum username.

Welcome @v0rt3x,

Streaming data is automatically saved to an AMI broker database locally. No need to worry about that part.

But as you stated, you are only able to download around ~30 minutes of 1sec data with the AMI plugin. This is problematic in a few ways, the most common issue is losing data connection for over 30 minutes, and not getting a full backfill because of that -- so you will have gaps in your data.

I've attempted to solve this issue with a python script that I wrote. THIS SCRIPT IS UNSUPPORTED. Do not ask anyone about it but me, they can't/won't help you. :grin:

You can take the saved csv from this script and import it to AMI. It will download however much data you specify. But note that IB will charge you for making a ton of requests.

Other pitfalls with 1sec data, see my thread here.

Thanks for replying.
I downloaded the data from TradingView in csv and imported it to Amibroker.
It is just 3-4 symbols for 2 and 1/2 hours a day so it is not something enormous and as you say probably will save me from troubles with IB

If you are doing any backtesting/optimizing with TV data you may find that it's different from IB data. This could lead to issues. As a rule of thumb you want to trade on the same data you test with.

My second link highlights issues that arise even with the same data source, though.

Using TV data in AMI could be useful for optimizing, and then trading on TV itself. The benefit of this option is you've got a cloud service handing all the data. I'm assuming what's printed doesn't change, but I don't know what their policy actually is.

I just realized that some of the data is missing despite it is being printed on the charts

I see the pre-market session chart on the US stocks. When I try to export CSV for the pre-market session for specific symbol on the 1 second chart the data before 9:30 is missing and also the time-periods on the csv file is completely different from what I have on the chart opened. Example:

I wanted to export 1s chart for SPCB

from 8:00 AM on 3/19/21
to 9:30 AM on 3/19/21.

I loaded the chart and exported CSV with ISO time format.

However the downloaded file contains the data

from 2021-03-18 15:21:43Z
to 2021-03-19 23:59:58Z

excluding the data between these time stamps

2021-03-18T23:53:54Z
2021-03-19T09:30:03Z

the pre-market data for 19 th of March is missing

Opened a ticket with them to see what the problem is...
If that is something common TV becomes useless as a data source for exports...

Guessing it's local/exchange time issue

Data is missing in the original export CSV file, it's always exported with the exchange timestamps
I hope their support is able to resolve