How to optimize backfill with IQFeed

Hi,

I have a python script downloading 750 instruments from IQFeed (1 month history of 1 minute data) , store data in database, compute statistics and update instrument with latest data and statistics. This take about 25-30 seconds. So I can scan this list every few minutes.

On the same list of instrument, if I do a backfill of the data to explore in AB, it takes 13 minutes. How could I optimize it to get similar time ?

Stephane

Reducing the number of data to 1000 doenst look to improve and AB is repetitively backfilling the data.

Is there a way to quickly keep data updated and then having explore working on data from database ?

I don't use IQFeed, so I might not be able to help you, but that seems very strange. Have you enabled Local data storage in Database settings?

Przechwytywanie

First/initial Scan/Exploration might take a lot of time but when it is completed and the database is updated, next scans should be performed much more quickly.

You might also be interested in some issues raised in this thread:

And there is a special IQFeed category on the forum. You might find a possible solution here:

http://forum.amibroker.com/c/data/iqfeed

Regards

Thanks @Milosz. Yes I do have the local data storage activated, but it seems to be refetching data every time I scan. I have activated option ‘wait for backfill’ but I think it will only get new data. And as I sent to support I have got also a weird behavior on the last bar, so maybe there is something wrong with my setup.

Thanks for the links I will give them a look now.

1 Like

Registered (purchased) version of AmiBroker does NOT refetch data that was already downloaded before. But for that to work database saving must be enabled.

@traderlabs - we don’t have your email in registered users list.

Saving is disabled in TRIAL / unregistered edition.

The only way to “optimize” it is to buy the program.

And you also need to make sure that you have run FULL SETUP of AmiBroker instead of copying files from somewhere. If you fail to use full setup, your database may be READ ONLY (i.e. blocked by Windows User Access Control “feature” from writing).