Mixed EOD/Intraday data & ODBC Plugin

I am creating from own historical database in Postgres and using the ODBC plugin to connect it to Amibroker.

From what I have understood about "Allow Mixed EOD/Intraday data" option, that if it is checked, a database can contain both intraday & EOD data. And when we select EOD timeframe, then Amibroker uses the EOD data points to display chart or run AFL, instead of re-sampling the intraday data. This increases efficiency.

To use this feature, I am time-stamping EOD & Intraday data in Postgres as follows:

intraday data section

2019-01-01 09:15, ..., ...
2019-01-01 09:16, ..., ...
2019-01-01 09:17, ..., ...
...

last of intraday data point

2019-01-01 03:30, ..., ...

this is the EOD data point stored in the same table

2019-01-01 00:00, ..., ...

My question: Is this the correct way to store, so that I can use the aforementioned feature of AmiBroker which directly picks up EOD data if present in the database?

On displaying chart, I can downsample & upsample data in timeframe. But I don't know, if it is happening because AmiBroker is re-sampling the intraday data or using the EOD data stored.

Maybe @Tomasz can confirm is this the right way to use ODBC plugin.

Can anyone confirm if they have experince with this?

I am noticing that backtest now is extremely slow when using data imported froom ODBC plugin. And this is just for 5 years worth of data. Amibroker hangs until backtest is completed and takes about ~2 minutes to complete.

After enabling local data storage, i turned the data source to "local database". As per Docs, it should speed up loading by loading it from amibroker db itself as described here.

Config:
image

Amibroker Local Database size(D:\ProgramData\ABPostgres): 2.5 GB

My guess for why so much time is taken in backtest is because when I run a backtest on EOD database, amibroker uses the intraday data to re-sample it to EOD instead of using the EOD data I have already saved in the database using the format described in the first post.

Question is: How to resolve this issue and speed up backtest?

I request @Tomasz or someone knowledgeable to drop in here because I don't see a resolution myself...

Would be greatly appreciated. Thanks.

Snapshop from quote editor:

image

Notice the highlighted row, it is an EOD row but Amibroker is not recognizing it as such, is it because of the time part(00:00)?

Apologies for making multiple posts but I can only post max 1 image/post.

Here is the format of my postgres data table:

image

As we can see, the Date column(dt) uses the "timestamp" data type of Postgres which automatically converts date only value to date time(appending 00:00). Since I am using the same table to store both EOD and intraday, I have to use the timestamp data type.

I’m not sure mixed EOD/intraday was designed for what your thinking. When I used IQFeed for instance , it allowed the user to request say 10 years of intraday if that was the max available for intraday lookback, but then they offered way more EOD data then 10 years so it would request EOD backfill past your bars requested.

So basically your dataset would be a bunch of EOD then would be followed by intraday and then the database would grow daily in intraday.

Believe this was made for a couple of the data vendors as it’s common to have short intraday backfill but large intraday.

I bet if you changed your database settings to show 24 hours, you’d see that your EOD import would show as a trade at midnight.

Could be wrong for your method. I know when I used IQFeed I ran two instances of Amibroker , one EOD and one intraday. Mostly because IQFeed doesn’t split adjust intraday data so was more of a quick way to see if it was a massive gap up or down or just an upcoming split I didn’t notice.

1 Like

Carefully Read the Users Guide:
http://www.amibroker.com/guide/x_performance.html

I am using Windows 10 64 bit with 16GB Ram and i7 processor.
Since I am working off intraday data, I have set the cache size to 6GB as I have noticed considerable speed up with this amount of RAM set.

When I run a backtest, it hangs at 100%. After manually stopping it, the trade list and results are generated. But I also see that the custom backtest procedures which is used to publish custom stats is not executed(not displayed in report).

Here is screenshot of settings and the hung analysis:

Untitled

I am unable to diagnose this issue. What is going wrong?

The issue(getting stuck at 100%) persists for even as simple AFL as this:

Buy = Short = Sell = Cover = 0;


Buy = 1;
Sell = 1;

I have permanently deleted all symbols and added them again. But this issue still persists. What should I do now?

UPDATE:

I re-deleted all the symbols and added them. The backtesting seems to work fine now.
I don't know what has changed. The settings are same as before. If there any more issues, I would let it be known here.

1 Like