Amibroker Export Capabilities

Presale / Trial
,
#1

Hi Everyone,

I am new and terribly impressed with Amibroker. Outside all of the outstanding features, I have a specific use case on data export / programmatic integrations in mind and am looking to see if Amibroker has these capabilities.

My attempts at researching programmatic export / API in this forum have given me some directional guidance, however as I am just starting I'm learning quite a bit. Would appreciate any help and your patience!

Scenario Core Requirements
If I run an analysis of say 200 technical indicators on 500 stocks for the past 10 years on EOD data, am I able to export these historical results/time series for the 500 symbols in an automated or externally initiated manner?

Additional Scenario Questions

  • I understand the above would be through OLE automation, is that correct? Is it also correct that I could control Amiquote through OLE Automation (say some scheduled PowerShell or Python script), have it update and download new EOD data, and use OLE in AmiBroker to perform the above analysis and export to a flat file?
  • Which data formats could I use for an export? Is CSV supported? Would this be to one file for all securities or multiple files for each security?
  • Corollary to that - is it possible to CRUD the exports directly into a database? Or would that be additional code / OLE process steps?
  • In terms of periodicity - say if I change this to 5 minute interval data received live via feed instead of EOD for say 20 stocks with 30 technical indicators, would I be able to export the incremental tick updates and calculations in a programmatic manner and on an ongoing basis too?

Appreciate your help in advance. Thank you.

#2

@pengo, welcome!

I understand the above would be through OLE automation, is that correct? Is it also correct that I could control Amiquote through OLE Automation (say some scheduled PowerShell or Python script), have it update and download new EOD data, and use OLE in AmiBroker to perform the above analysis and export to a flat file?

Yes. Depending on your needs and programming skills, you can use various techniques to achieve your goal.

Importing data in Amibroker is usually achieved by downloading and importing quotes in AB using Amiquote (that you can control via OLE).
Alternatively, directly importing quotes from ASCII .csv files (again via OLE), after you get them via some scripts (Python, R, JavaScript, etc.).

You can write an exploration, save it to a project (.apx) and execute the project as needed. This can be done via OLE or easily via the "Batch" functionality of Amibroker (.abb files).
Keep in mind that the resulting file will HUGE (200 indicators * 10 years * 252 trading days * 500 stocks - ~1.25M rows).
In the exploration, you should limit the names of the columns to approx. 20 chars to see them all in in the report listview.

I did a test on my not too powerful PC (with a loop that dynamically creates 200 WMA) on SP500 stock, and the exploration took a bit over a minute, plus another half a minute to save the file to disk (a slow non SD disk).

Which data formats could I use for an export? Is CSV supported? Would this be to one file for all securities or multiple files for each security?

Using an exploration, the default export file format is .csv (there is an option to save as .html, but I will not use it). It is a single file (below the file exported from my test, created via AB "batch"):

image

(A file that may be too big for many standard applications....)

Corollary to that - is it possible to CRUD the exports directly into a database? Or would that be additional code / OLE process steps?

There is an AFL plugin that can be used to execute custom SQL queries to any OBDC compatible database, but in case, I will not employ it.
In general, communication with an external database is slow.
I suggest using some scripts to get data from the flat files exported and update the required rows in your DB (I will probably write an extra column with a sort of checksum to speed up this kind of operations).

I think that you need to create the huge file ONCE, import it in a DB, then use a different batch to export each day only the new data and append it to the existing database.

In terms of periodicity - say if I change this to 5 minute interval data received live via feed instead of EOD for say 20 stocks with 30 technical indicators, would I be able to export the incremental tick updates and calculations in a programmatic manner and on an ongoing basis too?

This request is not clear to me. You refer to a 5-minutes bar and then to incremental ticks.
Do you want to update all the 30 indicators for the 20 stocks at each tick (while the 5-Mbar is still forming) or only when the bar is complete?
Anyway, since I DO NOT use intraday RT data, I leave this answer to other more experienced users.
It would be helpful to know also what RT data provider do you plan to use.

7 Likes
#3

Late to this thread - I use AB/AQ as my primary EOD database. I would like to access the AB database data without overwriting it. I was thinking of trying to access the data in the actual database directory, but it seems more reasonable to access it via an exploration, where I can export the Ticker, Date, OHLC, volume, and perhaps a few indicators Amibroker calculates as a addColumn with some filtering or watchlist restriction before hand.

Am I correct in understanding the steps are:

  1. Create Exploration and save in .apx format
  2. Write .abb batch function to:
    2a. Load database and exploration
    2b. Run Exploration
    2c. Export Exploration data as .csv
  3. Run batch file

Anything I'm leaving out here?