Export to csv not working with 1 minute data

AFL Programming
1

Hello,

1 year ago I used the code from knowledge database to export minute data to csv file (see below). it used to work well on EUR.USD for instance. The 1 minutes database is generated and populated with IB pulgin.

Now it doesn’t work anymore. AFL never stops and the file is not created. 2 strange points:

  • If I run the same AFL on the EOD database it works.
  • If I run just an AFL creating and closing a file without collecting data it doesn’t work (see code below).

Any idea ?

Thank you

  1. Limited code just creating a file
fh = fopen( "DataFor"+Name()+".csv", "w" );
if( fh )
{
  fputs("Symbol,Date,Time,Open,High,Low,Close,OpenInt,Volume\n", fh );
  fclose( fh ); // MUST CLOSE FILES !!!!
}

Buy=Sell=0; // for scan
Filter = Status("lastbarinrange");
AddTextColumn("Export done", "Status");
  1. Complete expport code as per knowledge database
 
n = Name(); 
dn = DateNum(); 
tn = TimeNum(); 

// until 2016 March 11th 6:59am 
exportFilter = ( n == "Symbol 1" AND dn <= 1160311 AND tn <= 65900 ) OR 
               ( n == "Symbol 2" AND dn >= 1160311 AND tn > 65900 ); 

fh = fopen( "DataFor" + Name() + ".csv", "w" ); 

if( fh ) 
{ 
    fputs( "Symbol,Date,Time,Open,High,Low,Close,OpenInt,Volume\\n", fh ); 


    for( i = 0; i < BarCount; i++ ) 
    { 
        if( exportFilter[ i ] ) 
        { 
            Line = Name() + 
          StrFormat( ",%06.0f,%06.0f,%g,%g,%g,%g,%g,%g\\n", 
           dn[ i ] % 1000000, 
           tn[ i ], 
           Open[ i ], 
           High[ i ], 
           Low[ i ], 
           Close[ i ], 
           OpenInt[ i ], 
           Volume[ i ] ); 

            fputs( Line, fh ); 
        } 
    } 

    fclose( fh ); 
} 

Buy = Sell = 0; // for scan 
Filter = Status( "lastbarinrange" ); 
AddTextColumn( "Export done", "Status" );
2

The code will only work if you actually have 1-minute data within selected from-to range
and if periodicity is set to 1-minute
Use

Buy=Status("lastbarinrange");

So you can see if there are any bars within selected range.

3

Hi Tomasz,

I have data in selected range. I used both “all quotes” and “from to” for just 1 month. Periodicity is set to 1-minute.
I changed the buy as per your indication, same results.

I even just ran this line of code with nothing else, the “Scan” is stuck at 99% and doesn’t end.

4

It is not stuck here. If you are using some old version - upgrade to most recent.
Even if it appears “stuck” it is not “stuck”. It has ended. And the files should be there. Just check the files.

5

I’m using version 6.10.
No file is created at all, I checked.

6

If no files are created it means that you installation is messed up (no write access to current working directory). You need to run FULL setup of AmiBroker again.

A hint for the future: you should send screenshots of Analysis window instead of describing. Screenshots show more information.

7

The installation has write access, as the scrip work with the eod database.
I have 2 database:

  • First one is EOD data filled in with manual csv upload => Export scripts works well on this one. File properly created.
  • Second database is 1 minute data filled in with IB plugin realtime.

On top I tried on 2 different computers, same behavior. Tricky one :))

8

Uncheck Wait for backfill option

1 Like
9

That’s it. Wonderful !!!

Thank you so much.

10

Beware that without wait for backfill enable you might use data not updated.
You might want to do a simple exploration, with wait for backfill enable and try to find which symbol is preventing exploration to end

11

Ok, so I think that we need big red screaming text right before users face “You have selected wait for backfill option so AmiBroker is doing precisely what you told it to do — Waiting For Backfill” For hundreds/thousands of symbols backfill usually means downloading mega/gigabytes of data.

1 Like
12

Well, if you allow me:

  • I did not check “wait for backfill” in the new install. It was there by default. Still sometimes I need it.
  • There is no backfill during weekend (and just 1 symbol of IB 1 minutes is not GB of data) :slight_smile: :slight_smile:
  • I don’t use the extract to csv regularly. Just when I want to do more in depth data analysis or play around with them. So indeed I forgot to uncheck it this time (As mentioned I used the script before). It happens

Maybe it might help mindless people like me, to modify the script in the knowledge database and add a comment saying:
“You have selected wait for backfill option so AmiBroker is doing precisely what you told it to do — Waiting For Backfill” For hundreds/thousands of symbols backfill usually means downloading mega/gigabytes of data."
:slight_smile: :slight_smile:

13

For what it is worth: Wait for backfill is not turned on by default.

14

Hi I am new to AB and would like to run the code that tomj posted in this post; would someone be kind and write the steps for me to run the above code to get a csv file from the the intraday db. Why does he wrote two separate codes. Can they be combined into a single script? Also, the date and time format seem odd; can someone point me to a link on how explaining why the are written that way? I would appreciate your help. Thanks, Paolo

15

Hi,

There is just one code to use which is the “complete” one, as per the knowledge database.
While using the code, you have 2 options:

  • Either use the export filter, which means you will have to hard code the name of the assets you want to export and the date.
    or
  • (best) you set exportfilter to 1 (or remove all reference to it) and use the standard filter of Amibroker. Like creating a watch list and exploring this watch list between 2 dates.
16

Hi Tomj, do you recall the title of the KB or provide me the link. Thanks for your fast reply…

17

http://www.amibroker.com/kb/2006/03/04/how-to-export-quotations-from-amibroker-to-csv-file/