Export quotes from WL1 to Path1, WL2 to Path2 and so on

Hi. My forum search did not yield wanted result. Trying to export quotes data of symbols present in 20 watchlists to their corresponding folders.

For example,
WL1 should get exported to D://AmiData//DB//1//
WL2 should get exported to D://AmiData//DB//2//
and so on..

Please help or guide on the best way possible.

Hello vtriv,

to export the quotes you could start with study this KB:

https://www.amibroker.com/kb/2014/11/14/how-to-export-quotes-to-separate-text-files-per-symbol/

Best regards,
Peter

Hi Pietro, i have been exporting databases from a long time using this one:

/*
Export intraday and EOD data to TXT files 
One file for each stock
In the first line insert the directory you want to save them to, make sure the
directory exists
Select your charts to export with the "Apply to" filter in AA window 
Select the timeframe period you want to save as using the AA "Settings"
Press Scan button
by Graham Kavanagh 05 Feb 2004
*/
Buy = ( DateNum () >= ParamDate("Start Date", "2005-01-01") ) AND  ( DateNum() <= ParamDate("End Date", "2019-12-31") );

fh = fopen( "H:\\AmiExport\\"+Name()+".txt", "w"); 

if( fh ) 
{ 
   fputs( "Ticker,Date,Time,Open,High,Low,Close,Volume,OpenInt \n", fh ); 
   y = Year() % 100; //Year in format "yy"
   m = Month(); 
   d = Day(); 
   r = Hour();
   e = Minute();
   n = Second();
   
   for( i = 0; i < BarCount; i++ ) 
   { 
   if( Buy[i] )
   {
      fputs( Name() + "," , fh );
      ds = StrFormat("%02.0f-%02.0f-%02.0f,", 
                     y[ i ], m[ i ], d[ i ] ); 
      fputs( ds, fh ); 
     
      ts = StrFormat("%02.0f:%02.0f:%02.0f,", 
                     r[ i ],e[ i ],n[ i ] ); 
      fputs( ts, fh ); 

      qs = StrFormat("%.4f,%.4f,%.4f,%.4f,%.4f,%.0f\n", 
                     O[ i ],H[ i ],L[ i ],C[ i ],V[ i ] ,OpenInt[i]); 
      fputs( qs, fh ); 
   } 
   }
   fclose( fh ); 
} 

//Buy = 1;

But what i find difficult is how to export several watchlists' symbols to several different folders. Need some guidance in that direction.

Do you think i need to take this approach?

WL_List = "WL00,WL01,WL02,WL03,WL04,WL05,WL06,WL07,WL08,WL09,WL10,WL11,WL12,WL13,WL14,WL15,WL16,WL17,WL18"; 

Path_List = "Path00,Path01,Path02,Path03,Path04,Path05,Path06,Path07,Path08,Path09,Path10,Path11,Path12,Path13,Path14,Path15,Path16,Path17,Path18";

for( i = 0; ( wl = StrExtract( wl, i ) ) != ""; i++ )
{
  if(wl == WL00)
  {
	path = Path00;
  }
  else
	if(wl == WL00)
	{
		path = Path00;
  	}
		else
  	 if 
}

Hello vtriv,

it depends what your result should look like ...

I understand processing AB quotes export :

  1. Use Analysis - Scan or Exploration
  2. Setup your necessary filters/data scope/settings (select watchlist)
  3. Run Scan or Exploration from your AFL
  4. Check your Export quotes data
  5. Save your Scan or Exploration as .apx
  6. Open/reuse your .apx in future

You can persist your Scan or Exploration via "Save as" .apx and reload these template(s) every time you plan/execute to export your quotes data. You could prepare single/standalone .apx for each watchlist. If you approach to design all exports with watchlist output separation in a single task this could be complex because of query symbol membership first and additional membership in multiple watchlists ...

https://forum.amibroker.com/search?expanded=true&q=afl%20watchlist

So i would prefer to start keeping things simple.

Additional i would prefer "switch Statement" instead of "if ... then ... else".

https://www.amibroker.com/guide/keyword/switch.html

To automate AB Explorations you could use AB batch processing

https://www.amibroker.com/guide/h_batch.html

or Windows scripting host and OLE from outside of AB

https://www.amibroker.com/guide/a_script.html
https://www.amibroker.com/guide/objects.html

Alternate you also could develop standalone apps with MS VisualStudio and OLE, parse/replace xml content in .apx ...

Here some example to export quotes from one defined/filtered watchlist via AB Scan or Exploration into an single output file:

/// @link https://www.amibroker.com/kb/2014/11/14/how-to-export-quotes-to-separate-text-files-per-symbol/

// 20240216 - did some changes ...

// create folder for exporting purposes
fmkdir( "C:\\DataExport\\" );

// open file for writing
// file name depends on currently processed ticker
//fh = fopen( "c:\\DataExport\\" + Name() + ".txt", "w" );

// result in single export file for [Apply to: *Filter ] Watch List : DOW 30
fh = fopen( "c:\\DataExport\\DOW 30.csv", "a" );

// proceed if file handle is correct
if ( fh )
{
    dt = DateTime();

    // write header line
    fputs( "Ticker,Date/Time,Open,High,Low,Close,Volume\n", fh );

    // iterate through all the bars

    for ( i = 0; i < BarCount; i++ )
    {
        // write ticker name
        fputs( Name() + "," , fh );

        // write date/time information
        fputs( DateTimeToStr( dt[ i ] ) + ",", fh );

        //write quotations and go to the next line
        qs = StrFormat( "%g,%g,%g,%g,%g\n", O[ i ], H[ i ], L[ i ], C[ i ], V[ i ] );
        fputs( qs, fh );

    }
    // close file handle
    fclose( fh );
}
 
// line required by SCAN option
Buy = 0;

// ----------------------------------------------
// Analysis : Exploration ---------------
// ... just to watch what happens ... -----------
// ----------------------------------------------

Filter = 1;

AddColumn( O, "Open" );
AddColumn( H, "High" );
AddColumn( L, "Low" );
AddColumn( C, "Close" );
AddColumn( V, "Volume", 1.0 );

Best regards,
Peter

2 Likes

Thanks, Peter, for your reply. I am exporting 1min data every minute for 400 bars every trading session. Perhaps i need to create a large script with required for loops for each watchlist with their details hardcoded. In this way each loop will handle only 1 watchlist and move on to next when finished.

Any further guidance in this matter is greatly appreciated. Thanks. :pray:

Hello vtriv,

perhaps this sample AFL Exploration could be starting point for your further research.

My approach is first to scan all existing watchlists for symbol membership, second dump assosiated quotes (append mode) ... please see my comments in AFL sample, there are "unanswered" todos because of time ...

/// @link https://www.amibroker.com/kb/2014/11/14/how-to-export-quotes-to-separate-text-files-per-symbol/

/*

20240217 - i am trying out some things ...

!!! depending on database size this exploration could take some time to finish !!!

----------------------
exploration settings : [Apply to: *All Symbols], [Range: All quotes]
----------------------

--------------------
quick changes for...
--------------------

symbols will be separated by watchlist membership and stored in different export files

----------------
todos for you...
----------------

clear/delete existing exports before execute following exploration(s) because of fopen "a" := append mode, or do anything other ...
add your naming conventions for export files (watchlist s)
add your directory structure for export files (watchlist s)
"switch" between conditions to select correct/desired export directory
adjust all of your "Analysis Exploration Settings/Parameter", like date range ...

*/

//------------------
#pragma maxthreads 1 // could prevent defective results through fputs() process
                     // i assume looping/write data concurrent inside multiple threads could result in i/o deadlocks/data loss on existing/same file
//------------------

dt = DateTime();

fmkdir( "C:\\DataExport\\" ); // create folder for exporting purposes

for( w = 0; CategoryGetName( categoryWatchlist, w ) != ""; w++ ) // iterate through all watchlists, multiple membership is possible
{
    if( InWatchList( w ) ) // match symbol in current watchlist-id
    {
        xid = NumToStr( w, 1, False ); // build for export file name
        xwl = CategoryGetName( categoryWatchlist, w ); // build for export file name

        fhname = xid + "_" + xwl + ".csv"; // APPEND TO WL EXPORT
        fh = fopen( "c:\\DataExport\\" + fhname, "a" );

        if( fh ) // proceed if file handle is correct
        {
            for( i = 0; i < BarCount; i++ ) // iterate through all the bars
            {
                qs = Name() + "," + DateTimeToStr( dt[ i ] ) + "," + StrFormat( "%g,%g,%g,%g,%g\n", O[ i ], H[ i ], L[ i ], C[ i ], V[ i ] );                
                fputs( qs, fh ); // write quote in .csv
            }

            fclose( fh );
        }

    } // end match symbol in watchlist
    
} // end iterate/scan all watchlists

// ----------------------------------------------
// Analysis : Exploration -----------------------
// ... just to watch what happens ... -----------
// ----------------------------------------------

Filter = 1;

AddColumn( O, "Open" );
AddColumn( H, "High" );
AddColumn( L, "Low" );
AddColumn( C, "Close" );
AddColumn( V, "Volume" );

Best regards,
Peter

1 Like

Hello vtriv,

here you find version 0.2 with creating directories and separating watchlist export.
This is prototyp, just an idea ... have fun

/// @link https://www.amibroker.com/kb/2014/11/14/how-to-export-quotes-to-separate-text-files-per-symbol/

/*

20240217 - i am trying out some things ...

version 0.2

!!! depending on database size this exploration could take some time to finish !!!

----------------------
exploration settings : [Apply to: *All Symbols], [Range: All quotes]
----------------------

--------------------
quick changes for...
--------------------

symbols will be separated by watchlist membership and stored in different export files
create directory structure based on watchlist-id
create/append export file based on watchlist-name

----------------
todos for you...
----------------

=================================
!!! clear/delete existing exports before execute following exploration(s) because of fopen "a" := append mode, or do anything other ... !!!
=================================
add/change your naming conventions for export files (watchlist s)
add/change your directory structure for export files (watchlist s)
"switch" between conditions to select correct/desired export directory if needed

*/

//------------------
#pragma maxthreads 1 // could prevent defective results through fputs() process
// i assume looping/write data concurrent inside multiple threads could result in i/o deadlocks/data loss on existing/same file
//------------------

dt = DateTime();

fmkdir( "C:\\AmiData\\" ); // create folder for exporting purposes
fmkdir( "C:\\AmiData\\DB\\" );

for( w = 0; CategoryGetName( categoryWatchlist, w ) != ""; w++ ) // iterate through all watchlists, multiple membership is possible
{
    if( InWatchList( w ) ) // match symbol in current watchlist-id
    {
        xid = NumToStr( w, 1, False ); // build for export file name
        xwl = CategoryGetName( categoryWatchlist, w ); // build for export file name

        fmkdir( "C:\\AmiData\\DB\\" + xid + "\\" ); // create folder for exporting purposes
        fhdir = "C:\\AmiData\\DB\\" + xid + "\\"; // *** APPEND TO WL EXPORT ***
        fhname = xwl + ".csv";
        fh = fopen( fhdir + fhname, "a" ); // watchlist separated export file

        if( fh ) // proceed if file handle is correct
        {
            for( i = 0; i < BarCount; i++ ) // iterate through all the bars
            {
                qs = Name() + "," + DateTimeToStr( dt[ i ] ) + "," + StrFormat( "%g,%g,%g,%g,%g\n", O[ i ], H[ i ], L[ i ], C[ i ], V[ i ] );
                fputs( qs, fh ); // write quote in .csv
            }

            fclose( fh );
        }

    } // end match symbol in watchlist

} // iterate/scan all watchlists

// ----------------------------------------------
// Analysis : Exploration -----------------------
// ... just to watch what happens ... -----------
// ----------------------------------------------

Filter = 1;

AddColumn( O, "Open" );
AddColumn( H, "High" );
AddColumn( L, "Low" );
AddColumn( C, "Close" );
AddColumn( V, "Volume" );

Best regards,
Peter

2 Likes

Hi Peter. Thanks for your time. Really appreciate it! :pray:

Let me know if i am wrong here. I was trying to replicate it on my system when this became clear that all the watchlists are getting processed. Not just a list of watchlists which i as the user can prescribe.

If that actually is the case then it will not only be time consuming but also do unwanted work.

However, if the script can be hardcoded with a list of watchlist names that should be processed then my objective will be achieved.

Please do share your thoughts. And, once again, thanks for your time.

Hello vtriv,

here you find version 0.3 with filter str "watchlist pool".
This is prototyp, just an idea ... have fun

/// @link https://www.amibroker.com/kb/2014/11/14/how-to-export-quotes-to-separate-text-files-per-symbol/

/*

20240218 - i am trying out some things ...

version 0.3

!!! depending on database size this exploration could take some time to finish !!!

----------------------
exploration settings : [Apply to: *All Symbols], [Range: All quotes]
----------------------

--------------------
quick changes for...
--------------------

symbols will be separated by watchlist membership and stored in different export files
create directory structure based on watchlist-id
create/append export file based on watchlist-name
selection/filter of "watchlist pool" for quotes/data export

----------------
todos for you...
----------------

=================================
!!! clear/delete existing exports before execute following exploration(s) because of fopen "a" := append mode, or do anything other ... !!!
=================================
add/change your naming conventions for export files (watchlist s)
add/change your directory structure for export files (watchlist s)
"switch" between conditions to select correct/desired export directory if needed

*/

//------------------
#pragma maxthreads 1 // could prevent defective results through fputs() process
// i assume looping/write data concurrent inside multiple threads could result in i/o deadlocks/data loss on existing/same file
//------------------

//------------------------------
wlpool = "DOW 30,FAANG,S&P 500"; // *** YOUR SELECTION/FILTER OF WATCHLIST(S) ***
//------------------------------

dt = DateTime();

fmkdir( "C:\\AmiData\\" ); // create folder for exporting purposes
fmkdir( "C:\\AmiData\\DB\\" );

for( w = 0; CategoryGetName( categoryWatchlist, w ) != ""; w++ ) // iterate through all watchlists, multiple membership is possible
{
    match = "*" + CategoryGetName( categoryWatchlist, w ) + "*";
    boolmatch = StrMatch( wlpool, match );

    if( boolmatch == True ) // match watchlist selection
    {
        if( InWatchList( w ) ) // match symbol in current watchlist-id
        {
            xid = NumToStr( w, 1, False ); // build for export file name
            xwl = CategoryGetName( categoryWatchlist, w ); // build for export file name

            fmkdir( "C:\\AmiData\\DB\\" + xid + "\\" ); // create folder for exporting purposes
            fhdir = "C:\\AmiData\\DB\\" + xid + "\\"; // *** APPEND TO WL EXPORT ***
            fhname = xwl + ".csv";
            fh = fopen( fhdir + fhname, "a" ); // watchlist separated export file

            if( fh ) // proceed if file handle is correct
            {
                for( i = 0; i < BarCount; i++ ) // iterate through all the bars
                {
                    qs = Name() + "," + DateTimeToStr( dt[ i ] ) + "," + StrFormat( "%g,%g,%g,%g,%g\n", O[ i ], H[ i ], L[ i ], C[ i ], V[ i ] );
                    fputs( qs, fh ); // write quote in .csv
                }
                
                fclose( fh );
            } // end proceed if file handle is correct
            
        } // end match symbol in current watchlist-id
    } // end match watchlist selection/filter
} // end iterate/scan all watchlists

// ----------------------------------------------
// Analysis : Exploration -----------------------
// ... just to watch what happens ... -----------
// ----------------------------------------------

Filter = 1;

AddColumn( O, "Open" );
AddColumn( H, "High" );
AddColumn( L, "Low" );
AddColumn( C, "Close" );
AddColumn( V, "Volume" );

Best regards,
Peter

2 Likes

Thanks a lot, Peter. This works! Thanks for your time and patience. :slightly_smiling_face:

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.