How to transfer data from one database to another for individual script

Hi,
I am looking for way to transfer data (ticker, date, time, O,H,L,C, V) of single script from one database to another.
The articles, that I found, do transfer of data for all scripts .

But is inter database transfer at script level possible?

Hi @saritguha,

Ideal steps (fastest exploration method) mentioned here:

You are specific on couple of things:

  1. There is a segregation required for Date & Time which is not available using Default Columns.
  1. And quotes required for one script only!

So, you may use below Exploration formula. Please note, this method is not recommended all the time; it involves looping of all quotes and fast-processing shall get diminished when applied on thousands of symbols, otherwise all-is-well.

/*_______________________________________________________________________________________________________________________
|This code is for AmiBroker Formula Language (AFL) learning (non-commercial) purposes only. Please do not copy this code|
|(or any other version of it) and paste it over on other forums or anywhere else on the Internet or in any other form	|
|without AmiBroker Forum owner's consent (https://forum.amibroker.com/).                                            	|
_______________________________________________________________________________________________________________________*/

_SECTION_BEGIN( "Export Quotes" );
	DT = DateTime();

	if( Status( "ActionEx" ) == actionExplore ) {
		Filter = 0;
		
		SetOption( "NoDefaultColumns", True );
		
		// To display column headers
		AddTextColumn( "", "Ticker" );
		AddTextColumn( "", "Date" );
		AddTextColumn( "", "Time" );
		AddTextColumn( "", "Open" );
		AddTextColumn( "", "High" );
		AddTextColumn( "", "Low" );
		AddTextColumn( "", "Close" );
		AddTextColumn( "", "Volume" );
		// AddTextColumn( "", "OI" );
		// AddTextColumn( "", "Aux1" );
		// AddTextColumn( "", "Aux2" );
		
		// To display quote rows
		for( i = 0; i < BarCount; ++i ) {
			AddRow(
				StrFormat( "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s",
					Name(),
					DateTimeFormat( "%Y%m%d", DT[ i ] ),
					DateTimeFormat( "%H%M%S", DT[ i ] ),			
					NumToStr( O[ i ], 1.2, False ),
					NumToStr( H[ i ], 1.2, False ),
					NumToStr( L[ i ], 1.2, False ),
					NumToStr( C[ i ], 1.2, False ),
					NumToStr( V[ i ], 1.0, False )
					// NumToStr( OI[ i ], 1.0, False ),
					// NumToStr( Aux1[ i ], 1.2, False ),
					// NumToStr( Aux2[ i ], 1.2, False )
				)
			);
		}
	}
_SECTION_END();

Before hitting Explore in Analysis, make sure you Apply to Current (wherein Current refers to the Symbol for which you intend to export quotes):
image

And choose Periodicity properly:

Then Explore to Export:
image

Once exported you may then import that CSV to New Database using Import Wizard; while importing, you may also save as a ASCII format for future quick imports.

Use Explore or Scan to write CSV files to C:\Data Collection\OneMin folder.
Then Import to any database.

//Export  all intraday data from amibroker
//Click on explore.
//It will save data in C:\\Data Collection\\OneMin
//Time frame must be 1m.
SetBarsRequired( sbrAll, sbrAll );

if( Status( "actionex" ) != actionExEditVerifyFormula )
{
    if( Interval() != in1Minute )
    {
        Error( "Timeframe Must be 1 min" );
    }
}

Make_Files = ParamToggle( "Make Files", "No|Yes", 1 );

fmkdir( "C:\\Data Collection" );
fmkdir( "C:\\Data Collection\\One Min" );

Get_Year = Year();
Get_Month = Month();
Get_Day = Day();

Get_Hour = Hour();
Get_Min = Minute();
Get_Sec = Second();

Date_Format = StrFormat( "%02.0f%02.0f%02.0f", Get_Year, Get_Month, Get_Day );

if( Make_Files )
{
    fh = fopen( "C:\\Data Collection\\One Min\\" + Name() + "_" + Date_Format + ".csv", "w" );

    if( fh )
    {
        //Heading Write
        fputs( "Name,Date,Time,Open,High,Low,Close,Volume,OpenInt\n", fh );

        for( i = 0; i < BarCount; i++ )
        {
            Name_Date_Write =  Name() + StrFormat( ",%02.0f%02.0f%02.0f,", Get_Year[ i ], Get_Month[ i ], Get_Day[ i ] );
            Time_Write = StrFormat( "%02.0f:%02.0f:%02.0f,", Get_Hour[ i ], Get_Min[ i ], Get_Sec[ i ] );
            OHLC_Data =  StrFormat( "%g,%g,%g,%g,%g,%g\n",  Open[ i ], High[ i ], Low[ i ], Close[ i ], Volume[ i ] , OpenInt[ i ] );
            Data_To_Write = Name_Date_Write + "" + Time_Write + "" + OHLC_Data;
            fputs( Data_To_Write, fh );
        }

        fclose( fh );
    }

    Buy = Sell = 0; // for scan
    Filter = Status( "lastbarinrange" ) ;
    AddTextColumn( "Export done", "Status", 1, colorDefault, colorDefault, 90 );
}

1 Like

Hi @Yogyatrader,

Thanks for participating!

There is one aspect that I would like to highlight in the code that you shared:

for( i = 0; i < BarCount; i++ )
{
    ....
    fputs( Data_To_Write, fh );
}

According to the manual, fputs() writes (puts) the string to the file - that file sits on the HDD. Imagine, Barcount of 50,000 (often way more) worth of 1-minute quotes; then, that many times a string is written onto the HDD. Even if this operation is for 1 symbol, it must be avoided at all times!

So, how about concatenating into one huge string and then fputs() once? That is also not a good practice either, as the string operation becomes costly resource-wise.

In several past posts such practices are discouraged, screaming loudly...

fxshrat's recommendation to use Batch export is most viable, if one intends to automate the Explore/Scan export process.

It is possible just to copy data file from one database to the other. Simple file copy. There is separate file for each symbol. As always read the manual: Understanding AmiBroker database concepts
To copy data for symbol "XYZ" copy 'XYZ" file from one database to the other (to appropriate A-Z folder)

If you copied files that were not present in target database, you need to delete broker.master file for AmiBroker to read new additions, as described in Files used by AmiBroker

With regards to concatenation strings vs fputs, @Cougar you are mistaken. In all major OSes disk writes are heavily buffered by both OS itself and C runtime library, so for this reason fputs() function does not directly make N writes to hard drive for every string and it makes it faster than what you suggested.

3 Likes

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.