Actually I want to make artificial symbols that are updated in real-time just like any other normal symbol , when I use ATC it calculates the entire history again , now if I want to update 2000 composites every second then this strategy fails miserably , I just want to calculate the composite's history once and then keep updating them so 2000 calculations and then updating my 2000 artificial symbols every second won't take much time , please point me in the right direction as to how should I approach this problem , just like when I use the spread afl file and the spread between two instruments is updated seamlessly , I want the same impact but added as symbols so that I can scan for signals on these 2000 Realtime composites that I have made .
You have already started this topic earlier and I had posted a snippet which i tried and update the last value in array.
You are not posting any code. Also, i suggested StaticVar as an alternative.
Using array indexer, you can read or write any specifc value and not deal with whole array.
Updating composites in real time - AFL Programming - AmiBroker Community Forum
I tried to implement the solution that you suggested and using temp[barcount-1] = value doesn't work , if I am making a composite then it gets assigned a certain size how can I peg it with Realtime updating data , so that on completion of every new bar in one minute time frame the composite also is updated in a similar fashion .
Post a working code. Maybe thats why you are not getting responses here
i do not know the answer. Or you have to use what is already suggested but probably it is possible with OLE. But I know too little about it. I know for sure you can add to an artificial ticker symbol. Beppe helped me on that, see (partial) code below. So I process tick data into 1min files every day and import them using the code below and they are added to the existing symbol. So in theory you could first save your data to file and then do an import but this is probably not what you want to do for 2000 symbols every second.
But there are also "Quotations" methods, see: AmiBroker Object Model Maybe you can get 1 of the OLE specialists willing to have a look. Like the "Retrieve" method seems suitable. For me however this OLE stuff is too abstract.
if( importprocesseddata )
{
Say( "import 1Min data" );
AB = CreateObject( "Broker.Application" );
for( s = 0; ( sym = StrExtract( symbollist, s ) ) != ""; s++ )
{
progressPerSymbol = 0;
for( i = 0; i < ndays; i++ )
{
n1 = DateTimeAdd( startDate, i, inDaily );
dow = DateTimeFormat( "%w", n1 ); // day of week
dd = DateTimeFormat( "%d", n1 ); // day of month
mm = DateTimeFormat( "%m", n1 ); // month
yyyy = DateTimeFormat( "%Y", n1 ); // year
fdate = yyyy + mm + dd;
_TRACE( "fdate: " + fdate + " dow: " + dow );
fn1 = dataPath + sym + "\\" + "1minMKTLMT\\" + sym + "_" + fdate + "_" + "1min" + ".csv";
fn2 = dataPath + sym + "\\" + "1minMKTLMT\\" + sym + "_" + fdate + "_Delta_" + "1min" + ".csv";
fn3 = dataPath + sym + "\\" + "1minMKTLMT\\" + sym + "_" + fdate + "_LMTDelta_" + "1min" + ".csv";
fn4 = dataPath + sym + "\\" + "1minMKTLMT\\" + sym + "_" + fdate + "_MKTDelta_" + "1min" + ".csv";
// Importing data via OLE
for( n = 1; n <= 4; n++ )
{
file = VarGet( "fn" + n );
_TRACE( "file: " + file );
if( fGetStatus( file, 3 ) > 0 )
{
//_TRACEF( "Importing [%s]", file );
AB.import( 0, file, "EMP2.format" );
}
else
{
//_TRACEF( "File [%s] does not exist...", file );
}
}
progressPerSymbol = Prec( ( i + 1 ) / ndays * 100, 2 );
_TRACE( "importing 1Min files progress: " + progressPerSymbol + " %" );
}
}
Say( "finished" );
}
OLE is slow compared to ATC. If you can’t do something with ATC or better yet StaticVarAdd, OLE is not the answer. Static variables are at least 100x faster than OLE.
The problem is not how fast you can write one var (because it is blazing fast in case of Static bars) the most time is spent reading thousands of symbols. This is potentially gigabytes worth of data. In real world the HARDWARE has limits.
thanks. I do not really worked with AddToComposite much but I think that it only takes an array. So you can add for instance the last value, like for instance ~ArtificialTicker[ Barcount - 1] but that will replace the entire composite. Meaning then the entire ~ArtificialTicker will have the value of ~ArtificialTicker[ Barcount - 1].
You will need to calculate the whole ~ArtificialTicker again and replace it. There is no point to add the last value when you use AddToComposite.
I am running this script again and again in the exploration to keep my composite updated , now what is the solution that you proposed through staticvaradd ? to keep my composite updated because when i use this code for 2000 composites , its way too slow .
y = "SYMBOL1,SYMBOL2";
for(i=0;i<StrLen(y);i+=2)
{
symbol_1 = StrExtract(y,i);
symbol_2 = StrExtract(y,i+1);
///////////////////////////////Symbol 1
SetForeign(Symbol_1);
C1 = log(C);
RestorePriceArrays();
//////////////////////////////////symbol2
SetForeign(Symbol_2);
C2 = log(C);
RestorePriceArrays();
// calculating the spread
tickerspread = symbol_1+"_"+symbol_2;
staticname = "~"+tickerspread;
spreadc = c1-c2;
AddToComposite(spreadc, "~"+tickerspread, "C", atcFlagDefaults|atcFlagEnableInExplore);
Buy = Sell = Cover = Short = 0;
}```For starters, see here
https://www.amibroker.com/guide/afl/setforeign.html
For a single "C" call, better use Foreign() instead of SetForeign()
As Tomasz mentioned, what are your hardware specs ?
Ryzen 3 2200G , 16gb ram but what is the alternate wayto achieve this through staticvaradd ?
Kindly provide proper techinical words going forward. you are just not accurate.
What is slow ? Time in seconds/min is expected.
Your specs should be properly written including OS, bitness, SSD/HDD etc The CPU is not really high-end as i can see. HDDs are also very slow.
Your DATA size should be clear in GB/number of bars or timeframe.
It may well be that you are just hitting Hardware limits as Tomasz suspects and in that case there you have to pursue many other alternatives.
In the other thread i wrote, after reading from Manual, ATC internally does SetBarsRequired( sbrAll, sbrAll );
so you figure out if you need all the bars etc
The approach is wrong. Why do you need 2000 composites updated in real time is beyond me. And you never explained that.
Let me give you example, someone would ask: "I am using 2000000 SetPixel calls and it is slow, how can I make it fast" (the "HOW" question - wrong) instead "I need to fill entire screen with one color" (describe the GOAL) And the answer is "don't use 2000000 SetPixel cals, use one call to FillRectangle"
You have failed to answer crucial question WHY? What is the GOAL? Please follow this advice: How to ask a good question
You are focused on technical details "how" but that is pointless unless you specify THE GOAL. And the goal is NOT creating 2000 composites per se.
See also my reply: Updating composites in real time - #4 The ideas shown in this thread are pointless as "writing" it is NOT what takes time.
Last thing - DO NOT create multiple threads on same subject.
So actually my goal is to calculate 2000 spreads and keep them updated which I have created from 180 stocks and I use these 2000 spreads as normal symbols and I keep scanning these 2000 spreads for signals in an exploration . I use the ATC function by running it repeatedly in the exploration to calculate these spreads again and again to keep them updated , so this is the problem statement and using the ATC function again and again as it calculates the entire history repeatedly is inefficient , so the motive for starting this post is to find out an efficient way of solving this problem .
That's not the goal. Why do you need 2000 spreads in database? Only for "fun"? You don't need to have 2000 spreads in order to use them in exploration. Spreads can be calculated ON THE FLY within exploration. Using composites makes no sense for this use case. There is absolutely zero gain in using ATC as opposed to doing calc straight in the exploration. Just the reverse, using ATC just to use them later in exploration is BAD IDEA and is slower than doing straight calc within exploration itself.
Also as it has been mentioned many times on this forum, StaticVarAdd is way, way faster than AddToComposite
Its a statistical arbitrage strategy thats why i need these spreads because i trade these spreads , i also want to maintain the history of these spreads to do further studies on them so thats why i need them as symbols available instantly .
As I wrote, you should calculate spreads on the fly. ATC is wrong approach for RT exploration. On the other hand, if you need them for "historical" reasons, you don't need to update them in real time.
I am so sorry for my lack of information , can you please guide me as to how can i create 2000 spreads on the fly ? . Also do you mean the entire history again and again as i use the history in Realtime to calculate indicators on it .