Updating composites in real time

I have been trying to update my composites in Realtime (5min) by running the composite afl file in AA at regular intervals but the problem is that I just want to update the latest value of the composite without resetting or touching the previous history of it and whenever I disable the ATCRESET or ATCDELETE flags the current value of the composite alters the entire history by adding the current value to all the previous values of the composite in the 5 min timeframe , the reason for disabling these flags is that I have some EOD history already present for the composites for the past 10 Years which resets every time whenever I try to update the composite in real time , I am sorry in advance if this problem is arising out of lack of understanding of the basics . How do i tackle this problem ?

Warm Regards

I am doing similar thing with static variable and that also has StaticVarAdd()

so what i do is take the variable to working variable in your case like this

temp = Foreign( "~MyIndex", "C" ) ;  // Your composite

temp[ BarCount -1 ] = <update only last value>

update static-variable with temp 

so in your case you can use atcFlagDeleteValues flag and update the whole thing.

You have to ensure QuickAFL is off to get all the data or test this part yourself.
I forgot whether under QuickAFL the new data updated is partial and old data is left intact or the whole variable is updated by the newer short one created in temp.

Cant i just enter an iif condition in the addtocomposite function stating that only update the recent missing values and do not touch the past history and run this script repeatedly , this also backfills the composite without any hassle , how can i pull this off ?

In practice there is ZERO gain in "updating only last value". That is not what takes time.
Majority of time is spent READING input data for hundreds/thousands of securities that go into composite.


As add to composite is so quick i dont mind recalculating every time in 5min timeframe but when i use the atcflagdefault it resets the entire composite and calculates it again in 5minute timeframe but then resets the EOD history of my composite , if i dont use this flag then it just adds to all the previous values of my composite making it unusable , what should i do in this case ?

I had the same problem with the composite where I wanted to update the last value of an array only and keep previous values of the array the same. I noticed I could not find a solution specifically using composite. Instead I am now loading this array as a new symbol to Amibroker database so all previous values are being kept as is while I am changing only the latest value of the array. With Amibroker there will be many ways to address a problem but that is how it worked for me. Just wanted to share my solution.


Then are you assigning the array back to the symbol and how are you doing that , could you please provide a sample code

Just to update what i had recalled and refreshed my memory regarding QuickAFL and bar behaviour of ATC.

so here AmiBroker Knowledge Base » QuickAFL facts
SPECIAL CASE: AddToComposite function

Since AddToComposite creates artificial stock data it is desirable that it works the same regardless of how many ‘visible’ bars there are or how many bars are needed by other parts of the formula.

For this reason internally AddToComposite does this: SetBarsRequired( sbrAll, sbrAll )

AFL Function Reference - STATICVARADD (amibroker.com)
This has KeepAll flag when it is set to true emulates the behavior of AddToComposite.

Other useful links:
StaticVarAdd is intended as alternative to AddToComposite
Calculating multiple-security statistics with AddToComposite function (amibroker.com)

And, Correct use of StaticVarAdd - AFL Programming - AmiBroker Community Forum

Tomasz documents everything very well, the only problem now is that we have tons of it :stuck_out_tongue: (This is intended as a compliment and no offense be taken )

1 Like

Code is just 3 lines, i mentioned in above post and use atcFlagDeleteValues

Read the ATC to temp, change value at last index ie. BarCount-1 and set ATC again with Delete flag.

But as Tomasz noted which i posted now, anyway all bars are required to calculate, so that's why i think he says you can recreate the whole thing again.
if you are struggling, you can post your code here.

i am sorry for my lack of understanding but can someone please explain what does "ATOMIC ADDITION " means which i come across reading the description of staticvaradd , also if i wanted to update a composite's latest value through conventional staticvar functions by fetching an array from the existing composite , changing it then reassigning it back to the ticker , how can i pull this off , sample code would be really helpful .

Atomic addition in lay man terms in AB is there are multiple threads running at same time, and all trying to add to same single Composite in memory.
So AB will do this task in a manner that is consistent and accurate to give guaranteed result.

just read all the links I posted.

temp = Foreign( "~MyIndex", "C" ) ;  // Your composite
temp[ BarCount -1 ] = 100;  // some scalar value
AddToComposite( temp, "~MyIndex", "C", flags = atcFlagDeleteValues );
// and other appropriate flags from manual

this still doesn't solve the issue as when i get the array from the foreign function it gets the array according to the timeframe i am in for example (5min) , it doesn't store the eod history and then the addtocomposite resets it :frowning: .

Then read TimeFrame functions and compress / expand accordingly and use the data.
Even while setting, do the appropriate transformation.

If it is just one array, see static variables and StaticVarAdd with keepAll and persistence flags.

Multiple Time Frame support (amibroker.com)

And also re-write a proper post on exactly what you want to achieve so its clear.

Actually I want to make artificial symbols that are updated in real-time just like any other normal symbol , when I use ATC it calculates the entire history again , now if I want to update 2000 composites every second then this strategy fails miserably , I just want to calculate the composite's history once and then keep updating them so 2000 calculations and then updating my 2000 artificial symbols every second won't take much time , please point me in the right direction as to how should I approach this problem , just like when I use the spread afl file and the spread between two instruments is updated seamlessly , I want the same impact but added as symbols so that I can scan for signals on these 2000 Realtime composites that I have made .

The approach is wrong. Why do you need 2000 composites updated in real time is beyond me. And you never explained that.

You have failed to answer crucial question WHY? What is the GOAL? Please follow this advice: How to ask a good question

You are focused on technical details "how" but that is pointless unless you specify THE GOAL. And the goal is NOT creating 2000 composites per se.

See also my reply: Updating composites in real time - #4 The ideas shown in this thread are pointless as "writing" it is NOT what takes time.

Last thing - DO NOT create multiple threads on same subject.

Topic is closed because it is duplicate: Realtime composites