How to Generate Aggregated Statistics in CBT Using Array-based Functions

Hi,

I’ve figured out from the AB online knowledge base articles how to generate P&L quoted in terms of ATR using the CBT module. As you see from the code sample below, I manually accumulate the average statistics for the summary report. Now, I want to expand the statistics from just the average to include also standard deviation, quartile values, e.g. Min, 25%-tile, 50%-tile, 75%-tile, Max for winners and losers. Is there a way to call some array based statistics functions with the array/list of individual trade’s ATR P&L, like one can do in Matlab or Python/numpy, instead of creating a whole set of static variables and “roll my own” statistical calculations for each of the statistics?

Thanks.

/*
	Example Code using CBT to generate ATR P&L stats.
*/

Buy = Sell = Short = Cover = 0;
SetOption("InitialEquity", 100 * 1000);
SetPositionSize(100, spsPercentOfEquity);

// A random (but reasonable?) set of trading rules.
Buy = 
	C == LLV(C, 20) AND C > LLV(C, 50);
Sell = 
	C == HHV(C, 50) OR
	C == LLV(C, 50);
	
// Compute ATR for "P&L in ATR" statistics.
StaticVarSet("ATR_" + Name(), ATR(30) / C * 100);

SetCustomBacktestProc( "" );

function AddATRPnL( trade )
{
	symbolATR = StaticVarGet( "ATR_" + trade.Symbol );
	ATRperc = Lookup( symbolATR, trade.EntryDateTime ); 
	trade.AddCustomMetric("ATR %", ATRperc);
	ATRPnL = trade.GetPercentProfit() / ATRPerc;
	trade.AddCustomMetric( "ATR Profit", ATRPnL );
	
	// Accumulate values for summary statistics.
    sumATRPnL = StaticVarGet("SumATRPnL", 0.0);
    countATRPnL = StaticVarGet("CountATRPnL", 0);
    sumATRPnLWin = StaticVarGet("SumATRPnLWin", 0.0);
    countATRPnLWin = StaticVarGet("CountATRPnLWin", 0);
    sumATRPnLLoss = StaticVarGet("SumATRPnLLoss", 0.0);
    countATRPnLLoss = StaticVarGet("CountATRPnLLoss", 0);	
	sumATRPnL = sumATRPnL + ATRPnL;
	countATRPnL++;
	if (ATRPnL > 0.0) {
		sumATRPnLWin = sumATRPnLWin + ATRPnL;
		countATRPnLWin++;
	} else {
		sumATRPnLLoss = sumATRPnLLoss + ATRPnL;
		countATRPnLLoss++;		
	}
	StaticVarSet("SumATRPnL", sumATRPnL);
    StaticVarSet("CountATRPnL", countATRPnL);
    StaticVarSet("SumATRPnLWin", sumATRPnLWin);
    StaticVarSet("CountATRPnLWin", countATRPnLWin);
    StaticVarSet("SumATRPnLLoss", sumATRPnLLoss);
    StaticVarSet("CountATRPnLLoss", countATRPnLLoss);
    
	return ATRPnL;
}

if ( Status( "action" ) == actionPortfolio )
{
    bo = GetBacktesterObject();
    // run default backtest procedure without generating the trade list
    bo.Backtest( True );
    StaticVarSet("SumATRPnL", 0.0);
    StaticVarSet("CountATRPnL", 0);
    StaticVarSet("SumATRPnLWin", 0.0);
    StaticVarSet("CountATRPnLWin", 0);
    StaticVarSet("SumATRPnLLoss", 0.0);
    StaticVarSet("CountATRPnLLoss", 0);

    // iterate through closed trades
    for ( trade = bo.GetFirstTrade( ); trade; trade = bo.GetNextTrade( ) )
    {
        ATRPnL = AddATRPnL( trade );
    }

    // iterate through open positions
    for ( trade = bo.GetFirstOpenPos( ); trade; trade = bo.GetNextOpenPos( ) )
    {
        ATRPnL = AddATRPnL( trade );
    }

    // generate trade list

    bo.ListTrades( );
    
    // Custom Statistics on ATR PnL
    sumATRPnL = StaticVarGet("SumATRPnL", 0.0);
    countATRPnL = StaticVarGet("CountATRPnL", 0);
    sumATRPnLWin = StaticVarGet("SumATRPnLWin", 0.0);
    countATRPnLWin = StaticVarGet("CountATRPnLWin", 0);
    sumATRPnLLoss = StaticVarGet("SumATRPnLLoss", 0.0);
    countATRPnLLoss = StaticVarGet("CountATRPnLLoss", 0);
    avgATRPnL = IIf(countATRPnL < 1, 0.0, SumATRPnL / countATRPnL);
    avgATRPnLWin = IIf(countATRPnLWin < 1, 0.0, SumATRPnLWin / countATRPnLWin);
    avgATRPnLLoss = IIf(countATRPnLLoss < 1, 0.0, SumATRPnLLoss / countATRPnLLoss);
    bo.AddCustomMetric("Avg P&L (ATR)", avgATRPnL);
    bo.AddCustomMetric("Avg Win (ATR)", avgATRPnLWin);
    bo.AddCustomMetric("Avg Loss (ATR)", avgATRPnLLoss);
    bo.AddCustomMetric("ATR Profit Factor", -avgATRPnLWin / avgATRPnLLoss);
}

The code you have created is little bit more complicated than it needs to be. You only need static variable for this:

StaticVarSet("ATR_" + Name(), ATR(30) / C * 100);

The other static variables are not necessary because you are calling AddATRPnl() only in the context of custom backtester, so they are not needed across executions. All those sums (sumATRPnL, countATRPnL, SumATRPnLWin, … ) don’t need to be static.

You would significantly simplify the code if you used normal variables for those sums.
Now for maximum, you can use

maxPnl = Max( maxPnl, symbolATR );

So your code could look just like this:

sumATRPnL =  countATRPnL = 0;
sumATRPnLWin = countATRPnLWin = 0; 
sumATRPnLLoss = countATRPnLLoss = 0;
maxATRPnL = 0;

function AddATRPnL( trade )
{
	symbolATR = StaticVarGet( "ATR_" + trade.Symbol );
	ATRperc = Lookup( symbolATR, trade.EntryDateTime ); 
	trade.AddCustomMetric("ATR %", ATRperc);
	ATRPnL = trade.GetPercentProfit() / ATRPerc;
	trade.AddCustomMetric( "ATR Profit", ATRPnL );
	
	// Accumulate values in regular variables
        sumATRPnL += ATRPnL;  
        countATRPnL++;

        if( ATRPnL > 0 )
        {
           sumATRPnLWin  += ATRPnl;
           countATRPnLWin++;
           maxATRPnL = Max( maxATRPnL , ATRPnl );

           // and so on

Percentiles are more complicated since you need to perform actual sorting on all values to find out the percentile or things like median, but you could place values inside matrix and then call MxSort to sort values and get percentiles you want.

8 Likes

Thank you, Tomasz.

Just to confirm my understanding of your answer: There is current no statistic functions working off matrix types, so I will have to write my own in AFL?

In your example you calculate scalar statistics. Average, Min, Max, Percentile, Quartile from all trades are still scalar values. For everything except percentiles you don’t need matrices, you don’t even need vectors. For percentile, you don’t need to write your own. Just use MxSort and pick middle value - that is your 50% percentile.

5 Likes

Point taken. Appreciate your insight, Tomasz.

Hi,

I need some help on debugging.

I followed Tomasz's advice and change the CBT code to use plain variables instead of static variables. However, when I ran the code in "Backtest", I got some puzzling errors:

The code in the AFL error is some low-level AFL code (e.g. not in my AFL script), and it wasn't clear which part of my script triggered this error. This is the AFL script I ran:

/*
	Example Code using CBT to generate ATR P&L stats.
*/

Buy = Sell = Short = Cover = 0;
SetOption("InitialEquity", 100 * 1000);
SetPositionSize(100, spsPercentOfEquity);

// A random (but reasonable?) set of trading rules.
Buy = 
	C == LLV(C, 20) AND C > LLV(C, 50);
Sell = 
	C == HHV(C, 50) OR
	C == LLV(C, 50);

// Compute ATR for "P&L in ATR" statistics.
StaticVarSet("ATR_" + Name(), ATR(30) / C * 100);

SetCustomBacktestProc("");

function AddATRPnL(trade)
{
    symbolATR = StaticVarGet("ATR_" + trade.Symbol);
    ATRperc = Lookup(symbolATR, trade.EntryDateTime); 
    trade.AddCustomMetric("ATR %", ATRperc);
    ATRPnL = trade.GetPercentProfit() / ATRPerc;
    trade.AddCustomMetric("ATR Profit", ATRPnL);
    
    // Accumulate values for summary statistics.
    sumATRPnL += ATRPnL;
    countATRPnL++;
    if (ATRPnL > 0.0) {
        sumATRPnLWin += ATRPnL;
        countATRPnLWin++;
        maxATRPnL = Max(maxATRPnL, ATRPnL);
    } else {
        sumATRPnLLoss = ATRPnL;
        countATRPnLLoss++;	
        minATRPnL = Min(minATRPnL, ATRPnL);	
    }
    return ATRPnL;
}

if (Status("action") == actionPortfolio)
{
    bo = GetBacktesterObject();
    // Run default backtest procedure without generating the trade list
    // Initialize accumulation variables
	sumATRPnL =  countATRPnL = 0;
	sumATRPnLWin = countATRPnLWin = 0; 
	sumATRPnLLoss = countATRPnLLoss = 0;
	maxATRPnL = minATRPnL = 0;
	
    // Iterate through closed trades
    for (trade = bo.GetFirstTrade(); trade; trade = bo.GetNextTrade())
    {
        AddATRPnL(trade);
    }

    // Iterate through open positions
    for (trade = bo.GetFirstOpenPos(); trade; trade = bo.GetNextOpenPos())
    {
        AddATRPnL(trade);
    }

    // Generate trade list
    bo.ListTrades();
    
    // Custom Statistics on ATR PnL
    avgATRPnL = IIf(countATRPnL < 1, 0.0, SumATRPnL / countATRPnL);
    avgATRPnLWin = IIf(countATRPnLWin < 1, 0.0, SumATRPnLWin / countATRPnLWin);
    avgATRPnLLoss = IIf(countATRPnLLoss < 1, 0.0, SumATRPnLLoss / countATRPnLLoss);
    bo.AddCustomMetric("Avg Trade (ATR)", avgATRPnL);
    bo.AddCustomMetric("Avg Win (ATR)", avgATRPnLWin);
    bo.AddCustomMetric("Avg Loss (ATR)", avgATRPnLLoss);
    bo.AddCustomMetric("ATR Profit Factor", -avgATRPnLWin / avgATRPnLLoss);
    bo.AddCustomMetric("Best Trade (ATR)", maxATRPnL);
    bo.AddCustomMetric("Worst Trade (ATR)", minATRPnL);
}
1 Like

Read this: http://www.amibroker.com/kb/2016/01/18/how-to-fix-error-61-in-printfstrformat-calls/

Hi Alan,

Thanks for responding to my problem. The issue I am having is that I
don’t see anything in my script that call printf() directly. There is no
stack trace message (like in Python) that shows what function call in my
code has triggered this error (actually 2 of them, but basically the same
thing). The error doesn’t show up during compiation/syntax check in the
AFL editor, but only in runtime.

As far as I can figure out, the error is triggered by some code in CBT.
How do I know? Well, when I replace the CBT code with my previous version
(using Static Variables that Tomasz says is unnecessary, see message
thread), the error goes away and the backtest report generated meets
expectation.

Re-post of the problem code posted 18 hours earlier with 2 changes:

  1. Fixed one bug -- on Line 51; it's immaterial to the error however.
  2. Post the code as image as the code tags used in that post doesn't format consistently and I don't know if it's possible to automatically show line numbers.

Note: You may need to click and open the image in a different browser tab and zoom in to read it.

Greyleaf

You have at least two main issues in this code. One is there is no call to do the actual backtest in your CBI code. After you engage the backtester object, you need a call to bo.Backtest(1);

The other issue is the variables you are using in your function are not initialised before they are being incremented. So you need to initialise them above the function, not in the main part of the CBI code.

The code below fixes those two faults and runs to completion without error:

/*
Example Code using CBT to generate ATR P&L stats.
*/

Buy = Sell = Short = Cover = 0;
SetOption("InitialEquity", 100 * 1000);
SetPositionSize(100, spsPercentOfEquity);

// A random (but reasonable?) set of trading rules.
Buy = 
C == LLV(C, 20) AND C > LLV(C, 50);
Sell = 
C == HHV(C, 50) OR
C == LLV(C, 50);

// Compute ATR for "P&L in ATR" statistics.
StaticVarSet("ATR_" + Name(), ATR(30) / C * 100);

SetCustomBacktestProc("");

sumATRPnL = countATRPnL = 0;
sumATRPnLWin = countATRPnLWin = 0; 
sumATRPnLLoss = countATRPnLLoss = 0;
maxATRPnL = minATRPnL = 0;

function AddATRPnL(trade)
{
symbolATR = StaticVarGet("ATR_" + trade.Symbol);
ATRperc = Lookup(symbolATR, trade.EntryDateTime); 
trade.AddCustomMetric("ATR %", ATRperc);
ATRPnL = trade.GetPercentProfit() / ATRPerc;
trade.AddCustomMetric("ATR Profit", ATRPnL);

// Accumulate values for summary statistics.
sumATRPnL += ATRPnL;
countATRPnL++;
if (ATRPnL > 0.0) {
    sumATRPnLWin += ATRPnL;
    countATRPnLWin++;
    maxATRPnL = Max(maxATRPnL, ATRPnL);
} else {
    sumATRPnLLoss = ATRPnL;
    countATRPnLLoss++;	
    minATRPnL = Min(minATRPnL, ATRPnL);	
}
return ATRPnL;
}

if (Status("action") == actionPortfolio)
{

bo = GetBacktesterObject();
bo.Backtest(1);
// Run default backtest procedure without generating the trade list
// Initialize accumulation variables

// Iterate through closed trades
for (trade = bo.GetFirstTrade(); trade; trade = bo.GetNextTrade())
{
    AddATRPnL(trade);
}

// Iterate through open positions
for (trade = bo.GetFirstOpenPos(); trade; trade = bo.GetNextOpenPos())
{
    AddATRPnL(trade);
}

// Generate trade list
bo.ListTrades();

// Custom Statistics on ATR PnL
avgATRPnL = IIf(countATRPnL < 1, 0.0, SumATRPnL / countATRPnL);
avgATRPnLWin = IIf(countATRPnLWin < 1, 0.0, SumATRPnLWin / countATRPnLWin);
avgATRPnLLoss = IIf(countATRPnLLoss < 1, 0.0, SumATRPnLLoss / countATRPnLLoss);
bo.AddCustomMetric("Avg Trade (ATR)", avgATRPnL);
bo.AddCustomMetric("Avg Win (ATR)", avgATRPnLWin);
bo.AddCustomMetric("Avg Loss (ATR)", avgATRPnLLoss);
bo.AddCustomMetric("ATR Profit Factor", -avgATRPnLWin / avgATRPnLLoss);
bo.AddCustomMetric("Best Trade (ATR)", maxATRPnL);
bo.AddCustomMetric("Worst Trade (ATR)", minATRPnL);
}
4 Likes

Many thanks for the debugging help, Alan. Yes, it works now.

For the benefit of others of the website, here is the working code for generating ATR P&L statistics, with a bug fixed added.

/*
    Example Code using CBT to generate ATR P&L stats.
 */

Buy = Sell = Short = Cover = 0;
SetOption("InitialEquity", 100 * 1000);
SetPositionSize(100, spsPercentOfEquity);

// A random (but reasonable?) set of trading rules.
Buy = 
C == LLV(C, 20) AND C > LLV(C, 50);
Sell = 
C == HHV(C, 50) OR
C == LLV(C, 50);

// Compute ATR for "P&L in ATR" statistics.
StaticVarSet("ATR_" + Name(), ATR(30) / C * 100);

SetCustomBacktestProc("");

sumATRPnL =  countATRPnL = 0;
sumATRPnLWin = countATRPnLWin = 0;
sumATRPnLLoss = countATRPnLLoss = 0;
maxATRPnL = minATRPnL = 0;
   
function AddATRPnL(trade)
{
    symbolATR = StaticVarGet("ATR_" + trade.Symbol);
    ATRperc = Lookup(symbolATR, trade.EntryDateTime);
    trade.AddCustomMetric("ATR %", ATRperc);
    ATRPnL = trade.GetPercentProfit() / ATRPerc;
    trade.AddCustomMetric("ATR Profit", ATRPnL);
    // Accumulate values for summary statistics.
    sumATRPnL += ATRPnL;
    countATRPnL++;
    if(ATRPnL > 0.0)
    {
        sumATRPnLWin += ATRPnL;
        countATRPnLWin++;
        maxATRPnL = Max(maxATRPnL, ATRPnL);
    }
    else
    {
        sumATRPnLLoss += ATRPnL;
        countATRPnLLoss++;
        minATRPnL = Min(minATRPnL, ATRPnL);
    }
    return ATRPnL;
}

if(Status("action") == actionPortfolio)
{
    bo = GetBacktesterObject();
    // Run default backtest procedure without generating the trade list
    // Initialize accumulation variables
    bo.Backtest(True);
    // Iterate through closed trades
    for (trade = bo.GetFirstTrade(); trade; trade = bo.GetNextTrade())
    {
        ATRPnL = AddATRPnL(trade);
    }
    // Iterate through open positions
    for (trade = bo.GetFirstOpenPos(); trade; trade = bo.GetNextOpenPos())
    {
        ATRPnL = AddATRPnL(trade);
    }
    // Generate trade list
    bo.ListTrades();
    // Custom Statistics on ATR PnL
    avgATRPnL = IIf(countATRPnL < 1, 0.0, SumATRPnL / countATRPnL);
    avgATRPnLWin = IIf(countATRPnLWin < 1, 0.0, SumATRPnLWin / countATRPnLWin);
    avgATRPnLLoss = IIf(countATRPnLLoss < 1, 0.0, SumATRPnLLoss / countATRPnLLoss);
    bo.AddCustomMetric("Avg Trade (ATR)", avgATRPnL);
    bo.AddCustomMetric("Avg Win (ATR)", avgATRPnLWin);
    bo.AddCustomMetric("Avg Loss (ATR)", avgATRPnLLoss);
    bo.AddCustomMetric("ATR Profit Factor", -avgATRPnLWin / avgATRPnLLoss);
    bo.AddCustomMetric("Best Trade (ATR)", maxATRPnL);
    bo.AddCustomMetric("Worst Trade (ATR)", minATRPnL);
}
8 Likes

no error msg, but no chart also. any help please...

Santy, I don't understand why your problem is.