# Safe-f parameters: a check of the resuts

Tags: #<Tag:0x00007f80568bf2d0> #<Tag:0x00007f80568beec0>

#1

Hi all

I have found very interesting the safe-f method to evaluate a trading system, as explained in the QTA book ; and I have tried to have an AFL code, using some suggestion I have found here and modifying myself a code found elsewhere done by Mr Talikka, based closely in the Python listing free on BlueOwlPress site, and with included the proprer author credit to Mr Bandy

But to test it, I have used as benchmark the page 139 of QTA that gives :
for SPY, 5% / 20% / 2y , 21d holding period , from 1/1/1999 to 1/1/2012 ,
safe-f is 0.563 and CAR25 is 5.4%

safe-f is 0.5134 and CAR25 is 2.44%

obvious that every calculation gives a bit different result,
but while the safe-f is in the approximate area of the book, CAR25 isn’t .

I hope in my misinterpretation of the inputs , or a print error, of course:
may anyone help me in verify the result, to be sure about the values ?

Cesare

#2

How are you calculating it? Are you using the Python code that @howardbandy includes in the book?

I’m not aware of it being possible to calculate CAR25 automatically in Amibroker, since it requires multiple Monte Carlo runs to nail down the level to hold the risk of a 20% drawdown at the 95th percentile. It would either involve some pretty clever programming and novel use of the batch functionality, or else you could do it manually by changing the input parameters each time and re-executing it until you hit the target. But Python does it all in a single run.

I’d love to stand corrected on this one if anyone has manged to achieve it through an automated process with Amibroker.

#3

Previously posted on the old AmiBroker Forum ----

Hi Cesare –

I am the developer of safe-f and CAR25.

Safe-f is the position size that results in the highest equity while holding probability of drawdown to the trader’s risk tolerance. Alternative trading systems (or any use of funds that have both risk and reward) can be compared by “risk normalizing” them through use of safe-f.

CAR25 is an estimate of future profit that is based on a set of trades taken at safe-f. It is the 25th percentile of the cumulative distribution of risk-normalized profit over some number of trades. CAR25 is the most universal objective function I have found.

Both of these are most useful in estimating the risk and profit potential of a trading system when they are applied to a set of trades that is the trader’s “best estimate” of future trades. These come from either real trades, paper trades, or the out-of-sample trades from the validation process.

As part of the validation process, we rely on “the future resembles the past.” That is, that the distribution of future trades and the distribution of the trades used for the analysis are the same. That the distribution of trades is stationary.

While we do everything we can to ensure the distributions are stationary, and trades that form the distribution are similar, we cannot assume that trades will occur in a particular order. Consequently, the procedure to compute safe-f includes a Monte Carlo process of randomly drawing trades from the best estimate set and creating many (say, 1000) equally-likely equity curves that extend through the forecast period (say two years).

While any set of trades can be analyzed using safe-f and CAR25, accuracy and confidence come from the trades used in the analysis being accurate estimates of the future. Since in-sample (backtest) trades have no value in estimating future performance, out-of-sample trades must be either captured during walkforward or read in as a separate data series. Given that, both safe-f and CAR25 can be computed in AmiBroker.

Best regards,
Howard

#4

Greetings –

As I have written, there are two systems in play – each system is a combination of a model and some data. The model is the set of rules that process the data, identifying important patterns. Note that this definition of a system is very standard in the modeling and simulation profession. The trading community, and most of its literature and advice, has been slow to adopt it and adapt to it.

The first system is the trading system. Its data is prices, its model examines the trade data (OHLCV) and indicators that are based on or coincide with the data and issues signals to buy and sell in response to patterns identified by the rules in the model. The only purpose of the trading system is to identify Buy and Sell.

The second system is the trading management system. Its data is recent trades, its model examines the trades and computes safe-f and CAR25 for the next trade. It computes position size. Importantly ---- if position size is removed from the trading management system and calculated within the trading system (as is common), there is no “knob left to turn” in trading management to help the trader determine, analytically, the health of the system and manage the system.

Trading is limited by risk. Each trader should have a clear statement of his or her risk tolerance. An example is: “I am trading a \$100,000 account and forecasting two years. I want to hold the risk of a drawdown in excess of 20% to a chance of at most 5%.”

safe-f and CAR25 can be computed from any set of trades. They Could be computed from in-sample (backtest) trades. But in-sample results always underestimate risk and overestimate profit. The degree of the over optimism depends on both the data and the model. Traditional trading system development platforms, including AmiBroker (and TradeStation, WealthLab, NinjaTrader, …) produce models based on “decision trees.” Decision trees have many attractive properties, including ease of understanding and not needing extensive data preprocessing. One of the disadvantages of decision trees is that they easily overfit the data. That is, they adjust the rules to fit whatever data is presented, often not able to distinguish between profitable and persistent signal and random noise. As a consequence, it is risky to base trading decision on in-sample data.

Best regards, Howard

#5

Dear Alan,

You could always aim to calculate it in Python - on a standalone basis - and link it to Amibroker very much like external scripting. I believe that is one of the preferred methods to tackle sophisticated statistical pieces elsewhere.

Many thanks,

Kind regards,

#6

There is no python required at all to calculate and output that CAR25 thingy.
All that is needed is AFL (Matrix feature (or slower VarSet/VarGet) and CBT).

#7

Very interesting @fxshrat. Would you care to elaborate on the implementation?

#8

Well, how about checking the User’s Guide:

http://www.amibroker.com/guide/h_montecarlo.html

(scroll down to “Now here is the sample code that presents how to add 30th percentile FinalEquity and CAR to the report:”)

It does NOT require matrices.
It is ALL built-in.

It requires TWO lines of code in custom backtester.

``````// add two lines to standard high-level custom backtest code
mc = bo.GetMonteCarloSim();
car25 = mc.GetValue( "CAR", 25 ); // here is your CAR25
``````

Full code included in the User’s Guide.

Really I add new features and I do describe them in the manual and examples are in the manual.

The functionality is there for about 2 or 3 years now.

#9

There is no elaboration required because AB has it built-in for a few versions back
https://www.amibroker.com/guide/h_montecarlo.html

My point I was trying to make was that even if it would not be built-in ANYTHING (any calculation/output) can be achieved from within AmiBroker. No Python etc. required.

#10

Greetings –

The chart is very nice. There might be a lot of interest is expanding this so that it could be an automatic report as part of validation of a trading system.

Would it be possible to put together an example showing the following:
*** Begin with a trading system that trades a single ticker, say SPY, frequently and holds 1 or 2 days. Use state signals so there is a signal every day and an equity measurement every day. All trades fixed position size of \$10,000. Begin with \$100,000. Assume the risk tolerance is a 5% chance of a 20% drawdown. Set a variable, call it “limdd” to 0.20.
*** For an example trading system: Buying when RSI(2) falls through 30 and Selling when RSI(2) rises through 30 will work. Since this is the system the trader is developing, every trader will insert his or her own ideas here.
*** Do all the testing and optimizing necessary to arrive at a candidate trading system, ready for validation.
*** Perform a walk forward validation. (The trader wold do this one time only.) For this example, set the in-sample period to 4 years; out-of-sample period to 1 year. Run for OOS period of 2010 through 2014. (5 years)
*** The steps in … that follow compute safe-f and CAR25 for the trading system development:
… Gather together the OOS trades for analysis. (Ignore IS trades.) This is “the best estimate” of trades.
… Determine how many trades will, on average, occur in two years. Call it “n”.
… Set a variable “estps” to 1.00. This is the estimate of position size.
… Repeat the … steps as necessary
… Make a Monte Carlo run of 1000 iterations. For each iteration, draw n trades with replacement and create an n-trade sequence, where each trade is made with a position size of 100*estps percent of marked-to-market current equity. Determine the maximum drawdown, measured as the greatest percentage drop from highest marked-to-market equity. Form a distribution of the 1000 maximum drawdown amounts and determine the drawdown at the 95th percentile. Call this “estdd”.
… Adjust estps as follows: New estps = prior estps * limdd / estdd
… Repeat … steps until estdd is close to limdd. estps is safe-f.
… Report safe-f.
… Make a single Monte Carlo run of 1000 iterations using position size of safe-f. Determine the final equity for the two year sequences. (It might be saved from the latest iteration of the steps that determine safe-f.) Form a distribution of final equity. Determine the TWR (terminal wealth relative – final equity divided by initial equity) at the 25th percentile. Compute the compound rate of return that produces that TWR. This is CAR25. Report CAR25.

All of the above relates to the development of the trading system – design, test, validation. If CAR25 is high enough to suggest that the system is worth trading, continue on to trading management. Trading management is a separate system. The two have the “best estimate” set of trades in common.

The trading management system uses trades as its input and computes safe-f and CAR25 using recent trades. There is a sliding window in which trades made after the system was frozen and was passed from development to trading replace trades from the walk forward validation. An approximation of these could come from running the trading system code used in development, but there are other considerations, such as length of the window used for trading management, how the trades in that window are weighted, fill prices, period over which safe-f is recomputed, etc.

In my opinion, it is cleaner to run the trading management program separate from the trading system program.

Best regards, Howard

#12

There is no quibble that the functionality exists to generate the final CAR25 value. However, the process to get there, as we know from the book and @howardbandy’s comments above, is that before you can do that final step, you first need to calculate safe-f. That involves multiple Monte Carlo runs, adjusting the position size up and down until you finally hit a value of 20% drawdown at the 95th percentile:

Once you have that, you can then do a final MC run to generate the CAR25 at the safe-f position size:

As @Tomasz rightly says, we can do that final step easily with 2 lines of code:

[quote=“Tomasz, post:8, topic:899”]

``````mc = bo.GetMonteCarloSim();
car25 = mc.GetValue( "CAR", 25 ); // here is your CAR25
```[/quote]
But  performing the first part to calculate safe-f involves multiple backtests, using the MC result of each one to adjust the position size as an input for the next one, until the drawdown at the 95th percentile is at 20%. The resulting safe-f can then be used to determine CAR25. Needless to say, Python manages this all in a single run without any user input.

My thoughts are something similar could be achieved using Amibroker's Optimize feature using a dummy variable to control the number of backtest runs, and a static variable to hold the newly calculated position size for each next run in the Optimize series until we find the one that gives the MC result of 20% drawdown at the 95th percentile, and therefore also the CAR25 required.   However, unless there is a way to interrupt the Optimizer run programatically once we have the value we need, we would always be constrained to a fixed number of tests, which could be way more than we need. Python's routine on the other hand, just completes once it has the answer.

I guess the question is, is there a better way to perform the process in Amibroker that is somewhat more elegant or efficient in its approach?``````

#13

You can easily automate variable number of backtests run automatically until you get the value you need using OLE http://www.amibroker.com/guide/objects.html. You can also stop optimization prematurely simply using

``````if( condition ) Error("my condition is reached");
``````

This coupled with batch processor can achieve the goal without scripting.

Or you can use optimizer DLL interface to write your own driver that would do the same without resorting to Error() - as custom optimization engines have complete freedom over optimization steps. There is endless ocean of possibilities.

#14

was the code you used this one? It has issues with concurrent trades, but for systems that trade infrequently it should produce very similar results to Howard’s python code.

``````SetCustomBacktestProc( "" );

if( Status( "action" ) == actionPortfolio )
{
accuracy_tolerance = 0.005;
number_forecasts = 400;
initialequity = 100000;
ddTolerance = 0.20;

bo = GetBacktesterObject();
bo.backtest();
stats = bo.GetPerformanceStats( 0 );

fraction = 1;
last_fraction = 0;
TWR25 = 0;
ObFnCAR25 = Null;
TWR = Matrix( 1, number_forecasts );
maxDD = Matrix( 1, number_forecasts );
dd95 = 1;
iterations = 0;

if( numberTrades > BarCount / 15 )
{

barResult = 0;

{
barResult += IIf( DateTime() == trade.EntryDateTime, 1 + trade.GetPercentProfit() / 100 , 0 );
}

barResult = IIf( barTrades, barResult, 1 );
forecast_horizon = 504;
years_in_forecast = 2;

while( abs( dd95 - ddTolerance ) > accuracy_tolerance AND iterations < 15 )
{
for( i_forecast = 0; i_forecast < number_forecasts; i_forecast++ )
{
ac_equity = initialequity;
maxEquity = ac_equity;
drawdown = 0;
maxDrawDown = 0;
horizonSoFar = 0;
while( horizonSoFar < forecast_horizon )
{
j = 1 + ceil( mtRandom() * ( 756 ) );
weightJ = 1 - j / 756;
horizonSoFar = horizonSoFar + weightJ;
tradeJ = ( barResult[BarCount - j] - 1 ) * weightJ;
ac_equity += fraction * tradeJ * ac_equity;
maxEquity = Max( ac_equity, maxEquity );
drawdown = ( maxEquity - ac_equity ) / maxEquity;
maxDrawdown = Max( drawdown, maxDrawdown );
}
TWR[0][i_forecast] = ac_equity;
maxDD[0][i_forecast] = maxDrawdown;
}

sortedDD = MxSort( maxDD );
dd95 = sortedDD[0][ floor( 0.95 * number_forecasts ) ];
last_fraction = fraction;
fraction *= ( ddTolerance / dd95 );
iterations++;
}

sortedTWR = MxSort( TWR );
TWR25 = sortedTWR[0][ floor( 0.25 * number_forecasts ) ];
ObFnCAR25 = 100 * ( ( ( TWR25 / initialequity ) ^ ( 1 / years_in_forecast ) ) - 1 );
}

bo.addcustommetric( "ObFnCAR25", ObFnCAR25, 0, 0, 4, 4 );
bo.AddCustomMetric( "Safe-F", last_fraction, 0, 0, 4, 4 );
bo.AddCustomMetric( "dd95", dd95, 0, 0, 4, 4 );
bo.AddCustomMetric( "iterations", iterations, Null, Null, 0, 4 );
}
``````

#15

good morning Dr Bandy and thanks for for answers

I was trying exactly this way to test my systems, and the use of OOS trades, examined with safe-f method, probably will give the best test possible

I used a ‘downsized’ method, that is
**I test my system full fraction ( that is, -100 spsofequity or similar) ;
** then I perform a MC test:
*** if the 5% percentile has an MDD > 20%, I’ll try with less fraction ( that is, e.g., -80 spsofequity)
** I perform another MC test and so on till I get an MDD = 20% more or less in the 5% percentile: and this is my safe-f

of course there are sono semplifications: the biggest are the holding period, non stated and so is that of the system itself, and the % of winning, again using that of the system

so, with the help of the Community here, I hope to find a better solution

#16

good morning Mr Janu and thank for your answer and for share the AFL

I found the code in the web, so it isn’t mine and I don’t feel free to post it here: I tried to find anything about the author or even some instructions, but is seems a piece of an old work and now is abandoned in the web:
as today, is easy to find it free in Google, but it opens as a single page to download the code and nothing else

the method it used is very clever and similar to that I used before ‘manually’: given the parameters, the AFL tries a MS test many times to find the best approximation of the fraction needed with the 5% / 20% / 2y parameters, and then computes CAR25. All is done external to the system and the result in given as a ‘commentary’ in ‘analisys’ , so even this aspect is in perfect accord to the suggestion given by Dr Bandy . And I wish to state that the programmer has given full credit to QTA book and Dr Bandy, to the point to full quoting the first lines of the Python code ( and that’s the way I found it )

A little gem I believe:
but maybe I am not able to set the parameters
or maybe there is a flaw in the code: and this is the motivation for my first question.

Now I’m happy to look at your kind AFL code:
and I hope there is the possibility to post here the code by Mr Talikka , because after all the code is free on the web, and gives an elegant solution to the safe-f computation problem

regards
Cesare

#17

Greetings –

I want to say nothing that detracts from the AmiBroker platform and its capabilities. I continue to believe and publicly state that AmiBroker is by far the best of the “traditional” trading system development platforms.

The difficulties being discussed in this thread and forum arise from limitations that restrict all traditional platforms.

1. The models that are available for development using traditional platforms are almost always limited to individual “decision trees.”
2. The paradigm is “identify a pattern or compute an indicator, then see what happens after.”
3. There are limited facilities to determine the “hyperparameters” of the model.
4. There are limited alternatives to measure the fitness of the model.
5. There are limited facilities to validate the model.

As background – I began studying and applying artificial intelligence, including to financial problems, in the late 1960s while I was in graduate school. My mentors were among AI’s first generation. Some of my early research contributed to what is now know as the K-nearest neighbor technique. I have continued study and application of AI to financial data throughout my life.

Neural networks were one of the first AI applications that had promise and could be run on desktop computers. In the 1990s, at the request of one of my employers, I tried very hard to fit NN models to stock data. And failed.

That has since changed. AI capabilities useful for home development of trading systems started to become available about 2000, and mushroomed over the past few years with Python and scikit-learn. Hedge funds and banks were among the first to show high profit using AI and machine learning (ML). Refer to David Shaw, James Simons, etal.

In April 2014 I spoke at a conference in Australia and presented a workshop introducing ML-based trading system development. During one Q&A session, I was asked when I thought AI/ML would overtake traditional trading systems. I said two years. I was too conservative. It happened before that. AI and ML facilities that can be run on desktop computers are now state-of-the-art for trading system development.

The techniques are continuing to improve rapidly. At that same conference, I was asked what was the best ML technique. There is a formal theorem that shows there is not a single technique that is always best, but I suggested beginning with support vector machines together with genetic algorithms. That is no longer the case. As of today, ensembles, boosting, and stacking are winning AI competitions. Refer to Kaggle. All of these are supported by Python and scikit-learn, will run on any home computer – Windows, Mac, or Linux – and are free.

As a very rough and admittedly unscientific comparison. Assume a single decision tree system fits data well. If it was a trading system, it would be tradable. Using one of the machine learning scoring metrics, it might score 70 on a scale of 0 to 100. Five years ago any of us would have considered it an excellent system. Today, at home, ensembles, boosting, and / or stacking fit those models to that same data better. And regularly score mid 90s.

All five of the limitations I listed above are lifted for machine learning.

1. There are dozens to hundreds of models available. Those available in scikit-learn have a common interface. You can test and compare twenty or more in a single computer run. Individual “decision trees” are essentially never at the top.
2. The paradigm is “identify an important event, then see what happened earlier.”
3. Cross validation routines are available to determine the “hyperparameters” of the model without using up degrees of freedom or contaminating out-of-sample data.
4. There are unlimited alternatives to measure the fitness of the model. All of the computations necessary to computer CAR25 can be done “on the fly” during development.
5. There are unlimited facilities to validate the model.

Additionally, the trading management system follows easily, computes CAR25 for any number of alternative fund allocation, ranks them, and indicates which to use and at what position size. Day-by-day, trade-by-trade.

The discussion we are having here is only one aspect of skills and facilities needed to improve trading systems. Among others, all of which you can do with AmiBroker, are:

1. Work with a single issue long/flat.
2. Mark to market daily and use state signals.
3. Understand that risk is “the” primary limitation. Be willing to forgo opportunity in order to avoid losing trades. Know your personal risk tolerance. Know the risk of whatever you are considering trading. Compute the risk-normalized profit potential using your best estimate of future performance.
4. Understand the importance of the mantra:
C. Hold a short period.
D. Avoid losses.
5. Modern portfolio theory is dead (actually, it was never alive – it just had very vocal supporters). Work with a single issue / model combination per system.
6. CAR25 is a Dominant metric. It is far superior to mean-variance / modern portfolio theory. Never use Sharpe ratio when its use penalizes winners. Rank all alternative uses of funds and allocate all available funds to that alternative that has the highest CAR25. Splitting funds is suboptimal.

Returning to the difficulties of implementing CAR25 as a development metric ----

Tomasz is providing an outstanding development platform in AmiBroker. It is powerful, fast, inexpensive, extremely well supported. By far, the best available.

I did not include in the list of limitations above those that AmiBroker has already overcome, but that many other traditional platforms have not. Limited objective function metrics, lack of two-stage processing as with CBT, inability to use state signals, lack of out-of-sample testing for validation, etc. AmiBroker is outstanding in its capabilities in these areas.

Traditional platforms are the extensions of the early days of trading and technical analysis – tape reading, chart reading, trend lines, flags, divergences, chart patterns, indicators, and eventually rule-based decision tree models.

The trading system development community has been extremely slow to adopt the scientific method. Many (most?) do not have technical educations. It is understandable that there is fear among them that their profession is becoming obsolete. I regularly receive bitter criticism from readers (or non-readers), whose names you would recognize, of my research and suggestions.

In summary.

If you want to stay with a traditional platform and individual decision tree models, AmiBroker is by far the best choice.

But machine learning is here to stay and it produces better trading systems. As long as you are trading an issue where your system has an acceptable return, continue to do that. Carefully monitor your performance and be quick to take a system offline when it begins to show signs it is no longer competitive.

Be aware that there is learning curve to machine learning. Expect to read a lot – dozens of books. (Begin with Daniel Kahneman’s “Thinking, Fast and Slow.”) And to brush up on your math (through linear algebra and calculus), statistics (it is pretty much all Bayesian now), and programming (Python).

Thanks for listening,
Best regards, Howard

#18

Hi Cesare,

I wrote that amibroker translation of Howard’s python code of listing 5.1 from the book QTA, which now seems to be stored forever online. The process was quite simply starting with the python file and translating line by line into afl, which was why i left all of Howard’s text at the top and all of the variables are named the same. I don’t recall posting the code anywhere, but did share it with some others via email. The “licence” from Howard’s original remains, and allows sharing when properly attributing source, so it is posted below.

The code i offered above calculates car25 using Howard’s method from the book during a system backtest, rather than just calculating the profit potential of a price series. There are limitations to my code as running the backtest MC multiple times is not possible, so i had to not use the built in MC in amibroker and code my own instead modelled on Howard’s python version. The method i chose uses each trade percent profit or loss, rather than daily equity change of the system. Each trade profit or loss is assigned to the day the trade is entered, and days with multiple trades have the result averaged. Thus it won’t be accurate for systems that have trades lasting significantly different durations. I should really recode it to use daily equity changes, but it works well enough for my purposes as it is.

Cheers,
Jani Talikka

``````// attempt to recreate Listing 5.1 from QTA
// computes risk, determines safe-f, and estimates profit potential
// Amibroker code written by J Talikka based on:
//
/* Listing 5.1
ComputeRiskLong.py

A Python program that implements the
Monte Carlo simulation analysis described
in Chapter 5 of the book
“Quantitative Technical Analysis”
written by Dr. Howard Bandy

Author: Howard Bandy
Blue Owl Press, Inc.
www.BlueOwlPress.com
May 29, 2014

Trimmed down version that only looks
at marked to market closing price.

This program is intended to be an educational tool.
It has not been reviewed.
It is not guaranteed to be error free.
Use of any kind by any person or organization
is with the understanding that the program is as is,
including any and all of its faults.
It is provided without warranty of any kind.
No support for this program will be provided.

The programming is intended to be clear,
but not necessarily efficient.
It was developed using Python version 2.7.

Be careful of line wrap and quotation marks.

There are several options within the program
that allow various information to be saved to
disc for further analysis, formatting, or display.
Using them requires understanding of the program,
modifications to the program’s control parameters,
and removal of comment marks.

Permission to use, copy, and share is granted.

Please include author credits with any distribution.
*/

SetBarsRequired(sbrAll,sbrAll);

accuracy_tolerance = 0.005;

startdateDN = ParamDate("Start Date", "01-01-1999", 0);
enddateDN = ParamDate("End Date", "01-01-2012", 0);
startdateDT = DateTimeConvert( 2, startdateDN );
enddateDT = DateTimeConvert( 2, enddateDN );

hold_days = Param("Hold Days", 5, 1, 20, 1);
system_accuracy = Param("System Accuracy", 0.75, 0, 1, 0.01);
DD95_limit = Param("95 percentile DrawDown limit", 0.20, 0, 1, 0.01);
initial_equity = Param("Initial Equity", 100000, 10000, 1000000, 10000);

forecast_horizon = Param("Forecast Horizon (trading days)", 504, 25, 1500, 1);
number_forecasts = Param("Number of simulated forecasts", 10, 1, 1000, 1);

printf("\n\nNew simulation run \n Testing profit potential for Long positions\n");
printf("Issue: \t\t\t" + FullName() + "\n");
printf("Dates: \t\t\t" + DateTimeToStr(startdateDT) + "\n");
printf("\t\tto: \t" + DateTimeToStr(enddateDT) + "\n");
printf("Hold Days: \t\t" + hold_days + "\n");
printf("System Accuracy: \t" + system_accuracy + "\n");
printf("DD 95 limit: \t\t" + DD95_limit + "\n");
printf("Forecast Horizon: \t\t" + forecast_horizon + "\n");
printf("Number Forecasts: \t" + number_forecasts + "\n");
printf("Initial Equity: \t\t" + initial_equity + "\n");

BarDT = DateTime();
BarDN = DateTimeConvert( 0, BarDT );

startdateBI = Lookup(BarIndex(), startdateDT, 1);
enddateBI = Lookup(BarIndex(), enddateDT, -1);

nrows = 1 + enddateBI - startdateBI;

printf("Number Rows: \t\t" + nrows + "\n");

number_trades = floor( forecast_horizon / hold_days );

printf("Number Days: \t\t" + number_days + "\n");

account_balance = 0; 	// account balance

max_IT_DD = 0; 			// Maximum Intra-Trade drawdown
max_IT_Eq = 0; 			// Maximum Intra-Trade equity

FC_max_IT_DD = 0; 		// Max intra-trade drawdown
FC_tr_eq = 0; 			// Trade equity (TWR)

gainer = 0;
loser = 0;

i_gainer = 0;
i_loser = 0;

for (i = startdateBI; i <= enddateBI-hold_days; i++)
{
if(C[i + hold_days] > C[i])
{
gainer[i_gainer] = i;
i_gainer = i_gainer + 1;
}
else
{
loser[i_loser] = i;
i_loser = i_loser + 1;
}
}

number_gainers = i_gainer;
number_losers = i_loser;

printf("Number Gainers: \t\t" + number_gainers + "\n");
printf("Number Losers: \t\t" + number_losers + "\n");

size = LastValue( BarIndex() ) ;

if(number_forecasts > size OR
number_days > size )
{
printf("Array not long enough, please select symbol with more data,\ncurrent symbol has " + size + " bars");
printf("\nPad data or select symbol with at least as many bars as \nnumber of days, trades and forecasts ");
}
else
{

////////////////////////////////////////////////////
//  Solve for fraction
fraction = 1;
done = False;

while( NOT(done) )
{
done = True;
printf("\nUsing fraction: \t" + fraction );
////////////////////////////////////////////////
//  Beginning a new forecast run
for( i_forecast = 0; i_forecast < number_forecasts; i_forecast++)
i_day = 0; // i_day counts to end of forecast
// Daily arrays, so running history can be plotted
// Starting account balance
account_balance[0] = initial_equity;
max_IT_Eq[0] = account_balance[0];
max_IT_DD[0] = 0;

{
// Select the trade and retrieve its index
// into the price array
// gainer or loser?
// Uniform for win/loss
gainer_loser_random = mtRandom();
// for long positions, test is "<"
// for short positions, test is ">"
if( gainer_loser_random < system_accuracy )
{ // choose a gaining trade
gainer_index = floor( mtRandom() * number_gainers );
entry_index = gainer[gainer_index];
}
else
{ // choose a losing trade
loser_index = floor( mtRandom() * number_losers );
entry_index = loser[loser_index];
}

// Process the trade, day by day
{
{
number_shares = account_balance[i_day] * fraction / buy_price;
cash = account_balance[i_day] - share_dollars;
}
else
{
i_day = i_day + 1;
profit = number_shares * ( C[j] - buy_price);
MTM_equity = cash + share_dollars + profit;
IT_DD = (max_IT_Eq[i_day - 1] - MTM_equity) / max_IT_Eq[i_day - 1];
max_IT_DD[i_day] = Max(max_IT_DD[i_day - 1], IT_DD);
Max_IT_Eq[i_day] = Max(max_IT_Eq[i_day - 1], MTM_equity);
account_balance[i_day] = MTM_equity;
}
{
sell_price = C[j];
if( i_day >= number_days )
{
FC_max_IT_DD[i_forecast] = max_IT_DD[i_day];
FC_tr_eq[i_forecast] = MTM_equity;
}
}
}

}
}
sorted_FC_max_IT_DD = Sort(FC_max_IT_DD, 0, number_forecasts);
DD_95 = sorted_FC_max_IT_DD[ floor(0.95 * number_forecasts) ];
printf("\tDD95: \t\t" + DD_95);

if( abs(DD95_limit - DD_95) < accuracy_tolerance )
{
done = True;
}
else
{
fraction = fraction * DD95_limit / DD_95;
done = False;
}
}
sorted_IT_DD_95 = Sort(FC_max_IT_DD, 0, number_forecasts);
IT_DD_95 = sorted_IT_DD_95[ floor(0.95 * number_forecasts) ];
printf("\nDD95: " + IT_DD_95);

years_in_forecast = forecast_horizon / 252;

sorted_TWR = Sort(FC_tr_eq, 0, number_forecasts);
TWR_25 = sorted_TWR[ floor(0.25 * number_forecasts) ];
TWR_50 = sorted_TWR[ floor(0.50 * number_forecasts) ];
TWR_75 = sorted_TWR[ floor(0.75 * number_forecasts) ];

CAR_25 = 100*(((TWR_25/initial_equity) ^ (1/years_in_forecast))-1);
CAR_50 = 100*(((TWR_50/initial_equity) ^ (1/years_in_forecast))-1);
CAR_75 = 100*(((TWR_75/initial_equity) ^ (1/years_in_forecast))-1);

printf("\tCAR25: " + CAR_25 + "  CAR50: " + CAR_50 + "  CAR75: " + CAR_75 + "\n");
/* //verify sorting works
printf("index95: " + floor(0.95 * number_forecasts) + "  index25: " + floor(0.25 * number_forecasts) + "  index50: " + floor(0.5 * number_forecasts) + "  index75: " + floor(0.75 * number_forecasts) + "\n");
for(i = 0; i < number_forecasts + 2; i++)
{
printf("\n " + i + " " + sorted_IT_DD_95[i] + " " + sorted_TWR[i] );
}
*/
}
``````

#19

Cesare,

The difference in CAR (but not in safe-f) may be due to dividend assumption in the database between your setting vs. Howard’s. If you use Norgate as your data source, for example, you can see the difference in results between using total returns (e.g. adjust for splits and dividend and special distributions) and just using split adjusted returns quite easily by changing the settings in Database configuration. And as Tomasz has pointed, not coding is required to compute either Safe-f or CAR25 – it’s part of the built-in backtest report.

#20

CAR25 as defined by Howard Bandy is not available in the built-in backtest report without manually (or using batch) re-running the backtest multiple times to get the 95th percentile MaxDrawDown to match your desired drawdown tolerance. Only then is the 25th percentile compounded annual return equal to CAR25.

My backtest code in the post above is forced to not use the MC built in to Amibroker because each Custom Backtest can only run the MC analysis once. Believe me, I tried finding a solution using the built in MC.

edit: here’s a comparison of the built in MC versus the output of the code above.

#21

Jani is right about the Safe-f computation, Cesare. For now, you probably
need to do your rough “Newton-Raphson” by batching a few runs with
different f to approximate what the safe f is. It should not take too many
iterations to figure out roughly the Safe-f is. To be specific, you can
decide that your f values should just be multiples of 0.1 (or 10%). Then
for practical purpose, you have at most 10 batch runs to do, assuming that
your basic position sizing rules related to portfolio initial value is not
unreasonably small.