MDD in WF Out-of-Sample summary report

[jptrader]

Hello,

When it comes to the WF Out-of-Sample summary report, I have noticed that the Max. System Drawdown ($ or %) does not represent that value for the whole test, as shown in the following screenshot.

In #10 (summary report OOS 1997-2023) MDD is - 11,72%. However in #9 (2021-2023) MDD is higher, up to -13,48%.

I have read the following entry from Amibroker Guide: Walk-forward testing and optimization. Nevertheless, after several attempts and search of examples I have been unable to implement a bo.AddCustomMetrics() call that provides the MDD for the whole OOS test in the summary report.

I should appreciate it if someone would let me know how to solve this.

Thank you.

Regards.

[Tomasz]

You are making typical mistake thinking that percents profit / drawdown works the same as dollar profit / drawdown.

It doesn't.

For reasons explained here: Math 101: Is profit table correct? (Why adding percents leads to nowhere)

If say equity moves from $10000 down to $8000 that is down $2000 and that calculates to 20% (2000/10000) drawdown. Then when equity later moves from $20000 down to $17000 that is down $3000 (i.e. MORE drawdown), the percentage is only 15%. More dollar drawdown, less percentage.

Producing summary report is not just taking those numbers and adding some and taking max. It is much much more complex, since summary report must be doing "compounding" of OOS periods.

[jptrader]

Hello Tomasz,

Thank you for your answer and explanation.

I understand your point, but from my point of view the example you have given is also applicable to a common backtest. In that case, the report would show 20% as %MDD, even if it corresponds to a $2000 DD and there are some greater $DD as the one for $3000 that corresponds to 15% as %MDD.

What catches my attention the most is the fact that on the second page of the report, the graph does show the %MDD for the entire date range, but the numerical value on the first page of the report does not match.

So let me restate my doubts:

1. If the system calculates the %MDD (as we are used to in a common backtest) and represents it in the graph on the second page, is it possible in some simple way to call that value to the first page of the report?

2. What represents the %MDD value that is shown as standard on the first page of the report. I mean the one that, as you have indicated, is obtained by composing the OOS periods? What are the $DD and the $Equity involved in the calculation?

Thank you for your time and attention.

Best regards