Median Yearly Profits Custom Metric Crashes Only in Walk-Forward Tests

Hi.

I'm trying to create a custom metric that calculates the median yearly profit of a backtest. The idea is that this ensures that the profits are reasonably stable throughout the backtest, instead of concentrated in one part of the backtest.

My custom backtest code is below. It works for backtests and optimizing, but for some reason it usually crashes and gives me an "error 47" when doing walk forward tests. It doesn't crash immediately though, it does a couple years worth of testing before the crash. The crash always occurs in 2001, which is also the first year in the backtest when my strategy spends the entire year completely in cash. I don't know whether that's happenstance or not.
image

When I put the period for the yearlyROC to 50 instead of 252 it did not crash. Even when using 252, clicking "Try to Recover" after the crash makes it continue and complete the test without issues.

The code gives a median of 0 if the number in BarCount - 252 is anything other than identical to the period used in the yearlyROC variable.

SetCustomBacktestProc( "" );

if( Status( "action" ) == actionPortfolio )
{
    bo = GetBacktesterObject();
    bo.Backtest();

    eq = bo.EquityArray;
    yearlyROC = ROC( eq, 252 );
    medianROC = LastValue( Median( yearlyROC, BarCount - 252 ) ); //If I remove this line, and switch the custom metric to yearlyROC it stops crashing

    bo.AddCustomMetric( "Median", medianROC );
}

Any ideas how to fix this? Is the way I calculate the median super inefficient or is there some other problem?

I have windows 10, 32 GB of RAM, Amibroker 6.40, NorgateData.

Your formula is incorrect. To get yearly median you should just pass 252 to Median, NOT BarCount - 252.

LastValue( Median( yearlyROC, 252 ) ); // DO NOT use BarCount here

If you are getting "Crash recovery" dialog, you need to send us full crash report.

Please do not send screenshot of this dialog to us as it only contains a small portion of information required
to help you. Instead of sending screenshot, press "Copy to Clip" button and
paste the text to your email. This way entire report will be included.

Also, it is good idea to always use latest version of AmiBroker.

If you want us to check futher we need reproducible scenario data

  • the formula that you used
  • the data that you used
  • steps you did
    that would allow us to reproduce crash in development environment.

No, your code will not give you median of Yearly profits. Your code will give you median of 252-bar profits calculated EVERY BAR (every day). You are calculating ROC every bar, not once per year. Periodicity of your input is DAY, not year, and your median is not from YEARLY profits but from moving 252-bar ROC calculated daily.

Profit table on the other hand calculates truly YEARLY profits (ONCE per year, not everyday).

Last but not least: you did not do what I wrote. I wrote

Instead of sending screenshot, press "Copy to Clip" button and
paste the text to your email.

You were supposed to send the details BY EMAIL to SUPPORT, not forum.

Forum is for "between users" help. No-one here is going to help you with that.

So again, WRITE TO SUPPORT.

Also, please send the formula WITHOUT NORGATE FUNCTIONS.

Support inquiries' formulas must not require any 3rd party plugins. We don't use Norgate here or any other 3rd party plugins. Only plugins originally shipped with AmiBroker can be used for inquiries .

I have minimized your code to

SetCustomBacktestProc( "" );

if( Status( "action" ) == actionPortfolio )
{
    bo = GetBacktesterObject();
    bo.Backtest();

    eq = bo.EquityArray;
    yearlyROC = ROC( eq, 252 );
    medianROC = LastValue( Median( yearlyROC, BarCount - 252 ) );

    bo.AddCustomMetric( "Median", medianROC );
}

Buy = 1;
Sell = 1;

and it does NOT crash on my end.

I guess that your BarCount may be incorrect (less than 252) and your calculation is yielding negative number and you in turn pass NEGATIVE 'periods' to Median function which should never happen.
You should either check the BarCount if it is really greater than 252, or protect calculation (BarCount - 252) from going negative.

To protect against passing amount <= 0 you should use this

safePeriod = Max( 1, BarCount - 252 );
medianROC = LastValue( Median( yearlyROC, safePeriod  ) );

The thing is that should you use Percentile, it will cry loudly about negative period, but Median is not doing this homework for you and will just pass the value that you provided to Microsoft's memcpy function that is the one that crashes.

1 Like

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.