MonteCarlo on randomized OHLC and timeframes

PWFores · May 23, 2018, 7:05pm

I red this.

https://www.amibroker.com/guide/h_montecarlo.html

I understand Monte Carlo changes the order and number of trades to find
estimations of more or less lucky combinations. You can randomize score or indicator result to change and randomize how the orders take place in a portfolio.

But what if I not only want to randomize orders but do the monte carlo calculations on the resulting trades itself that have randomized data, parameters or even timeframes?

For example when 60 min data is used for optimizing with the custom metric and Monte Carlo, the optimized parameters could work better OOS when the parameters setting also work to a lesser degree on 30 and 120 minutes timeframe. Also, when a same parameter setting also works for example at 80% NP of the original if the OHLC data is for example 0.1% randomized it could work better OOS. The last example is how the parameter settings would perform if the optimized parameters itself would change +/- 20% and how much that resulted in OOS degradation. I understand this could be a very timeconsuming calculation but it sure can help OOS results.

I use this in other software and wonder how I could do this in Amibroker. I could not find it directly in the manual as in Amibroker monte carlo seems to be based on orders only if I am correct (please correct me if I am wrong) . I hope someone could guide me in what direction I should look lower level and if this would be possible.

Thanks for your help

Tomasz · May 24, 2018, 12:20am

You can just run simple optimization with random component added to your data.

// place this on top of the formula
dummy = Optimize("dummy", 1, 1, 1000, 1 ); // for 1000 optimization runs 
// +/- 0.1% randomization of prices
price_random = 0.999 + 0.002 * mtRandomA(); /* creates a range between 0.999 to 1.001 */

O *= price_random;
H *= price_random;
L *= price_random;
C *= price_random;

The same technique can be used to randomize any value.

PWFores · May 24, 2018, 12:55am

I already tried that but that is not what I mean.
This will randomize the optimized results before Monte carlo
and after that do a Monte carlo on the trades from what I understand.
(I hope I am wrong)

If I randomize before Monte carlo I still can have a perfect curve fitted
result as you can still perfectly fit 1 time on a randomized OHLC data.

I would like the Monte carlo to run for example 1000 times and only then
randomize the OHLC data and then have a CAR30 etc in order to
find if the optimized results are still valid enough.

It will be much more difficult to have curve fitted results when running
1000 times the same parameters setting on randomized OHLC data.

Thanks

Tomasz · May 24, 2018, 12:59am

This is separate and independent from built-in monte carlo run.
The idea is to run OPTIMIZATION that would be run 1000 times (or more) to use randomized prices. The results will be optimization results themselves, not what you see in normal MC post processing that occurs after backtest.

I wrote about such technique here: http://traders.com/documentation/FEEDbk_docs/2009/03/TradersTips.html#TT3

PWFores · May 24, 2018, 1:34am

I understand but when needing to do an optimization with for example 100 found results of 10 optimized parameters means 100 times typing in 10 parameters in the AFL and review the results separately. I hope future version of Amibroker would at least avoid you to type in the optimized parameters every run.

It is not like a walk forward with Monte Carlo and have the best result on a custom based criteria. I don't optimize only parameters but also random generated OHLC patterns and some indicators out of a list of 20 indicator. (works great with Tribes)
I am just generating trading systems and there will be many false positives which you can filter out for a very big part by the conditions mentioned.
There are several other programs for doing this much more efficient and there is machinelearning but I found it also works with Amibroker only needing many times more interaction.

I am looking for the top 100 best after running for a day to find out how they do OOS with Monte Carlo and randomizing, changed timeframes etc before I trust any of them OOS. But that is extremely time intensive.

Some programs do this OHLC randomize and multiple timeframe analysis in the same Monte Carlo analysis which works very fast and efficient and it allows you to find OOS results in the first optimization with Monte Carlo.
For example in a certain program this option is called:
"Use the Monte Carlo simulation to generate more robust strategies by introducing noise to the price history"

Is there a way to achieve this in the Custom backtester?

Tomasz · May 24, 2018, 9:14am

Monte Carlo is a broad term. You've asked to randomize OHLC and the technique I given does just that - adds noise to OHLC price history and performs user-definable number of runs with randomized data. That is what MC is about. Running bunch of simulations and getting distribution. The technique I described above and in Traders' Tips can be used to randomize any parameter any price and get a whole bunch of results that you can analyse for distribution.

The built-in Monte Carlo process http://www.amibroker.com/guide/h_montecarlo.html that is run as additional step (post-processing) is separate and independent from that.
It is doing one of TWO things (selectable by the user):

either pick randomly trades from generated trade list (the random element is picking up the trades) OR
randomly pick price changes from generated equity.

There are NO OHLC prices at that stage. There is only trade list and/or portfolio equity.
In both MC modes there is already random element (random picking). If you wanted to add second random element, the only thing you could do at this stage is to add noise to equity. If that is what you want, I can consider adding such option.

PWFores · May 24, 2018, 3:56pm

Oke thanks. Confirmations tools is more important when generating systems automatically or doing Machinelearning. Randomize OHLC data is often not enough, testing on multiple timeframes and small random earlier optimized parameter changes are often very usefull.

There is one more way of testing robustness which I know of. I have a program that generates many systems and after that does a Walkforward . This program looks at the change in parameter in the different walk forwards of the same system and based on the stability of parameter it assumes that the system and parameters is robust. That is another way of finding stable parameters, but I don't find this works very well. You can still be in a peak.

beppe · May 24, 2018, 8:54pm

(OT but related...) Here is an interesting article titled Fooled By Monte Carlo Simulation.

By the way, the author uses AmiBroker for charts and trading systems simulations.

PWFores · May 24, 2018, 9:12pm

That is my experience too, trade reshuffling doesn't work very good like this article suggests.

That's why I am looking for other ways of doing Monte Carlo as written here.
I have seen much better results in that case.

This writer Mike Harris has a very good pattern software Price action lab also available for Amibroker.

aron · May 25, 2018, 7:20am

If you could explain the process step by step we would be able to help.

aron · May 25, 2018, 8:23pm

Here is a snippet that adds some noise to 2 different strategies with the intention to find the one to stick to .
noise applied to :

trade prices - by adding a variable spread to entry price
execution - by skipping entries and random position closing

the code first is run in optimization to collect the trade lists and perform MC then in exploration do display results. The output is a matrix of metrics ( return and drawdown ) , respective strategy and variations.
Enjoy.

SetOption( "priceboundchecking", false );
SetOption("ExtraColumnsLocation", 1 ); 
variations = 3;
strategies = 2;
j = Optimize( "variation", 0, 0, variations - 1,  1 );
k = Optimize( "strategy", 0, 0, strategies - 1, 1 );
Buy = Sell = Short = Cover = 0;

switch( k ) // strategies
{
case 0 : // strategy #1
    Buy = Cross( C, Ref( High , -1 ) );
    ApplyStop( stopTypeNBar, stopModeBars, 1 );
    break ;

case 1: // strategy #2
    Buy = Cross( C, Ref( C, -1 ) );
    ApplyStop( stopTypeNBar, stopModeBars, 1 );
    break ;
}

switch( j )// variations
{

case 0: //randomimze entry price
    buyprice *= 1 + 0.002 * mtRandomA();
    break ;

case 1:// skip entry
    Buy *= mtRandomA() < 0.2;
    break ;

case 2: // randomly close position
    Sell = mtRandom() < .1;
    break ;

}

PositionSize = 10000;


if( Status( "actionex" ) == 13 )
{

    StaticVarRemove( "*" );
    StaticVarSet( "counter" , 0 );

}

runs = strategies * variations ;
SetCustomBacktestProc( "" );

if( Status( "action" ) == actionPortfolio )
{
    bo = GetBacktesterObject();
    bo.Backtest( 1 );
    bo.ListTrades();


    list = StaticVarGet( "temp" );

    if( IsNull( list ) )
        list =  Matrix( 100000 , runs , - 100 ) ;

    i = 0;
    counter = StaticVarGet( "counter" );
    //_TRACEf ( "strategy %.f\t variation %.f", k , j  ) ;

    for( trade = bo.GetFirstTrade(); trade; trade = bo.GetNextTrade() )
    {
        list[i][counter] = trade.GetPercentProfit;
        i++;
    }

    StaticVarSet( "temp", list );


    if( counter == runs - 1 )
    {
        StaticVarSet( "TradeLists", list );
        StaticVarRemove( "temp" );
    }

    StaticVarAdd( "counter", 1 );
}

list = StaticVarGet( "TradeLists" ) ;

if( list )
{

    m = MxGetSize( list, 0 ) ;
    n = MxGetSize( list, 1 );

    // MC parameters
    metrics = Matrix( n , 4 );
    Random_samples = 100;
    simulations = 100;
    rank = 25;


    j = strategy = var= write = 0;

    for( ; j < n; j ++ )
    {

        item = MxGetBlock( list , 0, m - 1, j , j ) ;
        len = MxSum( item != -100 ) ;
        item = MxGetBlock( item , 0, len - 1, 0 , 0 ) ;
        item = item / 100 + 1;

        results = Matrix( simulations , 2 ) ;
        i = 0;

        for( ; i < simulations ; i ++ )
        {
            e = maxe = 1;
            mdd = 0;
            count = 0 ;

            while( count < Random_samples )
            {
                r = floor( mtRandom() * len ) ;
                e *= item[r][0];

                maxe = Max( maxe, e ) ;
                mdd = Max( mdd , maxe / e - 1 );

                count ++;
            }

            results [i][0] = e;
            results [i][1] = -mdd;
        }

        results = MxSortRows( results , True ,  0 );
        //results = MxSort ( results , 1 );

        p = floor( rank / 100 * simulations );

        metrics[j][0] = results[p][0];
        metrics[j][1] = results[p][1];

        if( J % variations ){
            write = 1;
            var ++; 
         }
        else
            if( write )
            {
                strategy ++ ;
                write = var =  0;
            }

        metrics[j][2] = strategy;
        metrics[j][3] = var;
    }

    _TRACE( "25-th Percentile {Return25, MaxDD25, Strategy, Variation}" );
    _TRACE( MxToString( metrics ) );

    SetOption( "NoDefaultColumns", True );
    Filter = 1;
    
    m = MxGetSize ( metrics , 0 ); 
    n = MxGetSize ( metrics , 1 );
    columns = "25-th Return,25-th MaxDD,strategy,variation";
    for ( i = 0 ; i < n; i ++ ) 
    AddColumn( null, StrExtract ( columns , i ) );
    

    for( i = 0; i < m; i++ )
    {
		row = StrFormat( "%.2f\t%.2f\t%.f\t%.f", metrics [i][0], metrics[i][1],metrics[i][2], metrics[i][3] );
        AddRow( row );
    }

}

PWFores · May 25, 2018, 8:56pm

Thank you so much for putting so much effort in this.

rocketPower · May 27, 2018, 12:34am

Hi

That's a great article. Trade reshuffling always bothered me intuitively unless you have a market neutral strategy. As somewhat of a trend follower that depends on market state, reshuffling trades creates market conditions that may never be representative of real life. When backtesting a strategy, I want to see robustness in:

Parameter tuning: Even if you optimize, testing a wide enough range of values around the optimized parameters and creating many return profiles should give you an idea of robustness. This gives you a multi-dimensional return profile based on all tunable prarameters. Sharp peaks and valleys may indicate a problem
Slippage randomness. If perfect entry/exit is your accidental edge you will soon find out
Skipping buy/sell signals: Some returns look really great because your strategy holds onto a couple of monster stocks by luck. Skipping this trades randomly will let you see if it is able to catch other signals that give similar profit

Anyway glad someone if pointing out the limitations of MC.

rocketPower · May 27, 2018, 12:36am

PWFores,

I agree with @aron, if you can give high level steps of what you are trying to accomplish, then maybe our answers will be more on point to your needs.

Thanks!
Mike

PWFores · May 27, 2018, 12:57am

Here an example of how it works in another program.
This is based on requests from users of that program.

I deleted the name and I hope I am allowed to post it here.
Multiple timeframe analysis is also a good check.

MONTE%20CARLO

rocketPower · May 27, 2018, 1:16am

Hi,

So you're looking to augment this strategy by checking adjacent timeframes as well?

I've never personally done that in AB, but if you had access to that data, it definitely seems plausible. @Aron's code represents well what this code is trying to do. You can use that code and try it on every timeframe you wish.

PWFores · May 27, 2018, 1:25am

You can do that in Amibroker as well.
Tools/preferences/intraday and set some custom timeframe like 55 or 65 minuts if you use hourly data.

If you don't have enough data or you like to have a small testset representing the current market it is also possible (in other software) to resample using another start interval. Like hourly data based on 1-61 61-121 or 2-62 62-122 minuts etc so you can have >50 times more bars. That doesn't make sens for daily analysis but for intraday it sure can.
Some machinelearning programs use that to generate more data and they have better results in that case. I can't post a link of that here.