Multi-Thread vs Single Thread Backtest Results are Different

Using multi-threaded portfolio (21 tickers) backtest, every time I execute my custom backtest formula, I receive a different number of trade results (sometimes I get 3 or 4 or 5 or 7 trades, etc.) The results never exceed my equity or position size or number of position holdings. Tickers that show up across multiple backtests have the same results.

When I run my formula with the “#pragma maxthreads 1” directive, (and continue using the same settings), everything works as expected, and EVERY time I execute my custom backtest formula, i receive precisely the same result (e.g. 12 trades and the same results.)

I do use static variables in my formula. All settings (according to back test report) are identical.
What could I be doing wrong?

You really need to carefully read the manual:
http://www.amibroker.com/guide/h_multithreading.html
especially part “Efficient and correct use of static variables”

Your code using static variables is simply wrong (serially dependent). For example codes like this are wrong:

// WRONG code (sensitive to multi-thread race conditions)
x = StaticVarGet("x");
// between get and set OTHER thread may be accessing the very same variable
StaticVarSet("x", x + 1 );

correct code

StaticVarAdd( "x", 1 ); // this is ATOMIC operation -i.e. SAFE
3 Likes

@Tomasz Thank you.
In an Analysis, using multiple threads, can I safely conclude that one symbol/ticker is analyzed by ONE thread at a time? Or is it possible multiple threads will concurrently operate on the same symbol?

My code typically uses the format of StaticVarSet(NAME()+“myVarName”, 1) for protection. I omitted the Name() prefix in my new formula.

It is described in great detail in the manual and answer to your questions is at the very beginning of it: http://www.amibroker.com/guide/h_multithreading.html

@Tomasz Among other information, a semaphore method" is described in the Efficient use of multithreading for enabling multiple threads to update the same variable.

However, I did not find suggestions to how to preserve data, generated during the backtest, when running several multi-thread analyses (and possibly several analysis runs using the same formula and watchlist).

For now, it appears to me that localizing my static variables with StaticVarSet(Name()+“varname”,x) works as long as I run one copy of my backtest formula at a time.

Thank you for the extensive documentation. I do take the time to search for and read the materials. The specific examples are very helpful!

@SwingTradeMonkey If you need to execute multiple backtests of the same formula concurrently, you can safely do that with multiple instances of AmiBroker. Each will have separate static variables. Also be extremely careful if you have formulas in charts writing to the same static variables. A chart refresh during a backtest could change the contents of a static variable and impact your backtest.

2 Likes

@steve Thank you for the valuable insight. Separate instances of Amibroker is a great workaround.

Static variables are global program wide. Obviously if you use the same name you would refer to same variable and if two backtest running simultaneously would use same static variable they would interfere. Like two kids in kingergarden wanting to play with the very same toy. That is obvious. You have to have unique names (each kid playing with his own toy).

1 Like