Real life execution not matches backtest results

[anupam]

Am using a simple stochastic RSI + macd strategy to trade intraday. The backtest results are good but the actual results don't match with backtests. In fact the results are very different to the backtest results.
Have done code checking and its fine and the code doesn't reference future quotes.

I save amibroker scan results to a text file and then use python to trade thru broker API. I place bracket orders when a signal is generated thru amibroker. Exits are when stoploss or targets are hit and I don't use amibroke exit signals. Python code is on 10 sec loop and amibroker scan is 1 min loop. Am trading on 15 min timeframe. Have also tried on 5 min but results vary there as well.

Request seniors to help and guide as am new to amibroker.

Here is my code:

_SECTION_BEGIN("Stoch 15 min Cross");

SetChartOptions(0,chartShowArrows|chartShowDates);

FirstTradeTime = 100000; // Earliest time to take a trade
LastTradeTime = 143000; // Latest time to take new trades

ExitAllPositionsTime = 150000; // Exit all trades

r1 = Param( "Fast avg", 12, 2, 200, 1 );
r2 = Param( "Slow avg", 26, 2, 200, 1 );
r3 = Param( "Signal avg", 9, 2, 200, 1 );

ml = MACD(r1, r2);
sl = Signal(r1,r2,r3);

Hist = ml-sl;
MACDN = (Hist < Ref (Hist,-1)) AND Hist < 5 ;

periods = Param( "Periods", 14, 1, 200, 1 );
Ksmooth = Param( "%K avg", 3, 1, 200, 1 );
Dsmooth = Param( "%D avg", 3, 1, 200, 1 );

k15 = StochK( periods , Ksmooth);
d15 = StochD( periods , Ksmooth, DSmooth );
k15_1 = Ref (k15,-1);
k15_2 = Ref (k15,-2);
d15_1 = Ref (d15,-1);
C12= Cross(d15_1,k15_1) AND ((k15_1<k15_2)) AND IIf((d15_1-k15_1)>(1.1),1,0) AND k15>10 ;

Short = (C12 AND MACDN) AND (TimeNum() >= FirstTradeTime AND TimeNum() <= LastTradeTime );
Cover = (TimeNum() >= ExitAllPositionsTime);

Short = ExRem(Short,Cover);
Cover = ExRem(Cover,Short);

Capital = 2000000;
Risk = 1000;

filepath = "Z:\Downloads\stocks\222.txt";
fh = fopen( filepath, "a", True );
if ( fh )
{
lastbuyDT = LastValue( ValueWhen(Short, DateTime() ) ) ;
str5= StrLeft(DateTimeToStr( lastBuyDT ),5);
str2= StrTrim(str5,"/");
str3 = StrRight(DateTimeToStr( lastBuyDT ),11);
str4= StrTrim(StrLeft(str3,8)," ");
str1 = str2+str4;
str6= StrTrim(NumToStr(Close)," ");
fputs( Name() +";short;" + str2+";"+ str4 + ";"+ str6 + "\n", fh );
// close file handle
fclose( fh );
}
else
{
_TRACE("Failed to open the file");
}

capital = Optimize ("Capital",16000,70000,200000,10000);
SetPositionSize (capital, spsValue);
_SECTION_END();

[Sean]

You should be more specific as to what "Don't Match" means. Are the symbols traded different and/or are the symbol trade prices different for the same symbol and/or is the trade size different for the same symbol?
What is not matching exactly etc.
-S

[anupam]

I mean the profits in real life is far less than the profits in backtest results. I am in process of checking the exact trades which is symbols, entry and exit prices. Have kept the trade size as same.

[ghanson]

@anupam, since you're new here it would be wise for you to review the following thread:
How to Ask a Good Question
Be sure to read Item 3 regarding the use of code tags. This will make it easier for others to help you.

[anupam]

I checked all trades, entry and exit price qty etc. the problems is that many of the trades which am taking as per scan results are not showing in backtest results. Understand that backtest results take into account capital allocated, individual position sizes, max no of open positions etc.. hence am putting a large capital, with very small individual position size and almost infinite no of open positions allowed. Still many of the trades in scan results are not showing in backtests and hence the profits in real life is far less than compared to backtest results. Can anyone please throw some light on what am I missing?

[mradtke]

You might benefit from using the Detailed Log for your backtest output, as that will show you which signals are being received during the second phase of the backtest, and why some trades are taken and others are not. Look on the Report tab of the Analysis Settings to set Detailed Log output.

[Sean]

You still have not specified what doesn't match exactly.
I.e.,...
1.) The whole trade doesn't exist?
2.) The trade exists, but entry or exit prices are different?
3.) The trade sizing is different?
4.) Trade happens, but at a different time/date offset than the backtest?
5.) Any combination of the above.

Regarding #1. Computer code often works off lists, arrays and structures which in the code behind the backtests exist in some type or order (unless otherwise implemented), and are sequential in nature. Meaning for example a list, array or structure might be Alphabetically sorted and then one after another symbol is check for something in code one after another.

So what does this mean? Well, let us say the symbol universe you are testing exists in a alphabetically sorted coded object of some type deep in AB's architecture. Then when testing, it starts at the top of the list and continues down looking for system entries whereby all your logic is satisfied and then looks for a cross of price data and thus a hypothetical trade fill. That possible fill and position then goes into a new list, array, structure for that interval maybe and that new list is dependent of the initial list et al. As the logic progresses this means that trade orders are being checked alphabetically with resulting hypothetical trade fills and positions (yet a new list) which come back in a "Not as it necessarily happened in real life Order", until such time as limited cash, margin or self imposed rick ceiling is reached. If that happens then the rest of the alphabetical symbols in your symbol universe will not be checked, or at least not hypothetically filled or giving you a trade position; because you have no cash, margin or risk limit left at that "Hypothetical ordered point".

This ordered nature of coded lists, arrays and structures can even affect how much (sizing of position) a trade gets (#3) based on where that instrument is on the list. Meaning for example that particular ordered instruments used cash, margin or risk that leaves less for the next instrument further down the list and as a result that later instrument gets less potential position sizing then it would have if it happened to be higher up that instrument list, in the A's.

This can also be in play and cause a #4 situation, whereby there you might have reached a limit in cash, margin or risk availability and for that reason the trade for an instrument was passed on and skipped. But then for what ever reason later in time from an exit somewhere else some cash, margin, risk was freed up and since all rules for the system for that instrument (that had initially been skipped) was still a "GO" for a trade (Buy=true), that trade happens later in time and looks like it was moved forward and offset into the future in the backtest etc.

In real life orders go out in an order, but fills come back in the order they filled/executed and not the order they went out. This can lead to a huge discrepancy between system and real-life trades, especially the lower the resolution of data, i.e., daily-weekly-hourly bars. With such high correlations nowadays this can be the case even at minute resolutions. For example 3yr, 5yr, 10yr, 30yr on economic news releases on Tuesdays and Thursdays. My rules say I can only risk .2 % on just one of those, because of correlation rules. Assuming I have enough cash and margin I have stop with limit buy orders across the board that pin the ever moving bid. On my first fill I try to cancel any remaining orders or exit the less strong fills or imeditely reduce positions so the aggregate risk across those instruments meets my 0.2% risk. My backtest gives me a position in the one at the top of my alphabetically sorted list. But which fill in real life do I actually get in real-time/life?

Just remember we live in cash, margin requirement limitations and self imposed risk ceilings, so be aware of any ordered lists that eat into those in a fake order.

I always recommend, even for very long term models and systems to test it on the highest granularity like tick and on a global clock, where every instrument is aligned to the date-time line and aligned for different time-zones AND dates. This will allow you to see the order as closest to when things actually happen chronologically in real life. The other more less and highly varied way is to randomize the orders placed for symbols in the backtest. Via AB this would use the ability to randomize AB instrument ranking feature. But that will give a huge swath and wide ranging results of worst order to best order, keeping in mind not the order thing actually happen at that time.

It's all about ordered processing in code that you have to very careful of, especially when doing portfolio level testing where you have constraints that prohibit you from trading "ALL" trades.

To make matters even worse, imagine an exit that is checked in a "order" that frees up some limitation that then puts a new trade or trade adjustment in play, which is also on its own is in an ordered situation in the backtest! With exponential growth of trading systems and this butterfly effect, you really need to drill down to see what is going on.

Happy Turkey.
-S

[anupam]

Thanks Sean for a detailed explanation. My issue is point no 1 which is some of the trades from scan results are completely not showing in backtest results. Am not worried about the difference in entry/ exit prices because it will be there, considering the time lag from scan result generation to actual order placement specially because I have my own python program running in between.

I analysed and one of the reason why trades are not showing may be because am using stochastic on current bar and I use 5 minutes/ 15 minutes. So if the signal is generated in 1st minute because the condition on stochastic was fulfilled at that moment but may be after that the price changed and the stochastic condition got unfulfilled at the end of the current bar at 5th/ 15th minute. Hence it will not show on backtest which will take a closure value but the scan takes the real time value in middle of the bar. It should have been flagged in code check that its looking forward in future if am taking buy price as bar open price in the backtest setting. but I don't get a message like that.