Trade Results Differ

I was running some AFL strategies which I thought should give the same results whether I ran them on a daily time frame or a monthly time frame because they were always trading at the close of the month.

The open of the first day of the month is identical to the open of the month and the close of the last day of the month is identical to the close of the month. So if you are trading the same symbols at only the open or close of the month you should get the same results on either time frame. And you do sort of but not exactly. I'm wondering why the results are not exactly the same. What follows is a test program that shows the differences in the results.

//--- DayMonth Test
// Open of the first day of the month is identical to the open of the month
// Close of the last day of the month is identical to the close of the month
// A strategy that trades only one of those two times should produce
// identical results whether traded when Periodicity is set to Daily or Monthly
// This program was designed to test the results produced by Amibroker on both.
//=============================================================================

//--- Standard settings not changed 
	SetBacktestMode(backtestRegular); // Setup for back test type 
	SetOption("InitialEquity", 100000); // Starting dollar amount
	SetOption("AccountMargin", 100); // Cash % traded (100% = no margin, 75% = 25% margin used) 
	SetTradeDelays(0,0,0,0); // Disable Amibroker trade delays 
	SetOption("SeparateLongShortRank", True); 
	SetOption("UsePrevBarEquityForPosSizing", False); // Use current bar equity
	SetOption("EveryBarNullCheck", True); // No null bars allowed 
	SetOption("PriceBoundChecking", True); // Prices constrained to between low and high values
	SetOption("MaxOpenPositions", 2); // Only two trades at a time
	SetPositionSize(50, spsPercentOfEquity); // Always trade 50% of equity
	SetOption("AllowPositionShrinking", True); // Limit trade size based on available funds in account 
	SetOption("SettlementDelay", 0); // Broker dependent. Proceeds from sale only available for new trades x days after sale
	SetOption("MinShares", 1); // Minimum trade size 1 share
	SetOption("AllowSameBarExit", True); // Allow entering new trades at same time previous trades are exited 
	SetOption("ReverseSignalForcesExit", False); 
	SetOption("FuturesMode", False); // Stock trading mode 

//--- Idenditify last day of the month
MonthEnd = TimeFrameExpand(1, inMonthly, expandPoint); // True on only the last day of the month

//--- Calculate a position score to rank symbols for trading
MonthClose = TimeFrameCompress(C, inMonthly, compressLast); // Monthly price no matter what Periodicity is set to  
PositionScore = ROC(MonthClose, 3); // Positionscore always based on 3 month ROC
PositionScore = TimeFrameExpand(PositionScore, inMonthly); // Expand score to current time frame

//--- Buy and sell on last day of the month
BuyPrice = SellPrice = C; // Buy and sell on close
Buy = MonthEnd AND PositionScore > 0;
Sell = MonthEnd AND !Buy;

I understand why some of metrics differ like Profit/Bar, MAE, MFE, but not the Profit, % Profit, and Shares. I included a lot of SetOptions to make sure everything was identical except for the time frame.

I run this program over a range of a year or more on sector symbols first on Daily Periodicity and then again on Monthly Periodicity. The symbols, dates, and prices appear to be identical on both time frames and for the first few trades everything else related to trade size appears to match exactly, but as time and the trades progress the trade size and profits drift. They are close, but not exactly the same. And in the end the differences show up in the CAR which is an important metric. I don't understand why. They are trading at the same time on the same symbols and at the same price.

Is there a setting I don't have right or is there something else I'm missing?

Are you sure? I suspect that if you do a line by line comparison of the trade lists, you will find one or more trades which are not the same. It’s pretty quick to do the comparison in Excel.

Assuming you find a discrepancy, the next task is to look at the first trade that’s different in each list and try to figure out why the other list (timeframe) didn’t take that trade. PositionScore is a likely culprit, although looking at your code I don’t see an obvious issue.

1 Like

Hello,
Thanks for taking a look. All of the trades taken are identical except for the size and results. Attached is an example of the results for just one year.


As you can see the symbols, dates, and prices are all identical, but the Shares, Position Value, Cum. Profit start to not match as time goes on.

For additional reference here are the back test summaries from Amibroker. Both tests were from 12/31/24 to 12/31/25.

Profit = 7938.29 (7.94%), CAR = 7.94%, MaxSysDD = -6703.90 (-6.39%), CAR/MDD = 1.24, # winners = 3 (50.00%), # losers = 3 (50.00%) Monthly Test

Profit = 8136.16 (8.14%), CAR = 8.14%, MaxSysDD = -15455.37 (-14.65%), CAR/MDD = 0.56, # winners = 3 (50.00%), # losers = 3 (50.00%) Daily Test

As you can see the Profit and CAR are different. Not by a lot over just a year, but enough to notice.

Looking at the Detailed Log for both a monthly and daily test, I can see that the equity values are different on the bar that the first discrepancy appears. I believe that the reason is as follows:

  • When you are entering and exiting trades for a bar, AmiBroker does not yet know the closing equity for that bar. Therefore, your “current” equity used for position sizing will be an estimate.
  • For open positions, the current value might be determined using the Open price for the bar, or perhaps it’s an average of several prices, like (O+H+L)/3. For now, it doesn’t matter exactly how it’s done, only that it’s not valuing open positions based on their Closing price for that bar.
  • The OHL prices for a monthly bar will be much different than for a daily bar, so in the case of a monthly backtest, AmiBroker will not be able to value the open positions as accurately, i.e. using more recent data.

Obviously, different equity values will change your position sizing. And you can’t simply fix this by setting the option for “Use Previous Bar’s Equity for Position Sizing”, because the previous bar for a monthly test is much different than the previous bar for a daily test.

The reason this doesn’t cause even larger discrepancies in the results is that in many cases the position size will be limited by the amount of available cash. In those cases, both daily and monthly may end up entering the same number of shares. But as time goes on, the paths will diverge more and more, as the first different position size will have a ripple effect.

2 Likes

Thanks again for taking a look.

What you are saying about the price used to calculate equity must be the answer and I can see how that might explain the difference in open positions mid trade. But on closed positions it seems like the right price to use when calculating current equity would be the price found in the Ex. Price column or the Buy price and Sell price which is identical between both daily and monthly tests.

Based on your explanation I thought maybe I would get identical results if I left the trade size at 50% of equity, but limited the number of trades to just 1 instead of 2. But instead the difference in the resulting CAR was even greater. I understand why there is a difference in the MDD, but having a harder time understanding the Profit and CAR differences.

Profit = 4638.97 (4.64%), CAR = 4.64%, MaxSysDD = -7189.48 (-7.04%), CAR/MDD = 0.66, # winners = 1 (25.00%), # losers = 3 (75.00%) Daily Test

Profit = 3549.34 (3.55%), CAR = 3.55%, MaxSysDD = -3846.88 (-3.78%), CAR/MDD = 0.94, # winners = 1 (25.00%), # losers = 3 (75.00%) Monthly Test

Which one is going to be closer to what you would actually get in live trading?

As a further test I changed the number of trades to 3 and the position size to 25%. I got a daily CAR of 7.06% and a monthly CAR of 6.15%. Even bigger differences than with just two trades at a time.

As soon as the two trade lists start using different position sizes, their overall P/L (aka equity curve) will also become different. That in turn will effect all portfolio level metrics like CAR, MDD, etc.

Using daily bars for the backtest is more likely to resemble live trading, especially if you use Previous Bar Equity for position sizing and also use your prior day trading account balance for sizing live positions. Or you can use the backtest to generate your trading signals, including position size.

1 Like

First you have to validate your assumptions.

You assume that your daily and monthly input data are the same.
Validate it.

Run simple checksum code of every variable that you want to check as explained in the KB

The problem here is not with data on different computers, but rather with equity from back tests trading at the same time and at the same price but on different periodicities (daily and monthly). I don't understand how the checksum helps in this case. The data checksum results look okay to me.

I agree with mradtke that the difference in CAR and related metrics is due to a difference in the calculated equity. But I still don't understand why the equity differs since it seems like it would be calculated using the Price when entering and the Ex. Price whenever a trade was exited. I must not understand how the equity is calculated.

You are correct about the equity for closed trades. The issue is the equity attributed to trades that are already open (i.e. not yet closed) when you enter a new trade.

1 Like

Consider a simple example where you start with $100K and allow only two trades. For now, we will assume that the value of open trades on a given bar is calculated using the Open price of that bar.

  1. On Jan 31, you enter your first trade in ABC at a price of $100. The value is approximately $50k, and you have $50K of cash.

  2. On Feb 1 at the open, ABC price is up slightly at $105.

  3. On Feb 28 at the open, ABC price is down to $90

  4. On Feb 28, you have an entry signal for XYZ. How many shares will you buy?

    A. If you’re trading monthly bars, the value of ABC on Feb 28 will be $52,500 and your total equity will be 102,500. 50% of that is $51,250, so you will attempt to buy $51,250 of XYZ. However, you only have $50,000 of cash, so you will end up with a position in XYZ worth approximately $50K.

    B. If you’re trading daily bars, the value of ABC on Feb 28 will be $45K and your total equity will be $95K. 50% of that is $47,500, so you will buy $47,500 of XYZ because you have more cash than that available in your account.

So now you have a different number of shares of XYZ in your monthly backtest than in your daily backtest. This will repeat every time you enter a new trade while there are other trades open. But of course, after that first divergence, your closed trade equity will also differ between the two backtests, because you were trading different numbers of shares for some of the trades.

3 Likes

I know. I did read what you wrote.

I gave you link showing generic technique of checking ANY input by comparing checksums. It is general purpose technique. The link was about different computers, but I think that everyone is able to expand the use of checksum to ANY situation when you want to check if INPUTS are the same.

Your results are different because your inputs differ. There is a technique called "divide and conquer" where you divide a problem into small pieces and test one by one, individually. So running formula and calculating if all variables you are interested in have same checksums (so you don't need to compare everything line by line). For example your code compresses close, but still (implicitly) uses daily volume when running on daily data.

For example if you use "previous bar equity for position sizing" but that previous bar
bar will be once previous Month bar and the other way previous day. Trade sizes would then obviously be different even if buy rule is the same.

And again this is just example, I know that you turned this particular thing off. I did read the formula. I am just telling that various settings have various non immediately obvious consequences and general approach is to assume nothing.

Another example is this: there is a setting "limit trade size to % of bar volume" which by default limits trade size to 10% of bar volume.
When you run it on monthly data, the limit will be 10% of MONTHLY volume, if you run it on daily, the limit will be 10% of DAILY volume (and that refers to actual volume in the database). This will change your actually entered position size and that is why it is extremely important to compare "DETAILED LOG"

3 Likes

Your explanation made sense. It occurred to me to test this the problem with different entry times could be eliminated by changing the Sell to sell at the end of every month. So I tried that and it confirmed what you said. All of the trade sizes now match exactly between daily and monthly time frames.

So thanks for taking the time to explain. My conclusion from this is back testing is only going to give an approximation of the results in live trading. Whether monthly or daily is the better approximation may require more testing.

To test it I had to add a line SetOption("HoldMinBars", 1); and change the sell line to Sell = MonthEnd;.

1 Like

Thanks for looking at this. I did have limit trade size as % of entry bar set to zero. So that wasn't the problem. After further testing, I think mradtke came up with the correct explanation. The problem was using a percentage of equity as the trade size and having entries and exits staggered resulted in slightly different share sizes.