Order of Signals

Hi All,

Is it possible to specify the order in which the signals provided within a custom backtest are generated for each particular bar, or at least get them to match the signal order of the default backtest engine, as when I use a low-level custom backtest and compare the generated trade list with that of the strategy without the custom backtest I can see differences between the generated order of trades for the same bar?

I should add, I read that the...

Signal list is sorted so first entry signals appear (top ranked first) and after that scaling and exit signals follow.

But in that case the signals should be the same. Could it be down to how the maximum number of positions are handled?

Have you looked at the PositionScore variable?

You could also do a multi-column sort on each set of back test results, for example by entry (or exit) date and then by ticker symbol.

Hi Mradtke,

Thank you for your reply.

In both tests I am using the same PositionScore.

I have been analysing the trade list using multiple column sorting (by date and symbol) which does make it easier to see discrepancies between the built in backtester and the custom one I am using, but the default is the 'Cumulative Profit' column, and it's the summary stats that do not match completely even though there are identical trades?

The default sort order for my trade lists is from oldest to newest exit date. There's probably a way to change that, but I never bother because I can easily sort on any other column. However, it sounds like your issue is neither the order of the signals sent to the back tester, nor the order of the trades in the trade list. The problem you're trying to solve is why your CBT produces different results than the built-in back tester, correct?

I would begin by manually comparing the entry date, entry price, exit date, exit price, position size, and scale ins/outs for the first few trades in each trade list. If you are not finding the reason for the discrepancy in your overall results, then I would pull both trade lists into Excel and do a more comprehensive search for differences. There may be better ways, but here's how I usually do that:

  1. Put each trade list into a separate sheet in Excel
  2. Add two columns to each trade list: Key and Match
  3. The Key column should contain a unique string to identify the trade. For starters, you might concatenate the ticker symbol, entry date, and exit date. If that's not enough to root out small discrepancies, you might need to add scale-ins or % Profit.
  4. In the Match column of the first trade list, use the Excel Match() function to look for the current key value in the entire column of key values in the second trade list. In the Match column of the second trade list, use Match() to find a matching key in the first trade list.
  5. If you've done this correctly, the Match() function should return a number when it finds a matching key, and #N/A when there is no matching key. The non-matching keys (trades) are the ones you want to investigate

Good luck!

3 Likes

Thank you for the detailed reply, I shall do some more investigating.

I'm not sure why, but comparing the built in backtester to my low-level custom backtester, for the very first trade both produce the same trade with the same entry and exit dates with all other columns (including 'Price' and 'Ex. Price') identical except the 'Cumulative Profit, which for the built in backtester is reported as '104.16' whereas the custom backtester reports '172.76'.

Does anyone know what I'm missing?

For the first trade which closes, the Profit and Cumulative Profit should be the same. Is that the case in each trade list? If not, are you generating interest (either fixed or dynamic) on free cash in your backtester settings?

1 Like

Sorting by 'Ex. Date, Symbol' the first trades are identical except for the 'Cumulative Profit' , and yet I am not generating any kind of 'free cash' to my knowledge. That said I am processing the exit signals first so that I can establish how many new entries can be made on the same bar - would that play a factor even though this is the very first trade?

Is there only one trade that exits on the day of your "first" trade?

Yes - only one trade on the first day

And to return to my earlier question, on a single trade list, are the Profit and Cumulative Profit the same for that first trade?

Using the default backtester they are the same...

Standard%20Backtester

Whilst using the custom backtester they are different and swapped around with the second trade exiting on the same date...

Custom%20Backtester

I specifically asked you "Is there only one trade that exits on the day of your "first" trade?"

And you replied "Yes - only one trade on the first day"

Now you're showing me a trade list that shows TWO exits on the same day, and clearly the cumulative profit after both trades have exited is the same. So there is no problem.

1 Like

Sorry, I do appologise about that, when I read your question, for some reason I was looking at the entry dates!

Do you know why the exits are processed in a different order?

I don't, but why do you care? If everything is exiting at the Close (or perhaps the Open), then why does it matter?

Are you using a High, Mid, or Low level CBT? Are you modifying the signal objects in any way?

It only matters for comparison reasons so I can be more confident I haven't made any mistakes in my code.

I am using a low-level CBT in which the only signal property I modify is the 'Price' similar to the code I use with the standard backtest.

If you're comparing trades for accuracy, then looking at Cumulative Profit shouldn't be necessary. That metric is not specific to the trade, but rather to the portfolio.

I guess in the future I'll just have to ignore it, although it does make comparing the two trade lists in the comparison tool I like to use more difficult.

Thank you for all of your help and patience!