Different backtest modes

AmiBroker
1

Hi, how are you?

I am studying the Different backtest modes based on the following info:
https://www.amibroker.com/guide/h_portfolio.html
https://www.amibroker.com/guide/afl/setbacktestmode.html

And although it is very well explained, I do not quite understand the difference between "backtestRegular" and "backtestRegularRaw" if I am doing a Backtest on a single symbol.

Don't they do the same thing? That is, although "backtestRegularRaw" does not eliminate redundant entry signals, as only one entry is allowed per symbol (and I only use one symbol), in the end, they act the same, right?

"backtestRegularRawMulti" will act differently from the other two since it will allow me to make more than one purchase on my single symbol.

Thank you very much

2

They are not always the same, even for a single symbol. For example, if your system enters on a limit order, you might generate a Buy signal when all of your setup conditions are met, but then in the CBT you would not actually enter a trade if the limit price is not breached. In that case, you don't necessarily want to wait for an exit signal to occur before you consider new entry signals, so backtestRegularRaw will make sure you see all the Buy signals.

3 Likes
3

Thank you very much mradtke,

As I didn't quite understand your answer, I asked Perplexity adding the question I asked and your answer and asking him to explain it better and with numbers.

This is his answer and I think I understood it (thanks mainly to you who bothered to give me an answer and then to the AI ​​that expanded it):

"Certainly, I'll explain the difference between "backtestRegular" and "backtestRegularRaw" , with visual examples:

  1. backtestRegular:
    • Removes redundant entry signals.
    • Once a position is opened, it ignores new entry signals until there's an exit signal.
  2. backtestRegularRaw:
    • Does not remove redundant entry signals.
    • Considers all entry signals, even if a position is already open.
      Visual example:
      Let's imagine a price chart with the following signals:
Day 1: Buy signal (Price: $100)
Day 2: Buy signal (Price: $102)
Day 3: Buy signal (Price: $103)
Day 4: Sell signal (Price: $105)
Day 5: Buy signal (Price: $104)

With backtestRegular:

Day 1: Buy at $100
Day 2: Ignore (position already open)
Day 3: Ignore (position already open)
Day 4: Sell at $105
Day 5: Buy at $104

With backtestRegularRaw:

Day 1: Buy at $100
Day 2: Record signal (but don't buy more as position is open)
Day 3: Record signal (but don't buy more as position is open)
Day 4: Sell at $105
Day 5: Buy at $104

The key difference is that backtestRegularRaw records all signals, which can be useful for:

  1. Post-analysis of signal frequency.
  2. Systems using limit or conditional orders.
  3. Strategies that want to "remember" missed signals for future decisions.

Practical example with limit orders:

Suppose your system generates a buy signal, but you only want to buy if the price drops 2% from the signal:

With backtestRegular:

  • Day 1: Buy signal at $100, places limit order at $98
  • Days 2-5: Ignores new buy signals
  • Day 6: Price touches $98, buys

With backtestRegularRaw:

  • Day 1: Buy signal at $100, places limit order at $98
  • Day 2: New signal at $102, updates limit order to $99.96
  • Day 3: New signal at $103, updates limit order to $100.94
  • Day 4: Price touches $100.94, buys

In this case, backtestRegularRaw allows for more flexible market entry, adapting to new signals and market conditions.

This explanation should provide a clearer understanding of the differences between these backtest modes, even when dealing with a single symbol.

Citations:
[1] Portfolio-level back testing
[2] AFL Function Reference - SETBACKTESTMODE
[3] Different backtest modes

Is it correct?

4

No, the Perplexity answer is not entirely accurate. For example:

"Certainly, I'll explain the difference between "backtestRegular" and "backtestRegularRaw" , with visual examples:

  1. backtestRegular:
  • Removes redundant entry signals.
  • Once a position is opened, it ignores new entry signals until there's an exit signal.
  1. backtestRegularRaw:
  • Does not remove redundant entry signals.
  • Considers all entry signals, even if a position is already open.
    Visual example:
    Let's imagine a price chart with the following signals:

Twice in this description Perplexity refers to open positions. However, backtestRegular removes redundant entry signals whether a position is actually opened or not. In practical terms, backtestRegular does something very similar to this AFL snippet:

Buy = ExRem(Buy,Sell);
Sell = ExRem(Sell,Buy);

The limit order example is also misleading. If you generate a Buy signal on Day 1, either a position will be opened on Day 1 or that signal will be skipped and discarded unless you prolong the signal with a function like Hold(). Similarly, the BuyPrice will change every day unless you take steps to select a BuyPrice from a past day, which you would need to do if you wanted to enter on Day 6 with the limit price from Day 1.

If you're still having trouble understanding the difference between backtestRegular and backtestRegularRaw, I suggest you run some simple backtests against a watchlist of symbols. Run a test in each mode, find the differences in the trade list, and then make an effort to understand why the differences occurred. Using the Detailed Log report type might also help you with your understanding, as AmiBroker will show you all the entry and exit signals generated and which ones did and did not result in a trade being opened.

5 Likes
5

Thanks again mradtke,

It's clear that nothing can replace this forum and the help offered by those of you who are more knowledgeable about AFL.