CBT - mixing low level method with mid level level approach

Hi,

I have a simple test CBT below that is having a problem.

It executes buys on even days and sells on odd days. It buys three stocks (AAPL, INTC, MSFT) using the Data database that comes prepackaged with AB. Since MaxOpenPositions is set to 2 and PositionScore for AAPL is lower than that of INTC and AAPL, only INTC and MSFT are bought using ProcessTradeSignals(). But since there is spare cash in the portfolio, we then try to use EnterTrade() to buy some shares of AAPL. When we try to do that, AB crashes.

SimpleCBT.afl code:

SetTradeDelays(0, 0, 0, 0);
Exclude = !(Name() == "INTC" || Name() == "MSFT" || Name() == "AAPL");
SetOption("MaxOpenPositions", 2);
SetPositionSize(40, spsPercentOfEquity);
Buy = DateNum() % 2 == 0;
Sell = DateNum() % 2 == 1;
BuyPrice = Close;
SellPrice = Close;
PositionScore = 100;
if (Name() == "AAPL") PositionScore = 0;

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio) {
    bo = GetBacktesterObject();
    bo.PreProcess();
    dt = DateTime();
    dn = DateNum();
    EnterAAPL = True;
    for (bar = 0; bar < BarCount; bar++) {
        _TRACE("AT: " +
                NumToStr(dt[bar], formatDateTimeISO) + ", " +
                "Cash = " + NumToStr(bo.Cash, 1.0)); 
        for (sig = bo.GetFirstSignal(bar); sig; sig = bo.GetNextSignal(bar)) {
            _TRACE("AT: Signal: " + sig.Symbol + ", " + 
                    "price = " + NumToStr(sig.Price, 1.2) + ", " +
                    "size = " + NumToStr(sig.PosSize, 1.0) + ", " +
                    "type = " + NumToStr(sig.Type, 1.0));
        }
        _TRACE("AT: Start ProcessTradeSignals()");
        bo.ProcessTradeSignals(bar);
        _TRACE("AT: Done ProcessTradeSignals()");
        if (dn[bar] % 2 == 0 && EnterAAPL) {
			bo.EnterTrade(bar, "AAPL", True, 100, 1000);
			bo.UpdateStats(bar, 2);
			_TRACE("AT: Done EnterTrade()");
		}
        for (pos = bo.GetFirstOpenPos(); pos; pos = bo.GetNextOpenPos()) {
			_TRACE("AT: Position: " + pos.Symbol + ", " +
					"entryprice = " + NumToStr(pos.EntryPrice, 1.2) + ", " +
					"entryDate = " + NumToStr(pos.EntryDateTime, formatDateTimeISO));
        }
        _TRACE("AT: ");
    }
    bo.PostProcess();
}

Debug output when EnterAAPL = False (EnterTrade will not be executed with this setting). Everything works fine. The Backtest here is with Apply To = "All Quotes" and Range = "3 recent bars"

Debug-1

Debug output when EnterAAPL = True (this causes EnterTrade to execute and causes AB to crash). AA settings are same as above. From the debug output, the crash happens in ProcessTradeSignals().

Debug-2

Screenshot of crash message:
Recovery

Steps to reproduce the problem:

  1. Use Data database that comes prepackaged with AB.
  2. Use SimpleCBT.afl code above.
  3. Use AA settings: Apply to = "All quotes", Range = "3 recent bars"
  4. Run first with EnterAAPL = False (this runs fine)
  5. Run next with EnterAAPL = True (this crashes)

Please advise on what I am doing wrong. Thank you.

I tried your code on my end with current version 6.35.1 and it did not crash. However, one thing I noticed, that is incorrect: if you are using ProcessTradeSignals() (midlevel), you should not be calling UpdateStats() (lowlevel).
The docs at https://www.amibroker.com/guide/a_custombacktest.html say:

UpdateStats() [....] You must NOT use this function in high-level and mid-level approaches.

Tomasz: thank you looking into it.

Yes, I had read that part about UpdateStats() to not be used in mid level and that is how the code originally was and it was still crashing. I put that in to see if it would prevent the problem. It crashes with or without UpdateStats() and the problem is completely repeatable at my end. Using EnterAAPL = False makes it work fine.

FWIW, I have tried some other CBTs from the KB that mix mid level and low level methods and they have worked fine in my setup.

As I wrote, I tested with current version 6.35.1 and it works. Try that one.
Also change your code so it does not set position score to 0.

if (Name() == "AAPL") PositionScore = 0; // change that to 1 for example

Hi Tomasz:

I changed AAPL PositionScore to 1 per your suggestion and it still crashes. However, I did made some further progress on debugging the issue. The code works fine if EnterTrade occurs before ProcessTradeSignals but crashes if EnterTrade occurs after ProcessTradeSignals.

Here is new version of program with switch enterBefore in the first line of the code. When the switch is True, the program works fine. When the switch is False, AB crashes. I have removed all the _TRACE statements in the code to remove the noise.

enterBefore = True;
SetTradeDelays(0, 0, 0, 0);
Exclude = !(Name() == "INTC" || Name() == "MSFT" || Name() == "AAPL");
SetOption("MaxOpenPositions", 3);
SetPositionSize(40, spsPercentOfEquity);
dn = DateNum();
Buy = dn % 2 == 0;
Sell = dn % 2 == 1;
BuyPrice = Close;
SellPrice = Close;
PositionScore = 100;
if (Name() == "AAPL") PositionScore = 0;

SetCustomBacktestProc("");
if (Status("action") == actionPortfolio) {
    bo = GetBacktesterObject();
    bo.PreProcess();
    dt = DateTime();
    dn = DateNum();
    for (bar = 0; bar < BarCount; bar++) {
        if (enterBefore && dn[bar] % 2 == 0) {
			bo.EnterTrade(bar, "AAPL", True, 100, 1000);
		}
        bo.ProcessTradeSignals(bar);
        if (!enterBefore && dn[bar] % 2 == 0) {
			bo.EnterTrade(bar, "AAPL", True, 100, 1000);
		}
    }
    bo.PostProcess();
}

Screenshot of backtest results with enterBefore = True.
Before

One other thing (don't know if this is material): you mention you tested with 6.35.1. I am running 6.30.5. I installed 6.30.5 with the understanding that is the latest release build since 6.35.1 is labeled as BETA on the website.

Thank you.

I also upgraded to 6.35.1 but I get the same results. Thanks.

Hi,

I have done some more experiments with 6.35.1 and arrived at the following results. Summary of key results:

  1. AB crashes if EnterTrade is invoked after ProcessTradeSignals and signal list has Buy signals on the same bar.

  2. There is no problem with the invocation of EnterTrade if the signal list has only Sell signals on the bar.

  3. There is no problem if EnterTrade is invoked before ProcessTradeSignals; in this case, EnterTrade can be invoked when signal list has Buy signals and everything works fine.

These results are based on the following formula. How to repeat:

  1. Run following formula with Data database that came prepackaged with AB
  2. Run Backtest with Apply to = "All", Range = "10 recent bars".

Code commentary: this formula has a mod-4 cycle. Buy and Sell signals occur for INTC and MSFT on day 0 and 1 (respectively) of the mod-4 cycle and are handled by ProcessTradeSignals. Entries and exits for AAPL are made using EnterTrade and ExitTrade on aaplEntryDay and aaplExitDay respectively. All positions, initiated by ProcessTradeSignals or EnterTrade, are exited the next day. If we set aaplEntryDay to 0, CBT crashes since AAPL EnterTrade is made on same day as the signal list having Buy signals. If aaplEntryDay is set to any other value such as 1, 2, or 3, the CBT works fine. If we change the order such that ProcessTradeSignals is invoked after EnterTrade, then there is no problem invoking EnterTrade on any day, including days with Buy signals on the signal list. _TRACE statements have been stripped out from the code to remove the visual noise bu they may be inserted for purposes of instrumentation.

This is posted here in the hope that it is useful to the community at large and with the hope that this issue may be triangulated in case someone else runs experiments confirming or contradicting these results. I have been able to repeat them using two different machines at my end.

My thanks to Tomasz for his earlier responses on the issue. Best wishes.

aaplEntryDay = 0;
aaplExitDay = aaplEntryDay + 1;
SetTradeDelays(0, 0, 0, 0);
SetOption("MaxOpenPositions", 3);
SetPositionSize(40, spsPercentOfEquity);
dn = DateNum();
Buy = dn % 4 == 0;
Sell = dn % 4 == 1;
BuyPrice = SellPrice = Close;
if (!((Name() == "INTC") || (Name() == "MSFT"))) Buy = Sell = 0;
SetCustomBacktestProc("");
if (Status("action") == actionPortfolio) {
    bo = GetBacktesterObject();
    bo.PreProcess();
    dn = DateNum();
    for (bar = 0; bar < BarCount; bar++) {
        bo.ProcessTradeSignals(bar);
        if (dn[bar] % 4 == aaplExitDay % 4) bo.ExitTrade(bar, "AAPL", 200, 1);
        if (dn[bar] % 4 == aaplEntryDay % 4) bo.EnterTrade(bar, "AAPL", True, 100, 1000);
    }
    bo.PostProcess();
}

@alpha1,
Try this

SetOption( "maxopenpositions", 4 ) ;
last = BarCount - 1;
Buy = Sell = Short = Cover = 0 ;
Buy [last] = True ;


symbol = Name() ;
switch( symbol )
{
case "MSFT":
    PositionScore = 3;
    break ;

case "INTC" :
    PositionScore = 2;
    break ;

case "AAPL":
    //Buy = 0 ;
    PositionScore = 1;
    break;

default :
    Exclude = True ;
    
}
PositionSize = 10000;

SetCustomBacktestProc( "" );

if( Status( "action" ) == actionPortfolio )
{
    bo = GetBacktesterObject();	//  Get backtester object
    bo.PreProcess();	//  Do pre-processing (always required)

    i = BarCount - 1;

    

    for( sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal( i ) )
    {
        _TRACE( sig.Symbol );
    }

    bo.ProcessTradeSignals( i );	//  Process trades at bar (always required)

    bo.EnterTrade( i , "AAPL", True , 100, 10000 ) ;


    bo.PostProcess();	//  Do post-processing (always required)
}

1 Like

Hi, what is up with the above post? It shows "awaiting approval" for past 5 days.

Hi @aron,

Thank you for the reply. I replied to your post immediately but my reply never showed up. So I am replying again.

You code runs for just one bar. You should see the issue if you run AB for two bars.

The following is essentially your code with one small change. It calls ProcessTradeSignals and EnterTrade on bar T and then tries to call ProcessTradeSignals again on bar T+1. The problem occurs on bar T+1.

Please run it on your machine and let me know if it works for you. Thanks.

SetOption( "maxopenpositions", 4 ) ;
last = BarCount - 2;
Buy = Sell = Short = Cover = 0 ;
Buy [last] = True ;

symbol = Name() ;
switch( symbol )
{
case "MSFT":
    PositionScore = 3;
    break ;
    
case "INTC" :
    PositionScore = 2;
    break ;

case "AAPL":
    PositionScore = 1;
    break;

default :
    Exclude = True ;
}
PositionSize = 10000;

SetCustomBacktestProc( "" );
if( Status( "action" ) == actionPortfolio ) {
    bo = GetBacktesterObject();	
    bo.PreProcess();	

    // this is bar T 
    i = BarCount - 2;
    for( sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal( i ) )
        _TRACE( "AT: " + sig.Symbol );
    bo.ProcessTradeSignals( i );	
    bo.EnterTrade( i , "AAPL", True , 100, 10000 ) ;
    
    // this is bar T+1
    i = BarCount - 1;
    for( sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal( i ) )
        _TRACE( "AT: " + sig.Symbol );
    bo.ProcessTradeSignals( i );	

    bo.PostProcess();	
}

A good rule of thumb is to avoid mixing commands from the different CBT levels. There are some cases where you can use both mid and low level CBT methods together (as @Tomasz has stated previously) but there are probably not many scenarios where you need to do this. If your strategy requires low-level CBT functionality then just do everything with the low-level methods.

@alpha1
You are calling ProcessTradeSignals() twice.
However, as @mradtke said if your strategy requires so, you can do everything in low-level, no need to mix methods.

Hi @mradtke: thank you for the reply. Yes, I understand it is best to not mix low level and mid level methods. However, the KB has examples of such mixing and @Tomasz said in his initial response above that on his machine ProcessTradeSignals will work alongside EnterTrade.

I have a workaround for above case. However, I would appreciate any other user running that tweak of @aron code that I posted and letting me know if they were able to run it flawlessly. If so, it would point to some configuration or DLL issues with my AB setup. Best wishes.

For the record, I did not tell that you can not use EnterTrade. What I told you was:

UpdateStats () [....] You must NOT use this function in high-level and mid-level approaches.

And that is quote from the manual.

I never told that you can not call EnterTrade. In fact KB article is doing ScaleTrade:
http://www.amibroker.com/kb/2006/03/06/re-balancing-open-positions/

And I am doing so because IT WORKS. And as I wrote - your first code posted in this thread DOES NOT CRASH on my end. It works perfectly, I tried it dozens of time and it never crashed. Maybe you have unstable computer, power supply issues, RAM trouble or have some 3rd party software that causes problems.

But your latest code is obviously incorrect. It is incorrect to call ProcessTradeSignal more than once or less than once per bar. It is also incorrect to call ProcessTradeSignals in any different order than sequentially from bar 0 to BarCount -1. You can not freely skip bars, or start from the one before last as you do. That is all incorrect. The signal processing loop must be written in one and only correct way:

for( i = 0; i < BarCount; i++ ) bo.ProcessTradeSignals( i, ... );

Sequentially from FIRST to LAST BAR and NOT any other way.

ProcessTradeSignal must ONLY BE CALLED ONCE PER BAR.

And again it is DESCRIBED in manual https://www.amibroker.com/guide/a_custombacktest.html:

bool ProcessTradeSignals ( long Bar )

This mid-level method processes all trading signals for given bar. It should be called once per every bar in your custom backtesting loop.

ONCE PER EVERY BAR

Every word in my manual counts.

Really I beg you READ THE MANUAL CAREFULLY. Every sentence counts. I don't have unnecessary words in manual. Don't skip words or sentences. Every word matters. This is not your literature lesson where you can skip paragraphs. This is highly technical document where every letter is important.

Generally CBT is NOT for beginners. If you are beginner and don't have programming background stay away from CBT.

Second thing is that sending screenshots from crash recovery window is totally useless, because screenshot is truncated (it shows just 10% of the report, without crucial information, like call stack). The only meaningful thing is using "Copy to clip" button and sending TEXT, not picture. Thirdly it is useless to post it here, because SUPPORT email is the place to get support from me, not here, because NO ONE is able to do anything with your pictures.

BETTER YET, press the "Send report" button instead instead of all of this. We have AUTOMATED system to send and receive bug reports. I have programmed this for reason. It does everything needed in ONE click, and allows reports to be programmatically and statistically analyzed. Screenshots can't. Yet people just seem not to see this "Send report" button. Just hit that button, it is 100x more useful than anything else.

I am sorry, but my time is really limited and it frustrates me to explain this over and over and over and over again, I really would like to spend my time in more creative ways.

I guess I need to totally change the design of this window and write in BIG RED LETTERS: "DO NOT SEND SCREENSHOTS OF THIS DIALOG!"

Or I will do the "Microsoft way" and assume users don't read what is written on screen and would only give two buttons, do you want to send report? "YES" and "NO" buttons and nothing else.

1 Like

@aron:
I am not calling ProcessTradeSignals twice. It is being called exactly once per bar. Please read the code carefully. It is a small tweak of your code. Also, to @Tomasz point about ProcessTradeSignals needing to be called exactly once on each bar - that is also the case. It needs to be run in AA window with Range = "2 recent bars". Since there are only two bars, the first call of ProcessTradeSignals is on the penultimate bar (bar T) and the second call is on the last bar (bar T+1).

The above code crashes on my machine. Let me emphasize that I do not have a show stopper issue here. I do not need a solution. I know how to sidestep the crash problem with a workaround.

Instead of seeking a solution, I posted above code to highlight a potential issue. Since the code crashes on my machine but not on Tomasz machine, it would be good for @aron or any other user who cares about AB to try it out and see. Perhaps there is a DLL issue on my machine. Perhaps not.

Finally, a few other points: (a) my machine is highly reliable; trading software and other software run on it 24x7 for weeks without booting, (b) having built products that work 24x7 that everyone on this forum uses, I may be new to AB but I am no neophyte in computer science, (c) if there is an issue here with AB crashing if EnterTrade is called after ProcessTradeSignals, that still does not take away from the fact that AB is marvelous product given it is the creation of one individual. I thank @Tomasz for having created such a fantastic platform for us to use.

Thank you and best wishes.

1 Like

Ok, thanks, still please use "Send report" button.