Grouping Symbols to avoid correlation

AFL Programming
#1

I have a strategy that is trading a universe of futures symbols. I am trying to stop positions of highly correlating symbols from being traded on the same day.

For example;

If there is an entry bar for ES and YM or NQ. I would only want to take one of them. Entries on subsequent bars are fine.

I looked into the custom backtester, however, from my understanding that works as a second phase of the backtest, meaning that the trades would have already been placed.

Is there anyway to do this? Any help would be greatly appreciated.

Thanks!!

1 Like
#2

Hi Glenn,

I think you surely have gone through How to display correlation between symbols.

Tweaking the concept discussed in the article and molding it in accordance to your backtest needs would presumably make it work.

1 Like
#3

It can be done using CBT.

While your problem statement is a little too vague to be put into a precise code, this concept will work:

  1. Define your correlation, cor=(your code)
    I don't know what you are trying to achieve here exactly, what you need to figure out is how you choose the symbol you want to trade. Example: ES,YM,NQ all signals on the same day. Which one to buy if your "cor" is too high.
  2. One way to address your problem could be to set create a variable, based on your condition, call it
    cor_too_high = True/False
  3. create a static variable that can be read in CBT. staticvarset(Name()+"cor_too_high",cor_too_high);
  4. CBT procedure: you will now manipulate the signal object to reject trades based on your static variable
if (Status("action") == actionPortfolio) 
{

    bo = GetBacktesterObject();	//  Get backtester object
    
    
    bo.PreProcess();
    	//  Do pre-processing (always required)

    for (i = 0; i < BarCount; i++)	//  Loop through all bars
    {
		

		rqmargin = 0; // initialize required margin as "0"

        for (sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal( i ) ) //loop through all signals
        {	
        
			reject=StaticVarGet(sig.Symbol + "cor_too_high");
			
			if( sig.IsEntry() AND reject[i]==True)								// If entry signal
            {
				sig.Price=-1;	//to skip entry, assign -1 to sig.Price
				sig.PosScore=0; //this is NOT required, however another way to address the problem is to assign a specific PositionSCore if correlation is too high
			}//entry
		}//for sig
		bo.ProcessTradeSignals( i );	//  Process trades at bar (always required)
	}//for barcount
	bo.PostProcess(); // Do post-processing (always required)
}//if backtest
  1. As mentioned in the code, another way to address your problem is assigning a specific PostionScore to prefer certain signals. It will depend on what you want to do.
1 Like
#4

A couple of things:

@gwil, your Phase 1 AFL defines the signal arrays (Buy, Sell, Short, Cover), prices (BuyPrice, SellPrice, etc), PositionScore, and other variables. Phase 1 is executed once for each symbol in your watch list. However, no trades are "placed" during Phase 1... you're just generating signals to be processed in the second phase.

The second phase is the actual back test in which trades are "placed". You can use the default AmiBroker back tester, or you can write your own Custom Back Test (CBT).

@Dionysos, while your basic approach is OK, I don't think you can have a single "cor_too_high" variable, because you have to consider all the pair-wise correlations between symbols. Therefore, for a watchlist of N symbols you will have N * (N-1) / 2 correlation pairs and therefore an equivalent number of static variables. That will be OK as long as the watch list is not too large, but could obviously get out of hand.

Before opening a new position, you will need to consider all other open positions, or perhaps all other symbols that have an entry signal on the current bar. If the watch list is to big to calculate correlations in advance for all possible pairs, then you would need to calculate correlation on the fly, which will likely make the CBT very slow. Another option would be to run a separate AFL to calculate correlations and store them in the database as artificial tickers.

1 Like
#5

Maybe, maybe no. The point here is to avoid taking new trades by sig.Price = -1. There is no say what correlation he is talking about. correlation to an index? correlation to other symbols?
EOD Trading? Intraday? What correlation? Pearson, Kendall, Spearman?

Howcome? He just said he wants to delay entry by 1 day.

#6

Hi Guys,

Thank you for the help, it was much appreciated.

I just hardcode the correlation as I know which symbols I want to avoid double entries with. The working code is below. I'm sure there is a more efficient way to write this but it does the trick.

SetOption("UseCustomBacktestProc", True );

signals = "";

if( Status("action") == actionPortfolio )
	{
	bo = GetBacktesterObject();
	bo.PreProcess();
	for( bar = 0; bar < BarCount; bar++ )
		{
		signals = "";
		index_count = 0;
		currency_count = 0;
		grain_count = 0;
		bond_count = 0;
		metal_count = 0;
		for( sig = bo.GetFirstSignal( bar ); sig; sig = bo.GetNextSignal( bar ) )
			{
			
			if(sig.IsEntry())
				signals += sig.Symbol + "," ;

			// separate the string into pieces
			x = "none";
			Count = 0;
			while(x !="")
				{
				// get the string
				x = StrExtract(signals ,Count);

				if(x!="")
					{

					//check currency
					if ( (StrFind(", &6A, &6E, &6J, &6M, &6S, &6C", x)) > 0)
						{
						currency_count ++;
						if( currency_count > 1)
							{
							printf("dont take currency trade" + x + "  ");
							if(sig.IsEntry())sig.Price = -1;
							}
						}

					// check index
					if ( (StrFind(", &ES, &YM, &NQ", x)) > 0)
						{
						index_count ++;
						if( index_count > 1)
							{
							printf("dont take index trade" + x + "  ");
							if(sig.IsEntry())sig.Price = -1;
							}
						}
						
					// check bonds
					if ( (StrFind(", &ZN, &ZT, &ZB, &ZF", x)) > 0)
						{
						bond_count ++;
						if( bond_count > 1)
							{
							printf("dont take Bond trade" + x + "  ");
							if(sig.IsEntry())sig.Price = -1;
							}
						}
					
					//Check Grains
					if ( (StrFind(", &ZC, &ZL, &ZM, &ZS", x)) > 0)
						{
						grain_count ++;
						if( grain_count > 1)
							{
							printf("dont take grain trade" + x + "  ");
							if(sig.IsEntry())sig.Price = -1;
							}
						}
					
					//Check Metals
					if ( (StrFind(", &GC, &SI, &PL, &PA", x)) > 0)
						{
						metal_count ++;
						if( metal_count > 1)
							{
							printf("dont take metal trade" + x + "  ");
							if(sig.IsEntry())sig.Price = -1;
							}
						}
					
					}
				Count++;
				}
				currency_count = 0;
				index_count = 0;
				grain_count = 0;
				bond_count = 0;
				metal_count = 0;
}

bo.ProcessTradeSignals( bar );
bo.AddCustomMetric("signals", signals);
}
bo.PostProcess();
}
2 Likes
#7

looks good. Does it actually help?
Correlation stuff was always a let down to me. It never worked when I needed it the most.

There can always be improvements to codes, if it gets the job done, I usually don't bother, that's me trading, instead of "coding". I believe in getting reliable results quickly for evaluation, then revisiting the code.

Signal object can also check the symbol: sig.Symbol
Below you could iterate through a list of strings separated by comma, that might be useful to you. You could make (if necessary) your code more flexible by putting all your high correlation symbols into a watchlist or sectorlist. Say Grains: wlnum=0, rates: wlnum:1 etc.

function CategoryCountSymbols( category, number )
{
   count = StrCount( list = CategoryGetSymbols( category, number ), ",");  
   return IIf( list == "", 0, count + 1 );
}    

Title = "Symbols in watchlist 0: " + CategoryCountSymbols( categoryWatchlist, 0 );

from here: count symbols

Then you could use

CategoryGetSymbols( category, index, mode = 0 )

category online examples

Basically what it "automates" is you can now iterate through all symbols in your watchlist and you don't need to type the symbol name out in CBT.

this one will iterate through your list of symbols:

// retrive comma-separated list of symbols in watch list 
   list = CategoryGetSymbols( categoryWatchlist, listnum ); 

   for( i = 0; ( sym = StrExtract( list, i ) ) != ""; i++ ) 
   { 
     
     // do something here;
      
   }

What might be even more interesting is to take only the signal from, i.e. the "grains" watchlist with the highest PositionScore (depends on what you exactly do).

1 Like
#8

Hi Dionysos,

That code looks much cleaner, thanks!! I'll give it a go.

Yeah, it works from a point of reducing risk. It's a pattern based strategy that risks 1% of the account per trade, with a maximum of 5 open positions.

Without this code stopping multiple trades of the same category I was often getting the same signal on the same day for ES, YM and NQ for example. This is basically the same trade, which ups my risk to 3% for the trade and fills 3 of the possible 5 slots for the same thing. It has helped to reduce the drawdowns significantly.

Adding a PositionScore is a great idea, i will have a go at that also.

Thanks again!!

1 Like