Nested loops optimization

Hello everyone,

I have been coding algos in ACSIL for the last 3 years and I am getting used to AFL programming for the last 2 months. I am quite impressed with the execution speed!

I am writing my first message today because I can't find a solution to my problem after multiple doc reading and forum digging. Here is the goal I want to achieve : I want to be able to detect when a previous swing is touched by a bar to open a trade or to scale out of a trade. For exemple : Buy when a swing low is touched and Sell when a swing high is touched.

On ACSIL, I have a vector that stores untouched swings and each bar will look into that vector to find touched swings and remove them.
On AFL, I didn't find a way to store untouched swings so I am looking at future bars when a swing is detected to find the bar that will touch it.

The following code works but is very slow on intraday tickdata because of the nested loops. I am pretty sure their is a powerfull AFL alternative but I can't put my finger on it.

////// Swing detection //////

SwingSize = Param("Swing : Size (Bars/Ticks)", 3, 1, 1000, 1);
SwingWindow = 2 * SwingSize + 1;

SwingHighCandidate = Ref(H, -SwingSize);
SwingLowCandidate = Ref(L, -SwingSize);

SwingHighPrice = IIf(HHV(H, SwingWindow) == SwingHighCandidate, SwingHighCandidate, -1);
SwingLowPrice = IIf(LLV(L, SwingWindow) == SwingLowCandidate, SwingLowCandidate, -1);


////// Swing touched detection //////
// This is the slow part that I want to convert into AFL logic 

CleanSwingHighTouched = 0;
CleanSwingLowTouched = 0;
CleanSwingHighTouchedIndex = -1;
CleanSwingLowTouchedIndex = -1;

for (i = 0; i < BarCount; i++) {
	
	// Swing high
	if (SwingHighPrice[i] >= 0) {
		// Look forward for the first bar that touches this level
		for (j = i + 1; j < BarCount; ++j) {
			if (H[j] >= SwingHighPrice[i] AND L[j] <= SwingHighPrice[i]) {
				// Mark touch price on that bar
				CleanSwingHighTouched[j] = SwingHighPrice[i];
				// Store at swing bar the index of the bar that will touch it
				CleanSwingHighTouchedIndex[i] = j;
				break; // only first touch counts
			}
		}
	}
	// Same for swing low
	if (SwingLowPrice[i] >= 0) {
		for (j = i + 1; j < BarCount; ++j) {
			if (H[j] >= SwingLowPrice[i] AND L[j] <= SwingLowPrice[i]) {
				CleanSwingLowTouched[j] = SwingLowPrice[i];
				CleanSwingLowTouchedIndex[i] = j;
				break;
			}
		}
	}
}

Thanks for your help!

Hello pepfox,

did you ask KI ? I did not checked your code but this code is a possible result ...

////// Swing detection //////

SwingSize   = Param("Swing : Size (Bars/Ticks)", 3, 1, 1000, 1);
SwingWindow = 2 * SwingSize + 1;

SwingHighCandidate = Ref(H, -SwingSize);
SwingLowCandidate  = Ref(L, -SwingSize);

SwingHighPrice = IIf(HHV(H, SwingWindow) == SwingHighCandidate, SwingHighCandidate, -1);
SwingLowPrice  = IIf(LLV(L, SwingWindow) == SwingLowCandidate, SwingLowCandidate, -1);


////// Swing touched detection (VECTORIZED) //////

// 1) Broadcast swing levels forward in time
SwingHighLevel = ValueWhen(SwingHighPrice >= 0, SwingHighPrice, 1);
SwingLowLevel  = ValueWhen(SwingLowPrice  >= 0, SwingLowPrice, 1);

// 2) Detect touches
HighTouch = (H >= SwingHighLevel AND L <= SwingHighLevel);
LowTouch  = (H >= SwingLowLevel  AND L <= SwingLowLevel);

// 3) First touch after each swing
CleanSwingHighTouchedIndex = ValueWhen(HighTouch, BarIndex(), 1);
CleanSwingLowTouchedIndex  = ValueWhen(LowTouch,  BarIndex(), 1);

// 4) Touch price only on the touch bar
CleanSwingHighTouched = IIf(HighTouch, SwingHighLevel, 0);
CleanSwingLowTouched  = IIf(LowTouch,  SwingLowLevel, 0);

Best regards,
Peter

Hello Peter,

Thanks for your answer and for your code!

Results are quite different compared to mine. Here is an exploration where yours are prefixed with "AFL".

And what/who is KI?

Best regards,
Pepfox

Hello pepfox,

sorry "KI" or better AI (Copilot, Grok, Claude ...) is able to optimize/transform your AFL code. So i suggested to ask AI for your further/additional ideas, but does not mean you have not to check/research your results from it ;).

Tomasz had build some cool features into last version of AB.

https://forum.amibroker.com/t/ai-setup-instructions-preliminary/40099

Best regards,
Peter

Based on my read of your issue, I think you might want to be pointed in a different direction. If you still need a suggestion, please post. And, BTW, it would be helpful to indicate the bars/day, history length, and number of tickers to be managed.

Thank you for your answer! I would indeed appreciate new suggestions.
The backtest is on 1 ticker only (ES), 12 ticks range bars, over 3 years. That's about 360 000 bars.

You didn't describe the use-case speed requirements, but let's assume that you need speed as fast as possible. I've found over the years, that approx. 75% of speed issues can be solved with AFL vector logic, 20% needs a creative approach that is beyond AI, and the remaining 5% needs an even more creative approach or a DLL. Yours, unfortunately, is in the 5%.

So, at a high level there are 3 approaches to your problem -

  1. Replace the "j" inner loop with vector logic. THIS WILL NOT IMPROVE PERFORMANCE. The existing code is efficient by executing a "break" after the first touch. Vector logic would continue to "scan forward". Quick test code for the touch index showed that it added more time.
  2. Write a DLL for the nested loop. THIS WOULD BE PREFERRED. Quick tests showed about a 200x improvement on a 1 second, 350K bar data array SPY. Took 20 sec to 76 msec. It may sound a little daunting, but since I assume that coming from ACSIL you are familiar with VC++, it should not be bad. And, your code easily translates to a very simple DLL.
  3. Other approaches come to mind, but one is to save the resulting detection scan in a StaticVarSet, and even compress it via SparseCompress to limit the AFL scan loop time. This can ba reloaded on startup and new bars added. THIS WILL INVOLVE SOME AFL CODING THAT IS MORE COMPLICATED.

BOTTOM LINE - I'd stongly suggest #2. If you can go that route, I'll forego explaining #3. Please advise if #2 is OK, and I'll post suggestions about how to structure it.

1 Like

@pepfox, the biggest performance boost you’re likely to achieve is by moving your nested‑loop logic into a DLL using the ADK (AmiBroker Development Kit).

If you’re not familiar with C, you could try using AI tools to generate the code or hire someone to implement it for you.
Just make sure your current AFL logic is fully correct before outsourcing or rewriting it, otherwise you may end up wasting both time and money/tokens.

1 Like

Thank you for your answer.

I since tried approach #1 with multiple Reverse and BarsSinceCompare. It worked but it was indeed slower than my nested loops. Thanks to your answer I now understand why.

I am curious about approach #2. Thanks @beppe for also suggesting it.

OK - I'll try to help and jump start you as much as time permits.

I originally got interested in your post as a vector "challenge", but quickly realized that you were in the 5% that I referred to that needs a DLL to "just get it done".

I will post some AFL code below that slightly restructures your original code as a function to make the DLL easier to see. The DLL call is commented out so that it executes just the AFL function. You will want to add exploration output as you did before to test results.

You didn't indicate if you are familiar with VC++ or an alternative, but let's assume that and jump in. As an intraday user, this need will probably come up again. If you are not familiar, then take Beppe's suggestion, as VC++ is a PITA to learn.

Just get the Sample sub-dir from the Samples directory at the link that Beppe gave you. You will be replacing the Functions.cpp file.

But, first there are a couple of subleties to be aware of for the DLL to keep it as simple as possible -

  • As you can see from the SwingTouchDetect() call in the code, I've kept a strict order (arrays, then scalar) to keep it simple.
  • The 4 "output" vectors are passed as call by reference parameters (with "&") and are pre-allocated in the AFL. The DLL overwrites them. BTW this makes the DLL simpler since it doesnt' have to do any array allocations, set variables, etc.
  • The 4 output vectors must be initialized as arrays since auto-promotion from scalars is not done in the DLL. For an array of -1's this is done as "C - C - 1", for example.

In the DLL, which you should do, there are a couple of things to point out -

  • The code from the AFLSwingTouchDetect() function plugs in ALMOST AS IS with a minor change of the "AND" to "&&".
  • The function table at the end of Functions.cpp looks like this -
FunctionTag gFunctionTable[] = {
								"SwingTouchDetect",		{ VSwingTouchDetect, 8, 0, 1, 0, NULL }, 

  • Parameters are retrieved as in this example for the first few -
float* H = ArgsTable[0].array;
float* L = ArgsTable[1].array;
float* SwingHighPrice = ArgsTable[2].array;

There are many more details that could be covered, but this should get you going, and I'm at a time stop. Come back if you hit a road block.

================

Here's the AFL -

SwingSize = Param("Swing : Size (Bars/Ticks)", 3, 1, 1000, 1);
SwingWindow = 2 * SwingSize + 1;

SwingHighCandidate = Ref(H, -SwingSize);
SwingLowCandidate = Ref(L, -SwingSize);

SwingHighPrice = IIf(HHV(H, SwingWindow) == SwingHighCandidate, SwingHighCandidate, -1);
SwingLowPrice = IIf(LLV(L, SwingWindow) == SwingLowCandidate, SwingLowCandidate, -1);


////// Swing touched detection //////
// This is the slow part that I want to convert into AFL logic 

CleanSwingHighTouched = 0;
CleanSwingLowTouched = 0;
CleanSwingHighTouchedIndex = -1;
CleanSwingLowTouchedIndex = -1;

AFLDetect = True;
DLLDetect = True;

_TRACE( " " );

//--------------------------------------------------------------------------------------------------
//  Function to detect swing touches using AFL looping -
//  Will be SLOW - even with first touch break
//--------------------------------------------------------------------------------------------------

function AFLSwingTouchDetect( H, L, SwingHighPrice, SwingLowPrice, 
						CleanSwingHighTouched2, CleanSwingLowTouched2,
						CleanSwingHighTouchedIndex2, CleanSwingLowTouchedIndex2, 
						BarCount )
{
	for (i = 0; i < Barcount; i++) {	
		// Swing high
		if (SwingHighPrice[i] >= 0) {		
			// Look forward for the first bar that touches this level
			for (j = i + 1; j < BarCount; ++j) {

				if (H[j] >= SwingHighPrice[i] AND L[j] <= SwingHighPrice[i]) {
					// Mark touch price on that bar
					CleanSwingHighTouched[j] = SwingHighPrice[i];
					// Store at swing bar the index of the bar that will touch it
					CleanSwingHighTouchedIndex[i] = j;

					break; // only first touch counts
				}
			}
		}
		// Same for swing low
		if (SwingLowPrice[i] >= 0) {
			for (j = i + 1; j < BarCount; ++j) {
				if (H[j] >= SwingLowPrice[i] AND L[j] <= SwingLowPrice[i]) {
					CleanSwingLowTouched[j] = SwingLowPrice[i];
					CleanSwingLowTouchedIndex[i] = j;

					break;
				}
			}
		}
	}

	return;
}

//--------------------------------------------------------------------------------------------------

if ( AFLDetect ) {
	GetPerformanceCounter( True );

	//  Order of parms is done to match DLL implementation - see DLL call comments
	AFLSwingTouchDetect( H, L, SwingHighPrice, SwingLowPrice, 
							&CleanSwingHighTouched, &CleanSwingLowTouched,
							&CleanSwingHighTouchedIndex, &CleanSwingLowTouchedIndex, 
							BarCount );
							
	elapsed1 = GetPerformanceCounter();
	_TRACE( "elapsed1 = " + elapsed1 );
}

if ( DLLDetect ) {

	//************************************************
	//  Make sure they are arrays before passing by reference - 
	//  otherwise they won't be promoted if scalar
	//************************************************
	CleanSwingHighTouched2 = C - C + 0;
	CleanSwingLowTouched2 = C - C + 0;
	CleanSwingHighTouchedIndex2 = C - C - 1;
	CleanSwingLowTouchedIndex2 = C - C - 1;

	GetPerformanceCounter( True );

	//  Small hacks can be used to change order - but conform to strict DLL rules
	/*
	res = SwingTouchDetect( H, L, SwingHighPrice, SwingLowPrice, 				// Input arrays
							&CleanSwingHighTouched2, &CleanSwingLowTouched2,	// Output arrays
							&CleanSwingHighTouchedIndex2, &CleanSwingLowTouchedIndex2, 
							BarCount );											// Input scalar
	*/
						 
	elapsed2 = GetPerformanceCounter();
	_TRACE( "elapsed2 = " + elapsed2 );
}```
2 Likes