Least Squares Fit (FIR Function / FIR_AFL function / ALMA Function)

AFL Programming
#1

Math question regarding the FIR function - “It is functional (but 2+ times faster) equivalent of following AFL”: - from the function definition.

function FIR_AFL( input, coeff, size ) 
{ 
 result = 0; 
 sumcoeff = 0; 
 for( i = 0; i < Min( size, BarCount ); i++ ) 
 { 
   sumcoeff += coeff[ i ]; 
   result += coeff[ i ] * Ref( input, - size + i + 1 ); 
 } 

 return result/sumcoeff; 
}
function ALMA_AFL( input, range, Offset, sigma ) 
{ 
 local m, im, s, Coeff; 

 m = floor( Offset * (range - 1) ); 
 s = range / sigma; 

 for( i = 0; i < Min( range, BarCount ); i++ ) 
 { 
   im = i - m; 
   Coeff[ i ] = exp( - ( im * im )/ ( 2 * s * s ) ); 
 } 

 return FIR( input, Coeff, range ); 
}

If I replace the 2 times faster FIR with FIR_AFL in the ALMA function (it produces a very smooth line but looks into the future) - Question is why does replacing FIR with FIR_AFL cross the threshold of looking into the future?

Thanks kindly

#2

hi Greg,

you refer to my last post here: Fast fourier transform FFT

I use the functions from the manual but tweaked them a little in such a way the Gauss function is centered around the point for which I calculate the smooth. Because I use it to filter (or detrend) the data I am not worried about looking into the future. This code was intended just to filter the data in the time domain before calculating the frequency spectrum.

that FFT code also can show the Gauss curve when you show the "Time Domain", "Filtered data" and scroll all the way to the first bar. Will look like this:

gs

#3

Excellent - Thank you Ed

#4

Why do you think it looks into the future??? FIR is definitely causal function that does NOT look into the future.

#5

no I tweaked the function a bit for smoothing. Best to illustrate when I replace the Bell function with a Boxcar. The smooth at index i is calculated using (C[i-1]+C[i]+C[i+1])/3 (when the filter width is set to 3). I did this on purpose for detrending purposes.

fw = Param( "Filter Width (Bars)", 3, 1, 300, 2 );
sigma1 = Param( "Sigma", 5, 0.1, 50, 0.1 );

GaussBell = Null; // store the Gauss Bell for display purposes

function FIR_AFL( input, coeff, size )
{
    result = 0;
    sumcoeff = 0;

    for( i = 0; i < Min( size, BarCount ); i++ )
    {
        sumcoeff += coeff[ i ];
        result += coeff[ i ] * Ref( input, - int( size / 2 ) + i );
    }

    return result / sumcoeff;
}

function ALMA_AFL( input, range, Offset, sigma )
{
    local m, im, s, Coeff;

    //m = floor( Offset * ( range - 1 ) );
    //s = range / sigma;

    for( i = 0; i < Min( range, BarCount ); i++ )
    {
        //im = i - m;
        //GaussBell[i] = Coeff[ i ] = exp( - ( im * im ) / ( 2 * s * s ) );
        Coeff[ i ] = 1; // boxcar function
    }

    return FIR_AFL( input, Coeff, range );
}


data = C;//( H + L + C ) / 3;

// calculate convolution filter
conv = ALMA_AFL( data, fw, 0.5, sigma1 );

SetChartBkColor( colorBlack );
SetChartOptions( 1, chartShowDates, chartGridMiddle, 0, 0, 0 );
Plot( Close, "Price", colorDefault, styleCandle, Null, Null, 0, 0, 1 );
Plot( data, "Raw Data", ColorRGB( 0, 0, 255 ), styleLine, Null, Null, 0, 0, 2 );
Plot( conv, "Filter", ColorRGB( 250, 250, 255 ), styleLine, Null, Null, 0, 0, 2 );
#6

Were you referring to this tweak that you made in FIR_AFL() function, Ed?

Although very potent, but it makes the values future dependent when compared to inbuilt FIR(). It is understandable that the inbuilt one uses Ref( input, -size + i + 1 ) as multiplier for the coeff.

However, is there a way to mould the for loop in such a way that we could still refer to size / 2 bars back, i.e. retain the same smoothing and yet the result won't depend on values of future bars?

Being working on this for several hours now, got no clue yet!

Thank you

#7

hi,

yes it had to do with size/2. I did this not for using it in trading but to detrend the data before doing a Fourier transform. So I only did this to see if I could find certain periodicities when performing the Fourier transform.

I would just ignore it or forget about that, it can't be used in trading. It was just a tool to use for doing the detrend. A moving average is calculated using only past data. A moving average is also what you could call a convolution of the data with a box function. For detrending I basically moved this box function (by half the number of points of the box function) so the center is below the data point at which I calculate the smoothed function.

sort of like this I think:

Convolution_of_box_signal_with_itself2

from: https://en.wikipedia.org/wiki/Convolution

1 Like
#8

Sorry for my previous vague question, Ed!

Never thought like that... :slight_smile:

Thank you very much for explaining... Hence, the other half of the data set look into the future. Still the concepts are very elusive for me, might take a week or two to fully understand!

Anyways, would like to confess that for good 2-3 hours I was deluded and the reptile in me was very happy, as I simply used your tweaked FIR() from the FFT thread into my model and back-tested which showed stellar results, something I had never seen before. Ha ha ha!

But then skepticism and rationality took over and found this thread. Effect of dopamine, gone! :smile:

#9

yeah I maybe shouldnt have posted it, but I posted it together with the FFT stuff. Back then there was also another user who took that piece of code and thought he had this amazing system. But I clearly said I tweaked it a bit.

It is basically very simple. A simple moving average like for instance MA( C, 3 ) calculates the average of the close of the current bar, the bar before that and the bar before that and the result is saved at the current bar.

In simple terms (I actually used a Guass type function) what I did is calculate the average of the current close, the previous close and the next close. So in real trading this "next close" will not have arrived yet, but for my purposes it gives a better average calculation at the current bar and therefor the detrend of the raw data would be better which would result in a better Fourier transform and my hopes were that I might more easily see if there would be any periodicities in the data I could use for trading.

I was just playing around, I would not put your time in it, just use the function Amibroker has provided.

1 Like
#10

But all your work inspires... Thank you...

Wish there was a formula for "The Arts" too!

#11

i dont know what "The Arts" means

there are some Cougar coins out there: https://www.monex.com/prods/gold_cougar.html

or a monsterbox: https://goldsilver.be/nl/monster-box-500-coins/152-monster-box-500-x-1-oz-silver-cougar-2016.html

haven got that one yet. It is more expensive. And silver is silver.

1 Like
#12

I meant by "The Art" of Trading - intuitive trading styles, on contrary to the systematic approach - ability to anticipate market moves, rather than predicting using indicators built on past data.

Trading based on Price action, Channels, Trend-lines, Elliot, Fibonacci, Pivots, etc. are subjective in nature, and there exists no comprehensible structure, so, back-testing such systems are very difficult - nearly impossible for me.

On the other hand, we have mathematical models based on Stochastics, Moving Averages (God knows how many of them are out there), Oscillators, Volatility, etc. I think for every System following trader, there exists an unique indicator (or a set of indicators). These systems can be easily back-tested and yes, no matter how hard I try, give my best efforts, I lose money or hardly breakeven on all of them. :smile:

So we have, subjectivity on one side which is obviously very esoteric and hard to follow, on the other we have objectivity which does not yield me any good. Science is not working in my case, so, my ignorant mind says, may be "The Art" will! Which I don't know either? :slight_smile:

#13

channels and trendlines can also be backtested. With respect to the FFT (fast fourier transform) that can be used to look for periodicities in the data. A while ago I found that an Amibroker user seems to use it, see: https://www.prescientrading.com/prescientsignals/#video

I can't tell you if it any good though.

2 Likes
#14

Good to see you back Ed.

#15

Hi ed,

I dont really undestand the math behind the FFT, is it posible to make it not look in the future/repaint/future leaks ?

Thank you

#16

hi,

the FFT does not look into the future I just "detrended" the data just to take out the very low frequencies which have the most amount of power and could mask posible higher frequency periodicities. This detrend function I used however looks into the future. You could ignore the "detrend" step I used. My aim was to show whether some periodicity exists (and did not succeed).

I did not do further work on this FFT myself. Many years ago I worked a bit with it and then used a Power Spectrum and wanted to make a similar Power Spectrum using financial data. But I am not sure how others use the FFT with financial data. You coud maybe ask the guy from the link i posted higher up.

2 Likes
#17

thanks ed,

You could ignore the "detrend" step I used

which part is that ed ?

fw     = Optimize( "Filter Width (Bars)", 150, 1, 150, 2 );//*********
sigma1 = Optimize( "Sigma", 29.6, 0.1, 40, 0.1 );//**************

function FIR_AFL( input, coeff, size )
{
  result = 0;
  sumcoeff = 0;

  for( i = 0; i < Min( size, BarCount ); i++ )
  {
      sumcoeff += coeff[ i ];
      result += coeff[ i ] * Ref( input, - int( size / 2 ) + i );
  }

  return result / sumcoeff;
}

function ALMA_AFL( input, range, Offset, sigma )
{
  local m, im, s, Coeff;

  m = floor( Offset * ( range - 1 ) );
  s = range / sigma;

  for( i = 0; i < Min( range, BarCount ); i++ )
  {
      im = i - m;
      GaussBell[i] = Coeff[ i ] = exp( - ( im * im ) / ( 2 * s * s ) );
  }

  return FIR_AFL( input, Coeff, range );
}

data = ( H + L + C ) / 3;
conv = FIR_AFL( data, fw, sigma1);

is it possible to make it this code not look in the future (to make it trading system)?

Thanks ed

#18

hi,

the detrending is done here:

// calculate convolution filter
conv = ALMA_AFL( data, fw, 0.5, sigma1 );
filtered_data = data - conv;

the array conv contains a sort of moving average but I shifted it so it looks in the future. If you leave that part out, like:

// calculate convolution filter
// conv = ALMA_AFL( data, fw, 0.5, sigma1 );
filtered_data = data; // - conv;

then the filtered_data is just the data. But this will give you a very big peak at low frequencies. The filtering I have done to remove these low frequencies. You could also try doing the filtering by just subtrating a moving average. So like:

// calculate convolution filter
// conv = ALMA_AFL( data, fw, 0.5, sigma1 );
filtered_data = data - ma( C, 50 );

This will then filter out moves in the data with periods of about 50 bars or greater.

The idea of all of this was to see if the data contains "hidden" periodicities. If the amplitude of the periodicity is low and not directly visible in the data you could make it visible when you have a large amount of data using the Fourier transform. But I did not continue with it, seems like a waste of time.

1 Like