# Fast fourier transform FFT

#21

yes ok, i understand that can be easily added. Indeed multiplication in the time domain is convolution in the frequency domain so I guess this windowing will soften the edges of the analysis window which otherwise would add additional noise to the frequency spectrum.

But as far as I can see we calculated the the power spectrum correctly. It also says so in the manual, “len” data points, FFT returns “len” bars which are “len/2” complex bins. What I do not understand is that the theory talks about a number “len/2 + 1”. Maybe it is no issue. I see IDL has the same, the fft also returns “len” points.

i will probably have to read Hurst’s book to use it all correctly

#22

You may ask author of the book. The fact is that FFT (FAST Fourier Transform) algorithm performs in-place (so the number of input data is exactly the same as number of output data elements). This design is a principle of FFT so it can be done without allocating extra memory for the result. You are using FFT correctly.

#23

@empottasch if you are referring to J.M. Hurst, then the “len/2 + 1” could be for his centering based on an odd number.

If you have an even number, the “centred” value would fall between observations. If you have an odd number, you can centre on the middle observation and have an equal number before and after.

BTW - I spelled centre correctly. (Proudly Canadian)

#24

no that number comes from this pdf, page 148, http://www.dspguide.com/CH8.PDF

i just mentioned Hurst if I ever plan to use frequency analysis correctly. Basically I have set the tools ready to do so but want to do other things first. Intuitively I have my doubts frequency analysis will work. It would however be quite easy to detrend the data and use some edge dampening window over the data set and see if is any significant periodicity to be found. Will do that when I finish what I am currently working on and if I find something interesting will get back on the subject.

#25

made some nice code to analyse if any significant periodicity is visible in the data. This time I filter the data in the time domain using a Gaussian type function. When you start the code it will show the raw data and the Gauss filter. In the param window you can toggle to filtered data. Also you can toggle to the Frequency spectrum.

When you display the filtered data you can slide the chart all the way to bar 0 and then have a look at the Gauss filter. The filter can be changed in the param window (width and sigma).

the more data you use the more apparent any periodicity hidden in the data will become. So far I do not see any.

``````/*
© AFL code E.M.Pottasch, 11/2017
originated here: http://forum.amibroker.com/t/fast-fourier-transform-fft/3098/11

code is an attempt to explore the use of frequency analysis in trading

this version filters the data in the time domain using a Gaussian type function
*/
SetBarsRequired( sbrAll, sbrAll );

len = Param( "Length of Data rray", 2048, 16, 4096, 2 );
view = ParamToggle( "Frequency/Time Domain Toggle", "Frequency Domain| Time domain", 1 );
view2 = ParamToggle( "Raw/Filtered Data Toggle (Time Domain)", "Raw Data| Filtered Data", 0 );
bins1 = 0;
bins2 = len / 2;
fw = Param( "Filter Width (Bars)", 21, 1, 150, 2 );
sigma1 = Param( "Sigma", 5, 0.1, 50, 0.1 );

GaussBell = Null; // store the Gauss Bell for display purposes

function FIR_AFL( input, coeff, size )
{
result = 0;
sumcoeff = 0;

for( i = 0; i < Min( size, BarCount ); i++ )
{
sumcoeff += coeff[ i ];
result += coeff[ i ] * Ref( input, - int( size / 2 ) + i );
}

return result / sumcoeff;
}

function ALMA_AFL( input, range, Offset, sigma )
{
local m, im, s, Coeff;

m = floor( Offset * ( range - 1 ) );
s = range / sigma;

for( i = 0; i < Min( range, BarCount ); i++ )
{
im = i - m;
GaussBell[i] = Coeff[ i ] = exp( - ( im * im ) / ( 2 * s * s ) );
}

return FIR_AFL( input, Coeff, range );
}

data = ( H + L + C ) / 3;

// calculate convolution filter
conv = ALMA_AFL( data, fw, 0.5, sigma1 );
filtered_data = data - conv;

// DFT
ffc = FFT( filtered_data, len );

// Real and Imaginary Parts in Frequency Domain (Amibroker manual)
fRe = fIm = 0;

if( len < BarCount )
{
for( k = 0; k < ( len / 2 ); k++ )
{
fRe[k] = ffc[2 * k];
fIm[k] = ffc[2 * k + 1];
}

// http://www.dspguide.com/CH8.PDF, page 153
tRe  = 2 * fRe / len;
tRe[0]  = fRe[0] / len;
tRe[len / 2 - 1]  = fRe[len / 2 - 1] / len;
tIm  = -2 * fIm / len;

if( view )
{
if( view2 == 1 )
{
SetChartBkColor( colorBlack );
SetChartOptions( 1, chartShowDates, chartGridMiddle, 0, 0, 0 );
Plot( filtered_data, "Filtered Data", ColorRGB( 0, 0, 255 ), styleLine, Null, Null, 0, 0, 2 );
Plot( GaussBell, "", ColorRGB( 0, 250, 255 ), styleLine | styleOwnScale, Null, Null, 0, 0, 5 ); // Gauss Bell Plotted in the first few bars
}
else
if( view2 == 0 )
{
SetChartBkColor( colorBlack );
SetChartOptions( 1, chartShowDates, chartGridMiddle, 0, 0, 0 );
Plot( Close, "Price", colorDefault, styleCandle, Null, Null, 0, 0, 1 );
Plot( data, "Raw Data", ColorRGB( 0, 0, 255 ), styleLine, Null, Null, 0, 0, 2 );
Plot( conv, "Filter", ColorRGB( 250, 250, 255 ), styleLine, Null, Null, 0, 0, 2 );
}
}
else
{
// power spectrum calculation
power = ( tRe + tIm ) ^ 2;

GfxSetOverlayMode( 2 );
GfxSetCoordsMode( 0 );
GfxSelectPen( ColorRGB( 90, 90, 90 ), 1, 0 );

pxwidth = Status( "pxwidth" );
pxheight = Status( "pxheight" );
margin = int( pxwidth / 20 );

// chart border
GfxMoveTo( margin, margin );
GfxLineTo( pxwidth - margin, margin );
GfxLineTo( pxwidth - margin, pxheight - margin );
GfxLineTo( margin, pxheight - margin );
GfxLineTo( margin, margin );

GfxSelectFont( "Tahoma", 15, 700, False, False, 0 );
GfxTextOut( " Frequency " , pxwidth / 2, pxheight - margin / 2 );
GfxSelectFont( "Tahoma", 15, 700, False, False, 900 );
GfxTextOut( " Power " , margin / 2, pxheight / 2 );

maxpower = 0;

for( i = 0; i < ( len / 2 ); i++ )
{
if( power[i] > maxpower )
{
maxpower = power[i];
}
}

xbin = ( pxwidth - 2 * margin ) / ( ( len / 2 ) - 1 );
ybin = ( pxheight - 2 * margin ) / maxpower;

GfxSelectPen( ColorRGB( 0, 255, 255 ), 6, 0 );

for( i = 0; i < ( len / 2 ); i++ )
{
GfxMoveTo( margin + i * xbin, pxheight - margin );
GfxLineTo( margin + i * xbin, pxheight - margin - power[i]*ybin );
}

nyquist = 1 / ( 2 * Interval() );
RequestMouseMoveRefresh();
px = GetCursorXPosition( 1 );
py = GetCursorYPosition( 1 );
GfxSelectFont( "Tahoma", 10, 700, False, False, 0 );

if( px > margin AND px < pxwidth - margin )
{
pxr = ( px - margin ) / ( pxwidth - 2 * margin ) ;

freq = pxr * nyquist;
wavelength = ( 1 / freq ) / ( 24 * 60 * 60 );

GfxSetTextColor( ColorRGB( 0, 0, 0 ) );
GfxSetBkColor( ColorRGB( 255, 255, 0 ) );

if( px < pxwidth / 2 )
GfxTextOut( " Frequency: " + pxr * nyquist + " Hz, Wavelength: " + prec( wavelength, 2 ) + " days " , px + 15, py );

if( px > pxwidth / 2 )
GfxTextOut( " Frequency: " + pxr * nyquist + " Hz, Wavelength: " + Prec( wavelength, 2 ) + " days " , px - 400, py );
}
}
}
``````

Least Squares Fit (FIR Function / FIR_AFL function / ALMA Function)
#26

Thanks Ed, the indicator looks real nice…

Cheers,
Scott

#27

thanks, it is not really an indicator yet, i was just playing around with the FFT function. At the moment working on something else but might get back on the subject. Have to look at what others have used it for first. Like Hurst and I believe also Ehler.

Looking at the power spectrum of de-trended price data, looks kind of like noise to me but I did not study yet how these guys use frequency analysis. I like to study that a bit soon.

#28

Hello Ed. Been far from AB for a while.
Nice code.

#29

thanks, did not work on it since then. Working on “machine learning” at the moment but going slow. Just working on Bayes’ rule for now. Did you work on this already? I might start a new topic on this maybe this would allow me to move ahead faster

#30

Hello Edward,

I am also, working on machine learning solution.
Do you think, it will be appropriate to start a discussion on AB forum in regards to ML?

It have been pointed out to me that, with AB batch functionality we can simply run ML schikit-learn algorithm prediction in Python or R or anything and then return the results for backtesting and processing (i.e. like using IB interface to automate the process).

Looking forward to hear from you.
Kind Regards
Richard

#31

hi Rich,

yes go ahead, I didn’t start 1 yet because I did not make too much progress yet. I wanted to avoid using Python and just first try to understand what I am doing. I worked on Bayesian analysis a bit and this is easy to program in Amibroker. Only problem is that it is a vast topic so I am not sure this is the right approach (to want to do it all in AFL). Also because some of the math goes beyond me and then the question is whether I implement it correctly in AFL.

I think my main issue at the moment is to find some simple examples and work them out for myself in Amibroker.

i however, did not work this 1 out in Amibroker AFL yet.

#32

@empottasch @richpach - with regard to machine learning, you might be interested in these threads from the Yahoo mailing list:

Regards

#33

thanks, yes I know Aron has been working on it for a while. Also Howard Bandy was working on a book about the topic as far as I know. I think both Aron and Bandy believe this is a better approach to develop trading strategies.

as I said it is a vast topic. One can approach ML using different techniques. I am afraid it will take me a long time to fully understand what I am doing. Just have to keep at it, keep going until the brain is wired up sufficiently

With respect to its success I am not sure but I think guys like James Simons, Schwab and Thorpe have been using these techniques for years (maybe decades ?). As far as I know Ed Thorpe also used the Kelly Criterion for his trading which is also easy to implement in AFL, I have been playing with that for a bit.

#34

Regrds & happy holidays

#35

thanks, will have a look