When adjusting parameter values, such as the number of bars to include in a moving average or the critical level at which an indicator issues a signal – such as cross(RSI(2),30) – each value that is tested represents a “degree of freedom” in the optimization. Testing a moving average crossover for lengths of 5, 10, 15 versus 10, 20, 30, 40 uses 12 degrees of freedom in the “search space.” “p” has a value of 12. One year of daily data has 252 data points. “n” has a value of 252. The total number of degrees of freedom used in an optimization is the product of all of the "p"s from all of the parameters.
The model fitting process is very much like fitting a polynomial through a number of data points. When the degree of the polynomial is equal to or greater than the number of data points – p greater than n, in the jargon – the fit is guaranteed to be perfect.
For example, a third degree polynomial (having four coefficients) will always perfectly fit four data points. It will not only perfectly fit fewer than four data points, it will give multiple perfect fits, and the modeler must treat all of them as equally likely.
Importantly, each symbol tested also represents a degree of freedom. Optimizing, or even just testing for a fixed model, a large number of tickers is “optimizing the symbol space.” Each ticker contributes to the “p”. That optimization will identify some great looking results. Be wary that they are not the result of a really good model, but rather the lucky fit to random data.
The post is rather old but I needed an individual optimization with custom backtester.
This one runs also with the new optimization on custom. It takes all symbols. Just skip the ~~Equity etc
/*
*
* BatchInidvidualOptiimize.js
* run with cscript.exe BatchInidvidualOptiimize.js
*
*/
/* The directory where AFL files are stored
** Also reports generated by the bactest
** will be saved here
*/
//AFLFolder = C:\\Program Files\\AmiBroker\\AFL; // MODIFY TO FIT YOUR SETUP
//AFLFolder ="C:\\AmiBroker\\Reports2\\afl";
AFLFolder ="C:\\Program Files (x86)\\AmiBroker\\Reports2\\";
WScript.Echo("Batch testing of all AFL files stored in" + AFLFolder );
var AB, AA;
var fso, f, f1, fc, s;
var filename;
/* Create AmiBroker object and get Analysis object */
AB = new ActiveXObject("Broker.Application");
AA = AB.Analysis;
AD = AB.Documents;
/*AA.LoadFormula(E:\\AmiBroker\\myScripts\\firstSystem3.afl);*/
AA.LoadFormula("r:\\Optionextremes\\optionextrems\\SingleOpt.afl");
//filename = "C:\\Program Files (x86)\\Reports2\\firstSystem3.log";
filename = "C:\\Batchami\\.log";
/* backtest over symbols and all quotes*/
AA.ClearFilters();
//AA.ApplyTo = 0; // use symbols
AA.RangeMode = 0; // all quotes
// to use filters you should uncomment lines below
//Analysis.ApplyTo - defines apply to mode: 0 - all stocks, 1 - current stock, 2 - use filter
AA.ApplyTo = 1; // current stock
//AA.Filter(0,watchlist) = 1 /* watch list number */;
// AA.Filter(0,group) = 0 /* group number */;
// Then Pick one option from below
/*==
Option one (loop through all stocks)
==
*/
var oStocks = AB.Stocks;
var Qty = oStocks.Count;
//iQty = Quotes.Count;
iQty = oStocks.Count;
// If you have ~~~EQUITY (etc.) use this
//Qty = Qty - 1 ; // remove ~~~EQUITY, increase for other ~~fields
//for( i = 0; i < Qty; i++ )
for( i = iQty - 1; i >= 0; i-- )
{
oStock = oStocks( i );
var ticker = oStock.Ticker;
WScript.Echo("Batch opt" + ticker );
AD.Open(ticker)
//reportname = filename.substr( 0, filename.length - 4 ) + ticker + ".CSV" ;
reportname = filename.substr( 0, filename.length - 4 ) + ticker + ".html" ;
AA.Optimize(Type = 1 ); //- runs optimization
WScript.Echo("Write to:" + reportname );
//AA.SortByColumn( 11, 0, 0 );
AA.Export( reportname );
}
I optimizded around 200 stocks individual. Afterwards I used a python script to exerpt the optimal parameters from the html files for each stock
The functionality is available OUT OF THE BOX, no code is necessary.
It is called "Individual Optimization" and is available from Analysis
DROP DOWN MENU that can be displayed when you click on ARROW on the "Optimize" button