I have an exploration that fills a watchlist with symbols but I want to rank those symbols so I made a new project (*.apx ) then I will add the project to a batch.
By searching forum I found a similar topic in which @Tomasz have suggested to rank before adding symbols to the list, that was simple and very effective.
shortly I used SetSortColumns function in the exploration and the results are satisfactory but upon updating the list with the new exploration results using CategoryAddSymbol function the symbols are not added as ranked in the exploration table instead they added in the original order before ranking.
Dear, @beppe thank you for your fast response. I dont know if my words describe the situation completely, but let me explain the situation in other wards. I already want to do the ranking on two steps using batch functionality so I have first exploration to assign symbols to a watchlist " say 55" in execution order Second I use second exploration to rank the symbols found in the watchlist "55" then re-update the whatchlist with the results of second exploration.
Tomasz have mentioned "addsymbolstowatchlist" I searched for this function but found nothing.
I revised Tomasz answer again, Now I have tried this but I don't know where is the missing part it stills not ranking at all is that right to use filter because I dont know
Filter=1;
symlist = CategoryGetSymbols( categoryWatchlist, 36 );
// delete static variables - DO NOT forget the asterisk (wildcard) at the end
StaticVarRemove( "ValuesToSort*" );
// fill input static arrays
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ )
{
SetForeign( sym );
Value = HMV; ///HMV is the variable I want symbols to be ranked accoding to their HMV value
RestorePriceArrays();
StaticVarSet( "ValuesToSort" + sym, Value );
}
// perform ranking
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode
StaticVarGenerateRanks( "top", "ValuesToSort", 5, 1224 ); // top-N mode
StaticVarGenerateRanks( "bot", "ValuesToSort", -5, 1224 ); // bottom-N mode
// read ranking
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ )
{
Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i );
}
//sdt = SelectedValue( DateTime() );
//Title = "{{NAME}} -{{DATE}} - {{VALUES}} TOP: " + StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt ) + " BOT: " + StaticVarGetRankedSymbols( "bot", "ValuesToSort", sdt ) ;
List=StaticVarGetRankedSymbols( "rank", "ValuesToSort", BarCount-1 );
//AddColumn (List3,"rank",1.2);
CategoryAddSymbol(List, categoryWatchlist, 55 );
Dear, @beppe thank you for your fast response. I dont know if my words describe the situation completely, but let me explain the situation in other wards. I already want to do the ranking on two steps using batch functionality so I have first exploration to assign symbols to a watchlist " say 55" in execution order Second I use second exploration to rank the symbols found in the watchlist "55" then re-update the whatchlist with the results of second exploration.
Tomasz have mentioned "addsymbolstowatchlist" I searched for this function but found nothing.
I revised Tomasz answer again, Now I have tried this but I don't know where is the missing part it stills not ranking at all is that right to use filter because I don't know the benefit for defining "symlist" in the beginning of the code
Filter=1;
symlist = CategoryGetSymbols( categoryWatchlist, 36 );
// delete static variables - DO NOT forget the asterisk (wildcard) at the end
StaticVarRemove( "ValuesToSort*" );
// fill input static arrays
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ )
{
SetForeign( sym );
Value = HMV; ///HMV is the variable I want symbols to be ranked accoding to their HMV value
RestorePriceArrays();
StaticVarSet( "ValuesToSort" + sym, Value );
}
// perform ranking
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode
StaticVarGenerateRanks( "top", "ValuesToSort", 5, 1224 ); // top-N mode
StaticVarGenerateRanks( "bot", "ValuesToSort", -5, 1224 ); // bottom-N mode
// read ranking
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ )
{
Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i );
}
List=StaticVarGetRankedSymbols( "rank", "ValuesToSort", BarCount-1 );
CategoryAddSymbol(List, categoryWatchlist, 55 );
@mohamed.gad try this way (change the values to rank and the watchlists as needed):
nSrcWatchList = -1;
// optionally hardcode a watchlist number
// nSrcWatchList = 8; // TO_CHANGE / PARAMETRIZE
nDestWatchList = 87; // TO_CHANGE / PARAMETRIZE
if( Status( "Action" ) == actionExplore )
{
cat = categoryWatchlist;
// if not hard-coded choose a single watchlist in the Analysis filter settings
if( nSrcWatchList == -1 )
{
if( GetOption( "ApplyTo" ) == 2 ) // filter
{
nSrcWatchList = GetOption( "FilterIncludeWatchlist" );
}
}
if( nSrcWatchList >= 0 )
{
if( Status( "Stocknum" ) == 0 )
{
// empty the dest watchlist
destSymList = CategoryGetSymbols( cat, nDestWatchList );
destSymCount = StrCount( destSymList, "," ) + 1;
for( i = destSymCount - 1; i >= 0; i-- )
{
sym = StrExtract( destSymList, i );
CategoryRemoveSymbol( sym, cat, nDestWatchList );
}
// delete static variables - DO NOT forget the asterisk (wildcard) at the end
StaticVarRemove( "ValuesToSort*" );
StaticVarRemove( "topValuesToSort*" );
symlist = CategoryGetSymbols( cat, nSrcWatchList );
symCount = StrCount( symList, "," ) + 1;
// fill input static arrays
for( i = 0; i < symCount; i++ )
{
sym = StrExtract( symlist, i );
SetForeign( sym );
values = ROC( C, 10 ); // TO_CHANGE HERE AND BELOW IN EXPLORATION
StaticVarSet( "ValuesToSort" + sym, values );
RestorePriceArrays();
}
// perform top ranking
StaticVarGenerateRanks( "top", "ValuesToSort", symCount, 1234 );
sdt = SelectedValue( DateTime() ); // in AA this is the Last date in range
// this next function retirns the sorted symbols!
listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );
for( i = 0; i < symCount; i++ )
{
sym = StrExtract( listTop, i );
CategoryAddSymbol( sym, cat, nDestWatchList );
}
// relase static var memory
StaticVarRemove( "ValuesToSort*" );
StaticVarRemove( "topValuesToSort*" );
}
// EXPLORATION USED ONLY TO VERIFY THE RANKING
// if hard-coded use the same nSrcWatchList as a filter in AA for the
// exploration to verify grid results vs. the ranked watchlist
// display only last bar from the Analysis range
Filter = Status( "lastbarinrange" );
AddColumn( ROC( C, 10 ), "ROC(10)" ); // Show the same array as used in ranking (values)
SetSortColumns( -3 );
AddRankColumn();
}
else
Error( "Please select a single watchlist to rank" );
}
Thank you @beppe , I tried the code it arranged the symbols in different way than original but unfortunately the arrangement differ than exploration results.
her is the result "The only change is nDestWatchList = 55 (the list number where the result of ranking will be stored in) is that right or I have to do any thing more?
@mohamed.gad, sorry for the late answer, but I was away from my computer (from time to time I sleep )
I verified it, and here it seems to work as expected.
I used it with ApplyTo set to 1 recent bar and since it depends on SetForeign() you should be sure that all the data in the source watchlist is valid aligned (no data holes). Enable Pad&Align using a reference symbol that is completely filled and updated.
Wrong results in ranking due to missing data/misalignments have been discussed multiple time is the forum - search for "holes").
So, please, check this condition and let me know if it solves the issue, otherwise, I will further investigate the matter.
Dear @beppe , thank you for your generosity, you saved me since I spent half Friday to find the solution and till I found your reply I was trying to solve it. You are absolutely right when you said
that solved the problem.
Only I want to say Thank you, before you go sleep.
Please, note that In my code I ranked with "top" ALL the symbols.
Obviously, you can decide to apply that ranking limiting it to topN (5, 10, etc) symbols to reduce the length of the destination watchlist.
And if needed you can repeat this process multiple times (all in the "stocknum==0 block) using different "values" to further reduce the final list candidates.
For this use the resulting previous "toplist" as a srcList for the additional steps, generating the actual watchlist ONLY when the complete "screening" process is complete.
Then you can use the "optimal" watchlist in any other formula.
This is the kind of stuff well suited for batch processing: in a formula like the above you create the "universe" of stocks to use, then in another one (via AB batch) you apply your trading system rules to it.
Thank you @beppe that indeed is very good idea sure I will further incorporate instead of hard
and & for statements of AFL coding specially for beginner like me.
Dears @beppe I have found some mysterious behavior of ranking upon using more complex ranking function the ranking get worse,when I use ROC as ranking value the ranking is the same in the explorer and the list but when I used Tushar Chande Momentum Oscillator instead of simple ROC the rank differs between the explorer and the list. I think this is related to the StaticVarGenerateRanks or StaticVarSet or may be data holes.
I have seen that the ranking order and the setSortColumns() order may be different when the two or more values are the same (a tie condition - check the 3rd column values to see if there are any ties).
In any case, later today, I may further test my formula with some other ranking values (but in principle, it should not change anything: it will function or not).
Anyway, mistakes in my code are always possible: if anyone spots an issue and suggests how to fix it, he/she is welcome.
Dear @beppe I don't mean your code Because It works nicely for any simple ranking function and I think it will work on more complex . what we are don't know is the sensitivity of the built in functions to data containing holes. especially the rank functions that contains logical statements like iif and < or > operators.
Thank you, and I will try to investigate more and to feed you back if I got something meaningful.
Dear , @beppe I did some trials and It seems that the data holes greatly affect the ranking process, indicated by applying the rank to carefully selected symbols (main market indexes) whom I believe the most complete data I have and the result was perfect even with moderate complex ranking formula but upon complicating the formula more by using some higher order factors (exponential) it get some troubles again may be the approximations play some rule here.
Thank you @beppe for sharing the above code,very kind of you as always.
About the aforesaid code:
We have 300 symbols in the Source WatchList.
And to get only Top 10 Ranked Symbols in the Destination WatchList for the second Exploration, slightly modified one line in the following part as follows:
for( i = 0; i < symCount-290; i++ )
// perform top ranking
StaticVarGenerateRanks( "top", "ValuesToSort", symCount, 1234 );
sdt = SelectedValue( DateTime() ); // in AA this is the Last date in range
// this next function retirns the sorted symbols!
listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );
for( i = 0; i < symCount-290; i++ )
{
sym = StrExtract( listTop, i );
CategoryAddSymbol( sym, cat, nDestWatchList );
}
This fills the Destination WatchList with only the Top 10 Ranked Symbols (Desired Result).
Is this the Right way to do it ? , is there anyway we can do this without the prior knowledge of the total number of Symbols in the Source WatchList ?
Thank you once again @Beppe for your time n patience.
Before the main part of the code, define a new variable that will limit the size of the destination watchlist items:
topN = 5; // max number of symbols to include in the destination watchlist - Parametrize it if you like
Then modify the code as follows;
// perform top ranking
StaticVarGenerateRanks( "top", "ValuesToSort", topN, 1234 ); // modified line
sdt = SelectedValue( DateTime() ); // in AA this is the Last date in range
// this next function returns the sorted symbols!
listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );
symCount = StrCount( listTop, "," ) + 1; // line added - get the top list symbols count
for( i = 0; i < symCount; i++ )
{
sym = StrExtract( listTop, i );
CategoryAddSymbol( sym, cat, nDestWatchList );
}
By the way, this is the "normal" way to use the "top-N" mode ranking.
My example was using all the symbols since the request was to put them all in the watchlist, but the real advantage of the "top-N" method is to limit the resulting list size to include only the top ranked values!
And if your topN value is greater than the original watchlist symbol count no problem: it will work anyway!
Completely misread the third argument there in the Function Reference , thought it was 0
for normal Mode(Top) and -1 for Bottom Mode, nothing more.
Is it possible to switch to the New Destination watchList for a new/next exploration execution in the same afl without manuaally changing the Applyto filter ?
@NowToLook re point 3 I think it will NOT work (when you execute the ranking process you are already in the exploration process using the original watchlist).
In my opinion, the proper way to use this stuff is to use a 2-step process: execute a first formula, based on this kind of code, do the ranking (optionally a multiple level ranking as I explained before), save the resulting "ranked" watchlist and then use a second formula to do your standard exploration utilizing the destination watchlist.
The entire process can easily be automated via a batch (.abb) saving the intermediate steps/settings as .apx files.
I was thinking that the code to clear a watchlist and fill it with the exploration results seems a good candidate for a new kind of "batch" operation to add to future versions of AmiBroker.
In practice, I wonder if it could be useful to post in the Feedback Center the request to add to the Batch Windows operations these two items available via a popup menu on the exploration analysis results:
Does not seems a lot of work for @Tomasz, but only he can tell us!