Adding symbols to watchlist in specific order

Dears,

I have an exploration that fills a watchlist with symbols but I want to rank those symbols so I made a new project (*.apx ) then I will add the project to a batch.

By searching forum I found a similar topic in which @Tomasz have suggested to rank before adding symbols to the list, that was simple and very effective.
shortly I used SetSortColumns function in the exploration and the results are satisfactory but upon updating the list with the new exploration results using CategoryAddSymbol function the symbols are not added as ranked in the exploration table instead they added in the original order before ranking.

any suggestion or help would be appreciated

Thank you in advance,

AddColumn(V1,"V1",1.2);
SetSortColumns(-3);
CategoryAddSymbol( "", categoryWatchlist, 55 );

@mohamed.gad I think that you saw this thread.

The answer is exactly in the @Tomasz answer:

Note that SetSortColumns() function sorts columns AFTER exploration is complete. Your formula is NOT running at that point....

Re-read his post and then either implement a ranking procedure or, as suggested, do it in 2 steps via the batch functionality.

Dear, @beppe thank you for your fast response. I dont know if my words describe the situation completely, but let me explain the situation in other wards. I already want to do the ranking on two steps using batch functionality so I have first exploration to assign symbols to a watchlist " say 55" in execution order Second I use second exploration to rank the symbols found in the watchlist "55" then re-update the whatchlist with the results of second exploration.

Tomasz have mentioned "addsymbolstowatchlist" I searched for this function but found nothing.

I revised Tomasz answer again, Now I have tried this but I don't know where is the missing part it stills not ranking at all is that right to use filter because I dont know

Filter=1;
symlist = CategoryGetSymbols( categoryWatchlist, 36 );

// delete static variables - DO NOT forget the asterisk (wildcard) at the end 
StaticVarRemove( "ValuesToSort*" ); 

// fill input static arrays 

for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
{ 
    SetForeign( sym ); 
    Value = HMV; ///HMV is the variable I want symbols to be ranked accoding to their HMV value
    RestorePriceArrays(); 
    StaticVarSet( "ValuesToSort" + sym, Value ); 
} 

// perform ranking 
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode 

StaticVarGenerateRanks( "top", "ValuesToSort", 5, 1224 ); // top-N mode 

StaticVarGenerateRanks( "bot", "ValuesToSort", -5, 1224 ); // bottom-N mode 

// read ranking 
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
{ 
    Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i ); 
} 

//sdt = SelectedValue( DateTime() ); 

//Title = "{{NAME}} -{{DATE}} - {{VALUES}} TOP: " + StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt ) + " BOT: " + StaticVarGetRankedSymbols( "bot", "ValuesToSort", sdt ) ;
List=StaticVarGetRankedSymbols( "rank", "ValuesToSort", BarCount-1 );
//AddColumn (List3,"rank",1.2);
CategoryAddSymbol(List, categoryWatchlist, 55 );

Dear, @beppe thank you for your fast response. I dont know if my words describe the situation completely, but let me explain the situation in other wards. I already want to do the ranking on two steps using batch functionality so I have first exploration to assign symbols to a watchlist " say 55" in execution order Second I use second exploration to rank the symbols found in the watchlist "55" then re-update the whatchlist with the results of second exploration.

Tomasz have mentioned "addsymbolstowatchlist" I searched for this function but found nothing.

I revised Tomasz answer again, Now I have tried this but I don't know where is the missing part it stills not ranking at all is that right to use filter because I don't know the benefit for defining "symlist" in the beginning of the code

Filter=1;
symlist = CategoryGetSymbols( categoryWatchlist, 36 );

// delete static variables - DO NOT forget the asterisk (wildcard) at the end 
StaticVarRemove( "ValuesToSort*" ); 

// fill input static arrays 

for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
{ 
    SetForeign( sym ); 
    Value = HMV; ///HMV is the variable I want symbols to be ranked accoding to their HMV value
    RestorePriceArrays(); 
    StaticVarSet( "ValuesToSort" + sym, Value ); 
} 

// perform ranking 
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode 

StaticVarGenerateRanks( "top", "ValuesToSort", 5, 1224 ); // top-N mode 

StaticVarGenerateRanks( "bot", "ValuesToSort", -5, 1224 ); // bottom-N mode 

// read ranking 
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
{ 
    Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i ); 
} 
List=StaticVarGetRankedSymbols( "rank", "ValuesToSort", BarCount-1 );
CategoryAddSymbol(List, categoryWatchlist, 55 );

@mohamed.gad try this way (change the values to rank and the watchlists as needed):

nSrcWatchList = -1;
// optionally hardcode a watchlist number
// nSrcWatchList = 8; // TO_CHANGE / PARAMETRIZE

nDestWatchList = 87; //  TO_CHANGE / PARAMETRIZE

if( Status( "Action" ) == actionExplore )
{
    cat = categoryWatchlist;

    // if not hard-coded choose a single watchlist in the Analysis filter settings
    if( nSrcWatchList == -1 )
    {
        if( GetOption( "ApplyTo" ) == 2 ) // filter
        {
            nSrcWatchList = GetOption( "FilterIncludeWatchlist" );
        }
    }

    if( nSrcWatchList >= 0 )
    {
        if( Status( "Stocknum" ) == 0 )
        {
            // empty the dest watchlist
            destSymList = CategoryGetSymbols( cat, nDestWatchList );
            destSymCount = StrCount( destSymList, "," ) + 1;

            for( i = destSymCount - 1; i >= 0; i-- )
            {
                sym = StrExtract( destSymList, i );
                CategoryRemoveSymbol( sym, cat, nDestWatchList );
            }

            // delete static variables - DO NOT forget the asterisk (wildcard) at the end
            StaticVarRemove( "ValuesToSort*" );
            StaticVarRemove( "topValuesToSort*" );

            symlist = CategoryGetSymbols( cat, nSrcWatchList );
            symCount = StrCount( symList, "," ) + 1;

            // fill input static arrays
            for( i = 0; i < symCount; i++ )
            {
                sym = StrExtract( symlist, i );
                SetForeign( sym );
                values = ROC( C, 10 ); // TO_CHANGE HERE AND BELOW IN EXPLORATION
                StaticVarSet( "ValuesToSort" + sym, values );
                RestorePriceArrays();
            }

            // perform top ranking
            StaticVarGenerateRanks( "top", "ValuesToSort", symCount, 1234 );
            sdt = SelectedValue( DateTime() );  // in AA this is the Last date in range
            // this next function retirns the sorted symbols!
            listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );

            for( i = 0; i < symCount; i++ )
            {
                sym = StrExtract( listTop, i );
                CategoryAddSymbol( sym, cat, nDestWatchList );
            }

			// relase static var memory
			StaticVarRemove( "ValuesToSort*" );
			StaticVarRemove( "topValuesToSort*" );
        }

        // EXPLORATION USED ONLY TO VERIFY THE RANKING
        // if hard-coded use the same nSrcWatchList as a filter in AA for the
        // exploration to verify grid results vs. the ranked watchlist

        // display only last bar from the Analysis range
        Filter = Status( "lastbarinrange" );
        AddColumn( ROC( C, 10 ), "ROC(10)" ); // Show the same array as used in ranking (values)
        SetSortColumns( -3 );
        AddRankColumn();
    }
    else
        Error( "Please select a single watchlist to rank" );
}
4 Likes

Thank you @beppe , I tried the code it arranged the symbols in different way than original but unfortunately the arrangement differ than exploration results.
her is the result "The only change is nDestWatchList = 55 (the list number where the result of ranking will be stored in) is that right or I have to do any thing more?
rank

Best regards

@mohamed.gad, sorry for the late answer, but I was away from my computer (from time to time I sleep :smiley:)

I verified it, and here it seems to work as expected.

I used it with ApplyTo set to 1 recent bar and since it depends on SetForeign() you should be sure that all the data in the source watchlist is valid aligned (no data holes). Enable Pad&Align using a reference symbol that is completely filled and updated.

Wrong results in ranking due to missing data/misalignments have been discussed multiple time is the forum - search for "holes").

So, please, check this condition and let me know if it solves the issue, otherwise, I will further investigate the matter.

2 Likes

Dear @beppe , thank you for your generosity, you saved me since I spent half Friday to find the solution and till I found your reply I was trying to solve it. You are absolutely right when you said

that solved the problem.

Only I want to say Thank you, before you go sleep.:grinning:

Thanks.

Please, note that In my code I ranked with "top" ALL the symbols.

Obviously, you can decide to apply that ranking limiting it to topN (5, 10, etc) symbols to reduce the length of the destination watchlist.

And if needed you can repeat this process multiple times (all in the "stocknum==0 block) using different "values" to further reduce the final list candidates.
For this use the resulting previous "toplist" as a srcList for the additional steps, generating the actual watchlist ONLY when the complete "screening" process is complete.

Then you can use the "optimal" watchlist in any other formula.

This is the kind of stuff well suited for batch processing: in a formula like the above you create the "universe" of stocks to use, then in another one (via AB batch) you apply your trading system rules to it.

Have a nice weekend

Beppe

1 Like

Thank you @beppe that indeed is very good idea sure I will further incorporate instead of hard
and & for statements of AFL coding specially for beginner like me.

Thanks again, Have a nice weekend

Mohamed

Dears @beppe I have found some mysterious behavior of ranking upon using more complex ranking function the ranking get worse,when I use ROC as ranking value the ranking is the same in the explorer and the list but when I used Tushar Chande Momentum Oscillator instead of simple ROC the rank differs between the explorer and the list. I think this is related to the StaticVarGenerateRanks or StaticVarSet or may be data holes.

I have seen that the ranking order and the setSortColumns() order may be different when the two or more values are the same (a tie condition - check the 3rd column values to see if there are any ties).

In any case, later today, I may further test my formula with some other ranking values (but in principle, it should not change anything: it will function or not).

Anyway, mistakes in my code are always possible: if anyone spots an issue and suggests how to fix it, he/she is welcome.

Dear @beppe I don't mean your code Because It works nicely for any simple ranking function and I think it will work on more complex . what we are don't know is the sensitivity of the built in functions to data containing holes. especially the rank functions that contains logical statements like iif and < or > operators.

Thank you, and I will try to investigate more and to feed you back if I got something meaningful.

Mohamed

Dear , @beppe I did some trials and It seems that the data holes greatly affect the ranking process, indicated by applying the rank to carefully selected symbols (main market indexes) whom I believe the most complete data I have and the result was perfect even with moderate complex ranking formula but upon complicating the formula more by using some higher order factors (exponential) it get some troubles again may be the approximations play some rule here.

1 Like

Thank you @beppe for sharing the above code,very kind of you as always.

About the aforesaid code:

  1. We have 300 symbols in the Source WatchList.
  2. And to get only Top 10 Ranked Symbols in the Destination WatchList for the second Exploration, slightly modified one line in the following part as follows:
    for( i = 0; i < symCount-290; i++ )
// perform top ranking
            StaticVarGenerateRanks( "top", "ValuesToSort", symCount, 1234 );
            sdt = SelectedValue( DateTime() );  // in AA this is the Last date in range
            // this next function retirns the sorted symbols!
            listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );

            for( i = 0; i < symCount-290; i++ )
            {
                sym = StrExtract( listTop, i );
                CategoryAddSymbol( sym, cat, nDestWatchList );
            }
  1. This fills the Destination WatchList with only the Top 10 Ranked Symbols (Desired Result).
  2. Is this the Right way to do it ? , is there anyway we can do this without the prior knowledge of the total number of Symbols in the Source WatchList ?

Thank you once again @Beppe for your time n patience.

@NowToLook, it is simpler!

Before the main part of the code, define a new variable that will limit the size of the destination watchlist items:

topN = 5; // max number of symbols to include in the destination watchlist - Parametrize it if you like

Then modify the code as follows;

			// perform top ranking
			StaticVarGenerateRanks( "top", "ValuesToSort", topN, 1234 );  // modified line
			sdt = SelectedValue( DateTime() );  // in AA this is the Last date in range
			// this next function returns the sorted symbols!
			listTop = StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt );
			symCount = StrCount( listTop, "," ) + 1; // line added - get the top list symbols count

			for( i = 0; i < symCount; i++ )
			{
				sym = StrExtract( listTop, i );
				CategoryAddSymbol( sym, cat, nDestWatchList );
			}


By the way, this is the "normal" way to use the "top-N" mode ranking.

My example was using all the symbols since the request was to put them all in the watchlist, but the real advantage of the "top-N" method is to limit the resulting list size to include only the top ranked values!

And if your topN value is greater than the original watchlist symbol count no problem: it will work anyway!

3 Likes
  1. Yes its working, thank you @Beppe .

  2. Completely misread the third argument there in the Function Reference :smile:, thought it was 0
    for normal Mode(Top) and -1 for Bottom Mode, nothing more.

  3. Is it possible to switch to the New Destination watchList for a new/next exploration execution in the same afl without manuaally changing the Applyto filter ?

thank you

@NowToLook re point 3 I think it will NOT work (when you execute the ranking process you are already in the exploration process using the original watchlist).

In my opinion, the proper way to use this stuff is to use a 2-step process: execute a first formula, based on this kind of code, do the ranking (optionally a multiple level ranking as I explained before), save the resulting "ranked" watchlist and then use a second formula to do your standard exploration utilizing the destination watchlist.

The entire process can easily be automated via a batch (.abb) saving the intermediate steps/settings as .apx files.

1 Like

Yes, halfway there ,thanks , God Bless

I was thinking that the code to clear a watchlist and fill it with the exploration results seems a good candidate for a new kind of "batch" operation to add to future versions of AmiBroker.

In practice, I wonder if it could be useful to post in the Feedback Center the request to add to the Batch Windows operations these two items available via a popup menu on the exploration analysis results:

immagine

Does not seems a lot of work for @Tomasz, but only he can tell us!

3 Likes