Exploration filter: get top 30 of positionscore

AFL Programming
#1

Hi guys, I'm trying to run exploration to get top 30 of postionscore., searched but failed to future it out.

Could you please share some code snippet?

Thanks!

1 Like
Producing a sorted report of only top N rows
#2

General purpose ranking
https://www.amibroker.com/guide/afl/staticvargenerateranks.html

///////////////////////////////// 
// Example 1. code for normal ranking mode 
// (everything done is done in one pass, can be used in indicator): 
///////////////////////////////// 

 symlist = "C,CAT,DD,GE,IBM,INTC,MSFT"; 

// delete static variables - DO NOT forget the asterisk (wildcard) at the end 
StaticVarRemove( "ValuesToSort*" ); 

// fill input static arrays 

for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
 { 
     SetForeign( sym ); 
      Value = ROC( C, 10 ); 
     RestorePriceArrays(); 
     StaticVarSet( "ValuesToSort" + sym, Value ); 
 } 

// perform ranking 
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode 

// read ranking 
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
 { 
     Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i ); 
 } 

///////////////////////////////// 
// Example 2. Code for top ranking mode 
// (everything done is done in one pass, can be used in indicator): 
///////////////////////////////// 

 symlist = "C,CAT,DD,GE,IBM,INTC,MSFT"; 

// delete static variables - DO NOT forget the asterisk (wildcard) at the end 
StaticVarRemove( "ValuesToSort*" ); 

// fill input static arrays 

for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
 { 
     SetForeign( sym ); 
      Value = ROC( C, 10 ); 
     RestorePriceArrays(); 
     StaticVarSet( "ValuesToSort" + sym, Value ); 
 } 

// perform ranking 
StaticVarGenerateRanks( "rank", "ValuesToSort", 0, 1224 ); // normal rank mode 
StaticVarGenerateRanks( "top", "ValuesToSort", 3, 1224 ); // top-N mode 
StaticVarGenerateRanks( "bot", "ValuesToSort", -3, 1224 ); // bottom-N mode 

// read ranking 
for ( i = 0; ( sym = StrExtract( symlist, i ) ) != ""; i++ ) 
 { 
     Plot( StaticVarGet( "RankValuesToSort" + sym ), sym, colorCustom10 + i ); 
 } 

 sdt = SelectedValue( DateTime() ); 

Title = "{{NAME}} -{{DATE}} - {{VALUES}} TOP: " + StaticVarGetRankedSymbols( "top", "ValuesToSort", sdt ) + 
         " BOT: " + StaticVarGetRankedSymbols( "bot", "ValuesToSort", sdt ) ;

How to create own exploration
http://www.amibroker.com/guide/h_exploration.html

So based upon upper StaticVarGenerateRanks examples of AB manual.

rank = StaticVarGet( "RankValuesToSort" + Name()); 

Filter = rank <= 30;

AddColumn( rank, "Rank", 1.0);

Another hint: Don't forget to add ranks creation within

/// only at first symbol of symbols list
if ( Status( "stocknum" ) == 0 ) {

    /// StaticVarGenerateRanks code here.
    /// (which includes StaticVarRemove and loop code of before also)
}
5 Likes
#3

Thank you but the StaticVarGenerateRanks look too complicated and consume much CPU. Below is my code. I created a rank column but i only need to limit it to the top 30.
Any better idea?

Filter = Buy OR Sell OR Short AND OR Cover; 	
AddColumn( Buy, "Buy", 1 );
AddColumn( Short, "Short", 1 );
AddColumn( Sell, "Sell", 1 );
AddColumn( Cover, "Cover" ,1);
AddColumn(O,"Open");
AddColumn(PositionScore,"PositionScore");
AddColumn( rank, "Rank", 1.0); 
SetSortColumns( -8 );
AddRankColumn(); // rank according to 4th column (descending)
#4

Seriously you got response already.
I wrote General purpose ranking (as written in AB documentation. So obviously it is made for that specifically).
And once again ranking should be applied on first symbol only and with pad and align being enabled.

Are you able to read post (someone took time to create) from top to bottom carefully??

Then invent something yourself if you know better. I am not posting to waste time.

#5

I do read the post and wrote the below code, but it is super slow. I explore 1000 symbols, generated about 300 symbol's signal but only use top 30 of them.

Value = PositionScore;
StaticVarSet( "ValuesToSort" + Name(), Value );
StaticVarGenerateRanks( "rank", "ValuesToSort", 30, 1224 );
Rank = StaticVarGet( "RankValuesToSort" + Name()); 
Filter = Rank <= 30;
AddColumn( Rank, "Rank", 1.0);
#6

Your code is incorrect!

No you did not read carefully!

And you have to iterate (on first symbol) as posted in the examples of AB manual.
Is it really difficult to read posts carefully?
Why do you invent own code instead of following examples?

And FYI, use forum search as topics and examples exist already.

#7

Hi @fxshrat,
Below are the updated full codes. When I run exploration, Amibroker takes hours. What's the problem?

// ################ Ranking Exploration #################

// watchlist should contain all symbols included in the test
wlnum = GetOption( "FilterIncludeWatchlist" );
List = CategoryGetSymbols( categoryWatchlist, wlnum ) ;

if( Status( "stocknum" ) == 0 )
{
    // cleanup variables created in previous runs (if any)
    StaticVarRemove( "rank*" );
    StaticVarRemove( "values*" );
    categoryList = ",";

    for( n = 0; ( Symbol = StrExtract( List, n ) )  != "";  n++ )
    {
        SetForeign( symbol );

        // write our ranking criteria to a variable
        // in this example we will use 10-bar rate-of-change
        values = PositionScore;

        RestorePriceArrays();

        // write ranked values to a static variable
        StaticVarSet( "values" +  "_" + symbol, values );

    }

    // generate  ranks
        StaticVarGenerateRanks( "rank", "values" + "_", 3, 1224 );
    
}


symbol = Name();

values = StaticVarGet( "values" + "_" + symbol );
rank = StaticVarGet( "rank" + "values" + "_" + symbol );

// exploration code for verification
AddColumn( values, "values" );
AddColumn( rank, "rank" );
Filter = rank <= 2;
#8

First positionsscore is not initialized. To be ranked array has to be within Setforeign!
Secondly you should use normal ranking.

On my end it is fast once symbols are in memory. Below 1 second on 1000 symbols.

// ################ Ranking Exploration #################

// watchlist should contain all symbols included in the test
wlnum = GetOption( "FilterIncludeWatchlist" );
List = CategoryGetSymbols( categoryWatchlist, wlnum ) ;

if( Status( "stocknum" ) == 0 )
{
    // cleanup variables created in previous runs (if any)
    StaticVarRemove( "rank*" );
    StaticVarRemove( "values*" );
    categoryList = ",";

    for( n = 0; ( Symbol = StrExtract( List, n ) )  != "";  n++ )
    {
        SetForeign( symbol );

        // write our ranking criteria to a variable
        // in this example we will use 10-bar rate-of-change
        values = ROC(C,10);

        RestorePriceArrays();

        // write ranked values to a static variable
        StaticVarSet( "values_" + symbol, values );

    }

    // generate  ranks
    StaticVarGenerateRanks( "rank", "values_", 0, 1224 );
    
}

symbol = Name();
values = StaticVarGet( "values_" + symbol );
rank = StaticVarGet( "rankvalues_" + symbol );

// exploration code for verification
AddColumn( values, "values" );
AddColumn( rank, "rank" );
Filter = rank <= 2;

I am out of this thread. Nothing more to add...

3 Likes
#9

As @Tomasz stated, StaticVarGenerateRanks is not a economical method since it use too much resources

Generally the process involves creating static variables with values to be used for sorting/ranking, i.e. "scores" and then calling a special function (StaticVarGenerateRanks) that generates new set of static variables that hold calculated ranks.

NOTE: This function is NOT intended to replace bakctester's built-in ranking via PositionScore. Just the opposite: whenever you can, you should use PositionScore as it is way way faster and less memory-consuming way to perform backtests with ranking.

StaticVarGenerateRanks is generally intended to be used for tasks OTHER than backtesting such as explorations or indicators that may require ranking functionality, but of course it can also be used for backtesting when/where PositionScore alone does not allow to implement what you need in your trading system.

WARNING: this function is computationally and memory intensive. It takes about 20ms per 15K bars and 7 symbols. Try to call it just once per scan/exploration/backtest using if( Status("stocknum")==0) or better yet, use separate scan just once to pre-calculate ranks and use it later (like composite creation scan). If you fail to do so and call StaticVarGenerateRanks for every symbol performance would drop significantly as this function not only needs lots of time to compute but it also has to lock the access to shared memory used by static variables so other threads trying to access static variables would wait until this function completes.

#10

You should read carefully!

He talks about BACKTESTING when talking about PositionScore.

You want to output rank limit in EXPLORER!

Got it?

You are comparing apples to oranges. Once again a beginner failing to understand.

StaticVarGenerateRanks

This function is NOT intended to replace backtester's

use PositionScore as it is way way faster and less memory-consuming way to perform backtests with ranking.

StaticVarGenerateRanks is generally intended to be used for tasks OTHER than backtesting

but of course it can also be used for backtesting when/where PositionScore alone does not allow to implement what you need in your trading system.

If you want to perform backtest then use BACKTESTER (with PositionScore) but not explorer.

You are really starting to troll here.

3 Likes
#11

:grinning:

What about using below code?


Filter = C > O;// add your initial filter here

/// output only n-recent true filter occurrences
/// @link https://forum.amibroker.com/t/limit-exploration-results-to-a-count/12622/2
n = 3;// set N most recent filter occurrences
cum_filt = Cum(Filter);
Filter = Filter AND cum_filt >= Max(1,LastValue(cum_filt)-(n-1));

SetSortColumns(-2, 1);

AddColumn(cum_filt, "Cum(Filter)", 1 );
AddColumn(O, "Open", 1.4 );
AddColumn(C, "Close", 1.4 );
#12

Another fail of understanding on your end. Apples to oranges.
Do you actually listen?

1 Like
#13

You could do that in Exploration without StaticVarGenerateRanks if you use batch consisting of:

  1. Exploration with SetSortColumns
  2. Export to CSV
  3. Formula that reads CSV and trims it at N-top rows

But I doubt if it would be faster than StaticVarGenerateRanks.

1 Like
#14

Thanks! My original procedure is:

  1. backtest code
  2. filter = buy or sell or short or cover;
  3. auto-explore 1000 symbols, generated about 300 symbol's signal but only use the top 30 of the signal,
  4. auto trading with IB

What could be the best approach?

1 Like