Capture the list of symbols returned by explore

[LinusVanPelt]

I want to capture the list of symbols that would be returned by explore. I would then post-process that list after the list of symbols are generated. This is for utility AFL, not for a system.

The filter could be complex, such as:

• Market=ASX, Market=ASX Delisted Stocks, Match Any
• Watchlist=All Ordinaries, Sector=Technology, Match All
• Watchlist=All Ordinaries, Exclude=Stocks that are outliers
• Etc.

In most cases Filter=1; I just need the list of stocks that match the filter ("funnel" icon) for further processing.

[beppe]

@LinusVanPelt, if I understand your request, you can achieve that using the Batch functionalities that were added in version 6.31:

CHANGES FOR VERSION 6.31.0 (as compared to 6.30.0)

5. Batch: add “add results to watchlist” action / WatchlistAddResults
6. Batch: add “clear watchlist” action / WatchlistClear

Essentially, you'll create an exploration analysis project (.apx) with your (complex as needed) rules. Then you add it to an AmiBroker batch (.abb), run it (explore), and export(clear/add) the results to a new watchlist that will contain only the "filtered" symbols.

Then you'll use such generated watchlist for further processing.

[LinusVanPelt]

Thanks @beppe.

What I want to do is take the results of explore and randomly split the symbols, creating an in-sample and out-of-sample watchlist. If there is a way to do this in a single AFL that would be good.

Otherwise, I deduce from your reply that I can create two .apxs, two .afls called by the two .apxs, with the first .apx running the explore, the second using that watchlist as input for the random split, and a batch file to glue them all together.

I was hoping something like this would work, but I suspect explore/amibroker doesn't work this way:

StaticVarRemove("svTest");
StaticVarSetText("svTest","");
if (Status("action") == actionExplore) {
   Filter=1;
   StaticVarSetText("svTest",StaticVarGetText("svTest")+Name()+",");
}
_TRACE(StaticVarGetText("svTest"));

// further processing downstream from the explore

If I could capture the symbol list in a comma separated string I think I could work out the rest.

[beppe]

@LinusVanPelt, yes. I think this is the proper way to do it.

You can do it easily using the batch (.abb), or you can achieve it using OLE and external scripts, but in any case, it seems a two-step process: create a new watchlist/process the new watchlist.

[Henri]

If you want more control over your watchlist, you could export your explore results to a csv and then manualy edit that csv, then after that create your watchlist.

You then have 100% control over your final watchlist...

But my guess is that you allready figured that out, you just want it all done automaticaly.

[LinusVanPelt]

My general design is:

• Capture the output from explore (somehow) into a comma separated string
• Get a count of the number of symbols in the string
• Create a 2D matrix of the size of the # of symbols in the string
• Column1: mtrandom()
• Column2: index (1, 2, 3, etc)
• Sort the matrix by the random number
• Determine the split point for the IS and OOS (eg. 100 symbols, 50% = 50, 35% = 35, etc)
• Loop through the randomly sorted matrix, extract the index, get that word, and add it to either the IS or OOS watchlist.

The main problem is the best way to capture the explore output into a comma separated string.

[Tomasz]

@beppe told you what to do:

1. Use Batch to run exploration
2. Use WatchlistAddResults to add results of exploration to watchlist
3. Use CategoryGetSymbols to get comma separated list of symbols from watch list

[LinusVanPelt]

I was hoping for a less convoluted solution but thanks for confirming this as the best practice approach.

[Tomasz]

Just one function call is convoluted?

Besides you can add symbols to watch list directly using CategoryAddSymbol() and retrieve it back from other formula using just ONE function call that gets comma separated symbol list: CategoryGetSymbols()

Really there is nothing more simple than that.

[LinusVanPelt]

@beppe told you what to do:

1. Use Batch to run exploration
2. Use WatchlistAddResults to add results of exploration to watchlist
3. Use CategoryGetSymbols to get comma separated list of symbols from watch list

How do you get "one function call" from this?

As I understand Beppe's solution, I need to:

• Create an AFL that runs an explore
• Wrap that in an APX
• Create a batch to run that APX
• Add a batch step to export the explore results to a watchlist
• Create a 2nd AFL that uses that watchlist as input (perhaps it can be one AFL with code block wrapped in Status("action") = actionExplore).
• Wrap that in an APX
• Add that to the batch

Correct me if I misunderstand his approach. Otherwise I assert this is more than "one function call".

Regardless, you've stated this as the best approach.

[Tomasz]

One function call is:CategoryGetSymbols() to do what you wanted: "capture the symbol list returned by explore".

You don't "WRAP" anything into APX. You just press "SAVE" button. One mouse button click to save Analysis setup to APX file, done ONCE in your lifetime.

And no it is neither "only" or "best". It is one of possible ways. And it is NOT convoluted. The other one is using CategoryAddSymbol as I noted previously.

If somebody offered me such EASY solutions to what I am facing each and everyday I would sing appreciations songs all day long.