Analysis - How to identify the end/finish of analysis

I need to execute a command just at the very end of analysis scan/explore only ONCE. How to identify the condition?

For start of run I can use this: if ( Status(“stocknum”) == 0 )

Anything I can use to identify the finish/end?

Thanks in advance.

1 Like

Here’s one way to do it. Count the number of symbols in the watchlist using StrCount. Then the last symbol will be when Status("stocknum") == SymbolCount

wlnumber  = GetOption( "FilterIncludeWatchlist" );
watchlist = CategoryGetSymbols( categoryWatchlist, wlnumber );
SymbolCount = StrCount(watchlist, ",");
LastSymbol = Status("stocknum") == SymbolCount;

Filter = Status("LastBarInRange");
AddColumn(LastSymbol, "LastSymbol", 1);
1 Like

It’s not that easy. It might not work properly in most cases. A quote from:
https://www.amibroker.com/guide/h_newanalysis.html

New Analysis window uses all available CPUs/cores to execute formulas in many threads in parallel providing significant speed ups. For example on 4 core Intel i7 that can run upto 8 threads, it can run upto 8 times faster than old Analysis window.

If you run an Analysis window (i.e. Exploration) it will use more that 1 thread by default and in multi-threaded environment you have no idea which thread finishes last. In general all symbols are processed in alphabetical order, but the last symbol in a watchlist might not be processed as the last. The above method will work properly only in one scenario : when you limit the number of threads that are used during entire Exploration (or any other multi-threaded Analysis operation) to 1. If you want strictly sequential execution (single-threaded) you can achieve that by placing the following pragma call at the top of the formula:

#pragma maxthreads 1

Such modification will allow to properly detect the last symbol, but in general should be used with caution, because it makes the formula run slower.

https://www.amibroker.com/guide/h_multithreading.html

It is very easy. But not the way you are trying to do. As @Milosz said, in MULTITHREADING execution there is no such thing as “last” symbol as there can be N “last symbols” executing in parallel.

And also it is NOT proper solution to use #pragma to disable threads (wayyy too sloooow).

First you need to think if you absolutely need to do that kind of “last processing”. You usually don’t. There is a difference between “absolutely need” and “want”.

If you absolutely need to do anything AFTER scan/explore the only way to do this PROPERLY is to use BATCH.

http://www.amibroker.com/guide/h_batch.html

Batch allows to SEQUENCE various Analysis runs in SAFE way and be 100% sure that one happens only after previous one has completed.

3 Likes

Can I ask why? I wrote, that it should be used with caution. But when I force strictly sequential execution (single-threaded), the last symbol on the watchlist should be really the last symbol processed. Am I wrong?

I use many explorations during the session which notify me (using Speech synthesis) if selected stocks meets some criteria. But because I screen lots of stocks, I don’t use Say() for every issue which meets some criteria (when the Exploration is being run), but as the Exploration progresses I collect the names of all those issues (plus some additional information) using StaticVarSetText() and use Say() only once when - the last symbol is being processed. I haven’t noticed any problems.

I am aware, that the above method allows to detect the moment when the last symbol is being processed, not the moment the Analysis has really finished - but it should not be a problem in my case.

Thank you.

I answered already - because it is slow and unreliable. There is only way “proper” way to do anything - THE BEST way (fastest, most efficient and safest).

Relying on “last” symbol from single watch list is unreliable in many ways. The “last” symbol in the watch list may be EMPTY (no quotes) and then your code will break. Also the code will break if filter is more complex than just one watch list (Filter dialog allows to select many watch lists, groups, markets, exclude some stuff, etc).

Batch solution will work ALWAYS.

Of course you are free to use whatever works for you but whenever I can I tend to give general solutions that work in every case (including corner cases), not conditionally.

1 Like

Tomasz, thank you for the information. I admit I haven’t used batches for that because my solution works for me. Using batches and scheduler in my case seems much more complex than just one exploration with Auto-repeat set to i.e. 1 Minute.

When you introduced the Scheduler, I suggested adding more “Repeat” intervals to the scheduler - for example 5m, 1m, but you said that batches and scheduler are not intended to be used for tasks that are repeated very frequently - for example every 1 minute or every 30 seconds. Analysis’ Auto-repeat option is for that. I would have to add lots of Batch tasks to the scheduler to be able to run a project every 1 minute because now tasks can be repeated only hourly or daily.

If I understand in this case I would need to load 2 projects to Batch. First one with the exploration (as apx file) which collects (and saves to one Static text variable) the names of all the issues meeting some criteria (plus some additional information) and the second project which would make AmiBroker say the content of this Static text variable after the first projest is finished. Batch “Say text” won’t read the content of the static variable… Am I right, two projects would be needed, or maybe it can be solved in another way?

Thank you

As I wrote whatever works for you is good enough. Everyone has different goals/needs. And yes with batch solution you need two APX files. And yes using batch just for say text is like using gun to kill a fly.

Dear Thomas, Helix and Milosz, thanks so much for the comprehensive answers. They are very useful for my cause.

2 Likes

I am aware, that the above method allows to detect the moment when the last symbol is being processed, not the moment the Analysis has really finished - but it should not be a problem in my case.

Hi Milosz, perhaps you can check out: https://www.amibroker.com/guide/afl/staticvarcompareexchange.html

Using semaphore, you can increment a static var at the end of the processing in a thread-safe manner, then have each thread check if it’s indeed the last thread to complete processing.

I too have a debug file that’s written after all processing is done but I often encounter cases when the last ticker ID is NOT the last to complete - it is merely the last to start processing. The problem gets worse when the number of threads increase. That was why I came up with this solution.

Hope this helps you.

1 Like

@phaser2679 thanks a lot for the info :+1: Sounds interesting - I will give it a try :slight_smile:

I’ve read about this function a couple of times, but I’ve never used it. Frankly - I’m not sure if I understand correctly how does StaticVarCompareExchange() work. To avoid any problems with Static Variables in multithreading environment I always stick to the rule laid out by Tomasz here: https://www.amibroker.com/guide/h_multithreading.html I quote:

As long as only one thread writes and many threads just read static variables, you are safe and you don’t need to worry about synchronization.

… for this reason it wasn’t necessary for me to make use of a semaphore or critical section (and StaticVarCompareExchange ). But this (alternative) application seems interesting. Could you please provide a short example (a few lines of code) illustrating your idea and application of it in this case? If not I will try to work it out myself…

Regards

@Milosz, happy to share and get feedback for my code :smile:

Basically, in multi-thread processing, one of the biggest issue is concurrent update of a shared resource. This is why Tomasz said what you quoted in your last msg. Using critical section overcomes this - essentially it ensures that only 1 thread can update (‘write’) at any one time.

So the idea is very simple: have a Static Var to record how many threads have finished, all the while using critical section to ensure no concurrent updates.

Code as below:

(Hmm… not sure why my indentations all went missing the moment I put my code as ‘block code’)

Define function:

function bf_AddOneCS()
{
// Fn adds 1 to inp StaVar in semaphore. Retn updated value of StaVar.

res = Null;

// Try obtaining semaphore for 10 sec
for( i = 0; i < 10000; i++ )
{
if( StaticVarCompareExchange( “semaphore”, 1, 0 ) == 0 )
{
tmp = StaticVarGet( “nofTickrDone” );
if( IsNull( tmp ) == True ) tmp = 0;
res = tmp + 1;
StaticVarSet( “nofTickrDone”, res );
StaticVarSet( “semaphore”, 0 ); // reset semaphore
break;
}
else
{
ThreadSleep( 1 );
}
}
return res;
}

Chunk 1 (to be placed at START of code):

if( Status( “action” ) == actionExplore AND Status( “stocknum” ) == 0 )
{
StaticVarSet( “nofTickrDone”, 0 ); // Reset counter
}

Chunk 2 (to be placed at END of code):

// Record completion
nofTickrDone = bf_AddOneCS();

// Do if last ticker to finish
if( nofTickrDone == u_NofTickers )
{
// Do whatever you need here. No worries about conflicts since this is the last thread to finish
}

Anyhow, a few more things to mention:
(1) In my code, you can see this var ‘u_NofTickers’ which gives the number of tickers in my watchlist. I hard-coded it for now since I rarely change it, but you can get this dynamically using code.
(2) I allowed the for-loop to run for a maximum of 10sec. This is to be absolutely sure that all and any conflicts have time to be resolved. Besides, there is no real penalty for a high cap since the moment it is successful, it breaks the loop.
(3) To credit Tomasz, the optimal solution is still the one he provided - this just suffices for my use (and hopefully yours, Milosz).

Happy to hear your feedback and whether it works for you. Cheers!

1 Like

@phaser2679 Once again, thank you for the codes and your detailed explanation.

Unfortunately I’m quite busy this weekend and didn’t have time to try out your solution, but as soon as I do, I will let you know :slight_smile:

Cheers!

Really, to add or subtract a number in thread-safe way you do not need critical section. You could just use StaticVarAdd()

StaticVarAdd("nofTickersDone", 1 ); // this is atomic operation (thread-safe)
2 Likes

@Kuba, by golly, you’re right! Thanks for sharing!

1 Like

After trying for a while, I finally realised StaticVarAdd() is available only from Amibroker v6.2. I’m using a previous version and thus couldn’t find the function.

Just wanted to point out to future readers of this thread.

Here is another way if you are feeling a bit adventurous

  1. Open a single Analysis window
  2. Load your formula
  3. Run analysis using ParamTrigger() from afl
ab = CreateObject( "Broker.Application" );
docs = ab.analysisdocs;
qty = docs.count ;

execute = ParamTrigger( "Run Analysis" , "" ) ;
Title = ""; 
if( qty == 1 )
{

    doc = docs.item( 0 );

    if ( execute ) 
    {
        doc.run( 1 );
    }

    if( doc.isbusy )
    {
      Title = "Analysis is Busy";  
    }
    else
    Title =  "Analysis is Complete"  ;

}


RequestTimedRefresh( 1 );


Warning:

While it is possible to access Broker.Application and underlying objects from AFL formulas you should be very careful NOT to touch any user interface objects (Documents, Document, Windows, Window , Analysis object) from AFL formula because doing so, you will be likely “Sawing Off the Branch You’re Sitting On”. Especially things like switching chart tabs from currently running chart formula are totally forbidden. Changing user interface objects via OLE from AFL that is currently running within those user interface parts is recipe for disaster. You have been warned.

2 Likes

2 posts were split to a new topic: Number of symbols belonging to category

Hi Forum,

I want to export all of Buy Signals of every Symbols into a csv AUTOMATICALLY from an AFL during clicking on Explore.

And before the process i want to call a callPreProcessOnlyOneTimesAfterClickingOnExploreButton() and callPostProcessOnlyOneTimesAtTheEndOfExplore() function exactly ONE TIMES.

How could i achieve my target?

so:

callPreProcessOnlyOneTimesAfterClickingOnExploreButton(); //call only once
callMainProcessForEverySymbols(); //call lots of times
callPostProcessOnlyOneTimesAtTheEndOfExplore(); //call only once

Do you have please any idea???
( Perhaps can the BacktesterObject get back somehow? )

Thanks in advance, Zoltan.