Running Batch Files - Takes long time to set-up

The attached shows the setup time required display for the 1st AFL file in the batch.
This may take 5 or more minutes to clear up the setup process and start running.

Wondering if this is related to disk indexing or is normal (It does not happen all the time, but frequently enough)
image

Normally it takes fraction of a second (0.1sec) to start a batch.
In AmiBroker every operation like this takes just very small fraction of second when program is used properly.

It is YOUR FORMULA that is the culprit. Check your own stuff first.

Especially formulas that use Status(“stocknum”)==0 with loops to iterate through all symbols creating zillions of composites or static variables.

Hello Ara,

Awhile back, while trading RT, we have had the exact same problem but never managed to definitively pinpoint the problem. Strange thing was that among different traders, trading the same code, the problem would occur for some but not others. We assumed it must be RT data related. We were scanning a large number of stocks in RT, starting before RTH. On some days it just didn’t work other days it worked flawless. It was quite frustrating.

We assumed the system was waiting for backfill. Try turning off backfill, or even switch your db to local as a diagnostic.

Herman

Herman, with all due respect, how could you possibly tell that you had “exact same problem”???

You don’t know:
a) the formula he is using
b) you don’t even know if he uses RT streaming source at all
c) even if he is using RT source you don’t know WHICH ONE (different sources use different plugins and different plugins have different code and different behaviour)

He provided ZERO details, yet expect the guidance.

I don’t know why people are so quick in their assumptions. It is beyond my imagination.

The fact that you have say cough does not mean that you automatically have lung cancer. Single symptom != disease.

Tomasz, yes I am using looping inside of Status(“stocknum”) ==0 condition.

Upon further testing, I noticed that the code is actually running, even though the header information implies otherwise.

I understand that the way my code is written does not allow good performance / progress measurements … however the erroneous header information is not always present … so it becomes confusing.

There are two things:

  1. Generally GUI (including header information) is updated "on idle", when CPU is not particularly busy. This is to ensure that GUI refreshes do NOT slow down processing.

  2. At least ONE COMPLETED STEP is required for progress bar to display any meaningful estimate. This is so because time estimate is TotalNumberOfSteps * (CompletedStepsTotalTime/NumberOfCompletedSteps). This does not happen in your case because your code executes very long for very first step (symbol) and this does not allow to provide you with time estimates. Your NumberOfCompletedSteps is ZERO and there is no meaningful estimate possible.

I almost never have codes that execute slower than 0.1 second per symbol so I never saw this but will add "estimate not available yet as first step hasn't completed" message so people doing long first step would see that instead.

Update: I made experiment and wrote a code that specifically WAITS long:

If( Status("stocknum") == 0 )
{
 // just wait for 30 seconds doing nothing
 for( i = 0; i < 30; i++ ) ThreadSleep( 1000 );
}

Buy = 0;
Sell= 0;

and here is how it looks on my end
image

So that raises a question: which version of AmiBroker you are using? Because I remember that I already made some changes in 6.16 to the way how this progress is displayed.

Tomasz

fyi -p per your request, I am using version 6.26. Thanks

I will be converting my code to use native lists (no loops) so I can also take advantage of multi-threading
Ara