Exploration takes a long time once it reaches 100%

AFL Programming
1

Hello,

I'm working on proprietary code (for someone else) on version 6.38. Unfortunately I cannot share the code as a result.

But after a few updates today of the function library I created to run explorations, the exploration runs fine up until 100% complete, but instead of finishing, gets stuck at 100% for what I would see is uncharacteristically long.

Can anyone explain what happens at the end of an exploration as a possible identification of what I've done to cause this?

When I use the function library to simply post charts (uses all the same code I used as the exploration), it seems quick and very usable.

So you can surmise this issue only happens when I use this code for explorations.

The exploration is using Norgate Date (NYSE and NASDAQ EOD).

I couldn't find any other post talking about this kind of issue.

Of course I will do a diff between pushes of my code to find what it is, but I'm more looking for an explanation of what is going on when we click 'Explore'.

Thanks,
Mike

Thanks!
Mike

2

Here's a snapshot of the Info of the last run
image

3

Also note that the it takes on 15 seconds for it to reach 100%.

Thanks!

4

PS: I now know which part of the code caused it, but not the root cause. So any information on the above is still very helpful.

5

Too few data provided to give you any answer. No details about how many symbols, how many bars per symbol, what this formula does, how much output it produces, analysis date range. Does it generate lots of composites, static vars, anything other unusual. Nothing. Really we don't have magic crystal balls here. Please follow this advice: How to ask a good question

6

@Tomasz, I know, I was just wondering where I can get information on how for example:

  1. The percent complete function is calculated in the exploration
  2. Is there any cleanup function that occurs at the end of the exploration when it reaches 100%

Basically the mechanics of the exploration around the formula being run.

Thanks!
Mike

7

This would help provide clues on what might be going on to help me find the root cause.

Note I did mention all NYSE and NASDAQ symbols, so it's in the thousands.

Usually this exploration takes < 30 s.

8

However to be more precise, if I see how many symbols actually have the function in question executed (after trend filtering) it's 190 symbols

9

Number of symbols that execute formula (APPLY TO) counts, not the filtering that is inside formula (because the formula is executed for ALL symbols in "apply to" selection, not just ones that produce actual output)

10

Right but I found the user function that is causing the issue, and it only executes on stocks that meet my trend criteria. So the function is only executing 190 times for that stock, otherwise it is skipped via IF condition. statement.

I'm just wondering if the 100% complete means in the exploration when it reaches that point but the exploration has not yet stopped. What is happening then?

11

I can't tell you what is happening on your computer, if you refuse to give any details.

12

I was just wondering about the mechanics of how an exploration is run in Amibroker, how the % complete counter is calculated so I can know what 100% really means. Because with any exploration, not just this problematic one, there is always some kind of work being done once it reaches 100%. I even hear the fans on my laptop kick in sometimes. Again this is with any exploration.

So sorry to confuse things by talking about my particular issue, but I'm really just wondering what happens in each step of a generic exploration.

Thanks!
Mike

13

I have a similar situation. I think my issue is the EXCLUDE. In my runs where the Excluded items are in the processing list, I get the 100% non stop, but when I don't have the Excluded items in the processing list it completes.

I have not done enough testing to verify or confirm the issue, as I am still developing the processing list and exploration.

Hope this helps.

1 Like
14

you mean here?
image

This is mine

15

exclude:
image

16

Yes. But if you don't use it, it obviously is not your issue.

1 Like
17

That's why I'm wondering, irrespective of actual formula and filtering and date ranges, what goes on in when a person clicks Explore in Amibroker until it stops on its own.

18

Zillions of things. That the same amount of useful information that you have provided.

Repeat:

When you talk to your doctor, do you say: "Hey please give me diagnosis what happens with me, unfortunately I can't tell you anything about my symptoms or anything at all, but anyway please give your best estimate"?

"Oh and I forgot, I won't visit your office, so you won't see me, so give me the answer over the forum. Thanks in advance".

Joking aside: you might have corrupted database/data file. There is theoretical possibility that it gets stuck trying to read the data file that is at the "end" of symbol list and there is corruption so OS re-tries and re-tries. But then pure guessing and really chances of hitting the nail without seeing it are close to zero.

2 Likes
19

Tomasz,

Thanks, I'm sorry I can't share the code, I know this would be easier.

Corruption could be the issue since the hold on happens at the end. I will rebuild it and try again.

My code use while loops searching for optimal values in a manner that array math would just be too convoluted to get a solution easily (or quickly). Maybe one day I can nail that.

So maybe for certain symbols, despite all my protections for endless looping, there is still this possibility :thinking:. Otherwise the code is doing nothing fancy. Just checking the highs and lows of consolidations and searching for contractions in volatility that give a low risk entry point.

So functions parameters, conditions used are:
LastValue
HHV/LLV
Ref
BarsSince
if/IIf
Barcount

Regards,
Mike

20

@rocketPower why don't you divide your code into blocks and use a lot of _TRACE() /_TRACEF() functions and/or the debugger to pinpoint problematic parts of your code? Besides if you add Now() to _TRACE you will be able to additionally see/confirm which parts of your code suspend or delay finishing of the exploration or which symbols pose problems.

4 Likes