IQF plugin: symbols stuck in Backfil state - log show terminators delivered

[danielparkinson]

Hi Tomasz,

I've been chasing a backfill issue and have captured some diagnostics that may be useful. I read the prior thread
Trouble backfilling certain IQFeed tickers in scan/exploration before posting, and your reply there framed the issue as the plugin waiting indefinitely for
data from IQFeed that never arrives. My recent testing produced evidence that may change that picture in some cases — I wanted to share it in case it's useful.

Environment

• AmiBroker 7.0.1 (7001)
• IQFeed.dll 6.1.0.0 (dated 12 Nov 2022)
• IQConnect 6.2.3.5 (current latest, 15 Apr 2026)
• Full universe backfills, ~12,000 US symbols, 1-min bars
• Running on a well-provisioned Windows 11 PC, ethernet/WiFi both tested

Symptom
Typically 3–15 symbols per full universe backfill end up permanently stuck in Back... state. The backfill dialog counters show "0 running backfills, 0 queued" but the stuck symbol's status stays
at Back... indefinitely. Only restarting AmiBroker clears it. The stuck symbols are different every run, which rules out a consistent subscription/coverage issue for specific tickers (unlike the
continuous-contract futures case in thread 30812).

New diagnostic
I enabled IQConnect's own logging (Diagnostics Utility → Logging tab → Lookup Requests + Lookup Data + Lookup Error) and ran a full universe backfill. This captured every HIX command the plugin
sent and every response IQConnect returned, at the IQConnect ↔ plugin boundary.

Results from one representative run:

• 18,997 HIX requests dispatched by the plugin
• 18,905 !ENDMSG! terminators returned by IQConnect
• 96 error responses returned (77 × E,!NO_DATA!,, + 19 × E,Could not connect to History socket.,)
• 0 requests lacking any response
• 15 symbols still stuck in AmiBroker's UI at the end

The IQConnect-level log proves that every request received either !ENDMSG! or an error from IQConnect. The previous framing — "data never arrives" — doesn't fit this run. Data did arrive. The
stuck symbols split into two patterns:

Pattern A — terminator received, flag not cleared (7 of 15 in this run)
Symbols: QUBX, PSA-J, NIOBW, NDOW, MUU, MSGS, METC.
For each, the IQConnect log shows:

1. Plugin sends HIX,,60,...,10000
2. IQConnect returns !ENDMSG!, within 100 ms to 6 s
3. Plugin sends the next HIX,<OTHER_SYMBOL>,... on the same client channel (so the receive thread clearly processed something)

…yet AmiBroker's UI keeps the stuck symbol's flag as Back.... Given the terminator was delivered and the channel advanced, this looks more consistent with a race between the receive thread and
the flag-clearing code path than with "no response arrived". Supporting this, the stuck rate increases materially when DebugView is attached to the plugin (attached listener slows the producer
slightly, seems to widen the race window) and when IQConnect's own Lookup Data logging is enabled (heavy disk I/O slows response delivery).

Pattern B — HIX never dispatched (8 of 15 in this run)
Symbols: RMEN, INSP, FTWO, FNX, BTO, BOBS, BIGY, +1 unrecorded.
Zero appearances of these tickers in the entire IQConnect log — no HIX command was ever sent. Yet they appear in the Backfill dialog with Back... status. This looks like a different failure mode
than Pattern A — the request dispatcher appears to be losing entries between the analysis engine flagging the symbol as needing backfill and the actual socket write.

Files I can share if helpful

• Full IQConnect log (~777 MB, compressible to ~60 MB)
• DebugView capture showing the plugin's own state messages (Waiting for backfill of X without matching Backfill complete. Resuming X for the stuck symbols)
• Line-by-line trace of each stuck symbol in the IQConnect log

Questions I'd appreciate your view on

1. Does this look like something you've encountered in internal builds? Particularly the Pattern B case (HIX never leaving the plugin) seems hard to explain with a waiting-for-response model.
2. Given the plugin has no public release since 6.0.3 (2016) / the 6.1.0 bundled build (2022), would you be open to sharing an experimental or test build if these reproducible cases would help
   isolate either pattern?
3. Even if the underlying races in Pattern A are hard to fully eliminate in the current architecture, would a defensive per-request timeout on the receive loop be something you'd consider?
   Turning a permanent hang into a recoverable error would let users build retry logic around it.

Happy to share any of the logs on request, or run additional captures with any configuration that would help. Not trying to stir up an old discussion — just wanted to provide cleaner evidence
than was available in 2022 when thread 30812 was posted.

Thanks,
Dan

[Tomasz]

Thank you for your report. Will look into it as soon as I have some time. April is unfortunately hectic month.

[Tomasz]

TWO very important questions:

1. WHY did you exceed your subscription limits by factor of 24x ?
   
   I guess your subscription is 500 symbols, and yet you asked for 24x more data. Even if your subscription is largest available at IQFeed (1300 symbols) what you are doing is outside of limits. Exceeding subscription limits is strictly forbidden
   
   
   I explained many many times that you must never exceed the subscription limit.
   
   Most recently here: Incorrect config, was: Backfill not fully completing - #6 by Tomasz

2. Did you TURN ON "WAIT FOR BACKFILL" option?
   
   
   Without "WAIT FOR BACKFILL" and run scan, because of amazing AmiBroker speed exceeding ANY data source capabilities, you are going to flood IQFeed with the amount of requests it is NOT able to handle, especially during market hours. To prevent completely knocking IQFeed servers and IQClient, the plugin allows maximum of FOUR (4) concurrent backfills in progress. If you bomb data source with more backfill requests that it could handle (beyond 4 simultaneous), the requests that EXCEED the limit will be just NOT sent. Therefore it is absolute MUST to use "Wait for backfill" in experiments like yours as it guarantees that IQFeed is NOT bombarded with flood it can't handle.
   You need to understand that AmiBroker is able to access THOUSANDS of securities per second, but data source that has 100ms minimum response time is able to deliver only 10 maximum, without exploding number of parallel requests. Such abuse (if allowed without safety guard) would result in immediate block of your data account.

[danielparkinson]

Hi Tomasz,

Thank you for the detailed response about the 4-concurrent safety mechanism and subscription limits. Before I share more evidence, I want to gently raise one point where I think there may be a misunderstanding
about what my backfill was actually doing, and I'd genuinely appreciate your clarification if I have this wrong.

My understanding of how your plugin works — which the screenshots I captured appear to confirm — is that the plugin's own state machine IS the subscription-limit safety mechanism:

• When a symbol is picked up for backfill, it briefly takes a streaming subscription
• Once backfill completes, the subscription drops and the symbol's status moves to Ready, then eventually to Old
• Concurrent streaming at any moment is kept small by design (a rotating set, typically 4–5 active plus a short tail of recently completed)
• Iterating a large universe does NOT mean holding thousands of streaming subscriptions at once — the plugin rotates through them

If this is correct, then for my run I was not actually exceeding the subscription limit. The total universe iterated was ~12,000 symbols, but the concurrent streaming load at any moment was always well under 500.
This is visible in every single screenshot I captured:

• Backfill A: 0 running backfills, 0 queued backfills, RT watches free: 454 (46 in use, 9% of the cap)
• Backfill B: 0 running backfills, 0 queued backfills, RT watches free: 499 (only 1 in use)
• Backfill C: 4 running, 16 queued, RT watches free: 281 (219 in use, still under cap)
• Backfill D: 0 running backfills, 0 queued backfills, RT watches free: 384 (116 in use)

At peak, the most I ever observed was 307 concurrent streaming subscriptions — never close to 500, never exceeding it. The plugin's safety mechanism was working throughout. I'm grateful for it — that's clearly
what you built it to do.

To confirm: Wait-for-Backfill was enabled throughout all runs. I can see the plugin queueing excess requests correctly rather than dropping them (the "4 running, 16 queued" state in screenshot C confirms this),
which is consistent with how you described correct behaviour.

Is my understanding of how the safety mechanism works correct? If so, then I think the evidence I gathered is showing issues that occur WITHIN the protection you built, not because it was being circumvented.

With that clarified (or corrected by you, if I have it wrong), the residual case I'm trying to understand is shown in three screenshots (attached). In each one:

• Backfill dialog reports 0 running backfills, 0 queued backfills — plugin is idle, no request in flight, no queue waiting
• RT watches free is near 500 (454, 499, 384) — subscription is barely touched
• A single symbol remains permanently in Back... state until AmiBroker is restarted
• The run completed cleanly for every other symbol

This doesn't seem to match any of the documented failure modes:

• Not a subscription overage (RT count shows 1–46 in use)
• Not a flood-protection drop (0 running, 0 queued; plugin has nothing to refuse)
• Not "waiting for data that never arrived" (there's no request in flight for the plugin to be waiting on)

It looks more like a case where the response WAS delivered and processed (the plugin moved on, drained its queues, reported idle), but the per-symbol status flag didn't get cleared for one specific symbol. My
IQConnect-side logs from one of these runs confirm every HIX received either !ENDMSG! or a proper error response — nothing was left hanging at the IQConnect ↔ plugin boundary.

I'm not trying to relitigate the 2022 thread or question the design of the safety mechanism — both make total sense and I'm grateful for the work you've put into them. I just want to understand whether the
symptom in those three screenshots matches anything you've seen in internal builds, and whether there's a code path where the status flag clear can fail despite the queue draining cleanly.

Happy to share the IQConnect logs or run any additional captures you might find useful. No urgency whatsoever — I know April is hectic.

Thanks again for your time.

Dan

[danielparkinson]

[danielparkinson]

thanks Thomaz.

Dan

[danielparkinson]

Just a follow up which may help;

Since my last post I ran another full-universe backfill and noticed something specific that I think narrows down where the problem lies.

All of the symbols that got stuck in "Back..." state on this run occurred while the plugin was sustained at 4 concurrent backfills. I watched the "running backfills" counter on the
status bar throughout the run. For the first ~25% of the universe it sat at 4, and during that window around 20 symbols ended up permanently stuck. Once the counter dropped and began
fluctuating between 0, 1 and 2 for the remainder of the run, zero further symbols became stuck.

The correlation was very clean — stuck only happens while slots are saturated at 4/4. As soon as responses start arriving with time gaps between them, everything resolves normally.

Combined with my earlier finding that IQConnect's own log shows !ENDMSG! being sent for every request (including the stuck ones), this looks to me like a race condition in the
response-handling path when two !ENDMSG! terminators arrive in close succession and contend for the same slot/flag state — one completion clears its slot cleanly, the other's flag
update gets lost, and that symbol sits at "Back..." forever even though the data arrived.

This also matches my day-to-day experience on live runs. Most days my watchlists are small enough that 4/4 is never sustained, and I see no stuck symbols. But on busy days when my
scanner fires a lot of signals and a live watchlist grows to ~300 symbols, that's when I see the stuck-backfill problem — exactly the condition that would hold slots at 4/4 long
enough for the race to fire repeatedly.

I'm sharing all of this because I'm genuinely trying to get to the bottom of what's happening and understand the behaviour — I'm not assuming I've got the internals right, but the
pattern is very repeatable on my end so I wanted to put it in front of you in case it helps.

Happy to capture another timestamped OutputDebugString log if it would help.

Thanks again for looking into this.

[nsm51]

@danielparkinson

This also matches my day-to-day experience on live runs. Most days my watchlists are small enough that 4/4 is never sustained, and I see no stuck symbols. But on busy days when my
scanner fires a lot of signals and a live watchlist grows to ~300 symbols, that's when I see the stuck-backfill problem — exactly the condition that would hold slots at 4/4 long
enough for the race to fire repeatedly.

How the server throttles will never be clear, but all your symptoms are pointing to "limit" abuse that their server seems to detect and react to.

They are generous to paid subscribers, but aggressive abuse like in the case of yahoo will block the IP address.

IQFeed 500 Limit Question

It was explained many times by IQFeed director that you are not allowed to exceed symbol limits. Once you access given symbol data it becomes subscribed to real time updates and also historical (missing) data are requested. Only that way you can have both history AND up-to-date current quote. If you exceed, AmiBroker would unsubscribe oldest symbol(s) and subscribe newest (FIFO) automatically and it works to some extent but if you abuse the service significantly do not expect it to work. 2000…

There are numerous posts on it on the forum.

Iqfeed limit exploitation - AmiBroker - AmiBroker Community Forum

IQFeed 500 Limit Question - Data Sources - AmiBroker Community Forum

Abuse of subscription, was: IQFeed data problems - Data Sources / IQFeed - AmiBroker Community Forum

IQFeed plugin - Streaming data different from backfill - Data Sources / IQFeed - AmiBroker Community Forum

[danielparkinson]

the 4-concurrent backfill cap (plugin-side) and the 500-concurrent streaming cap (server-side) are independent limits. My runs sustained 4/4 for a period (plugin-side, by design) while peaking at 307/500 streaming (server-side, well under cap). The stuck 'Back...' status appears in the plugin-side code path, not the server-side subscription state."

[danielparkinson]

One more data point that may help clarify, the screenshot above shows a single stuck symbol (ZSL in "Back..." state) at a moment when the plugin reports:

• 0 running backfills, 0 queued backfills (plugin has nothing in flight or waiting)
• RT watches free: 454 — only 46 concurrent streaming subscriptions in use, 9% of the 500 cap
• Every other symbol in the batch (SKYQ, SM, SMST, TALO, UCO, USO, VG, VIST, VLO) resolved cleanly to "Ready"
• The dialog even shows "History arrived BMNZ" — the prior symbol's data returned fine

So in this specific state, the plugin is idle, the subscription count is nowhere near the cap, and the server has been responsive for every other symbol in the batch, yet ZSL sits permanently in "Back..." until AmiBroker is restarted.

I really am just trying to get to the bottom of this, not make a point. If it turns out to be user error I'll happily accept that, but evidence like the screenshot above is why I keep coming back to the question. If the underlying cause is understood and fixable, it'd make IQFeed behaviour more reliable for everyone using it at scale.

Thanks again.

[nsm51]

@danielparkinson

Since my last post I ran another full-universe backfill and noticed something specific that I think narrows down where the problem lies.

Do you still understand why this is an issue?
12000 symbols right?

on a fresh day, try and backfill upto you subscription limit "only"
Maybe Iqfeed should start sending specific rate limit or such behaviour so this recurring issue on the forum is settled.