You really should re-read my posts and the ADK docs. The data are auto-appended. BY THE DATA PLUGIN. See also http://www.amibroker.com/guide/h_workspace.html
Data plugin has TOTAL control over data and it communicates with external data vendor APIs, subscribes to RT stream and builds (auto-appends) data to array that is passed AS POINTER in GetQuotesEx call. Everything (with examples) is in the ADK documentation.
I originally was speaking of the already built in support for feeds like esignal, IQfeed, Dtn etc. From my reading above they do not auto append data files unless things like in quote window, currently charted scan or exploration etc. When you refer to data-plug in are you referring to a data plug-in we code ourselves and implemented via the SDK? If so then all quotes are stored, no matter if the symbol is in quote window, currently chartedâŚetc.?
Not everything that you read from somebody else than me is actually precise. Except me, everyone else is just a user. They may or might not be correct or may be partially correct, because no one except me actually coded the program. The problem is that people take my description from somewhere take 5 bits out of 10 that I wrote, fill the remaining with their âown interpretationâ and then post somewhere. And information noise of half-truths is created this way.
What you âread from the aboveâ is not (entirely) correct. In short with eSignal or IQFeed ALL symbols WITHIN subscription limit, will get updated and appended, if only you access them once. This is more or less the STANDARD way of working with ANY real-time data source as you need to SUBSCRIBE to updates. And symbol once subscribed remains subscribed and data are collected as long as you donât EXCEED your subscription limit.
Things are A LOT more complicated that very simplified picture presented above. If I were to describe ALL inner workings of AMIBroker I would need to write a book that has 100000 pages. I donât have time for that and I have no desire to do that because all the competition with everything handled on silver plate would steal ideas from us (they already do that).
lol ok understood regarding debth of material and disclosing.
Just so I understand then, even if you ran an empty exploration Filter=True, which creates the subscription (an automatically subscribed watchlist might be nice to have, then we can explore logic to that list etc); Anyway, then all those symbols (post subscription) will be on autopilot, meaning initial backfill and real-time updates (10/sec each) appended price array and local storage? Then completely referenced thru ADK?
Again, this is your interpretation, not facts. The readerâs interpretation is as always skewed a bit. That is similar to a kids game called âChinese whispersâ or âChinese telephoneâ. The person on one end says something but on the receiving end we get something completely different. Yes an exploration would subscribe to RT updates, but for example you donât need to even set Filter = True. It can be set to false. Also real-time updates are appended not 10/sec but immediately (if there is 5000 ticks per second then it gets updated 5000 ticks per second). 10/sec is NOT data update rate, but the maximum frequency of single CHART redraws when triggered by plugin. But if you use other refreshes (such as mousemove) it can be 1000 per second. As I wrote, there are GAZILLIONS of details. And RT data are kept in RAM, not in âlocal storageâ (i.e disk).
Tomasz, thanks to your replies, I know, that the part of my reply in which I wrote that after running a Scan/Exploration on some Issues, those issues are not fed in real-time after the Scan, is not true. Those symbols become subscribed and are updated - as long as they stay within RT subscription limit. Sorry for that. I donât want to misinform anybody, so you can edit or delete the part of my reply containing wrong information. I hope, that some other things that I have mentioned are useful to some usersâŚ
⌠but let me briefly explain why I have written it. Iâve been scanning the market (using Statica plugin) for many years. I always start with a pre-scan to allow AmiBroker subscribing issues and filling data holes. Only when this phase is completed (it takes some time) I am ready to run live scans during the session. The session starts at 9 a.m. and the market closes at 5 p.m. What I have observed right from the beginning, is that as long as those scans/explorations are run, all symbolsâ quotes are updated and when I save the database, the quotes are complete. But if my last scan is run for example at 4 p.m. (and I save database at 5 p.m. when the market closes) when I later review the charts/quotes after the session (offline) most symbolsâ history ends at 4 p.m. (their quotes have not been updated after 4 p.m.) What might be the reason for that? Is it because:
or
I have asked âHow can RT subscription limit be recognized if the data provider doesnât specify it?â because I canât find any information concerning Staticaâs limit.
One more question. I have not used eSignal or IQFeed, so forgive me my ignorance, but what happens if a user who uses a plugin with RT subscription limit letâs say 1000 issues tries to scan 2000 symbols? He should be able to scan the first 1000 issues (because they are within the RT subscription limit) but when he exceeds this limit, (according to the FIFO rule) those 1000 issues should be gradually removed from the RT subscription list and replaced by the newest issues⌠So the first 1000 symbols should be replaced by the newest 1000 symbols. If that is the case, is the user able to get the latest quotes of all those 2000 issues when running one scan? Important remark: I am aware of the fact, that the latest quotes for those issues which are outside the RT subscription limit may not be available at once (but they should be available in subsequent scans even if the Wait for backfill option is not available). Of course I assume that the database is updated regularly (for example every day) and misses only the newest quotes. A quote from Tomaszâs reply from another thread (I have already provided the link to it):
Wait for backfill flag instructs Analysis window to wait until IQFeed sends all requested data. Without that Analysis would progress with your existing data (stored already in the database), without waiting for update. The update will eventually arrive later and will be included in subsequent scans.
Thank you
Statica is NOT our plugin. It is 3rd party development that is done without our supervision. We are not responsible for 3rd party developments. Everything what I wrote applies only to official AmiBroker.com plugins that are developed by us.
Thatâs not how it typically works. Itâs a hard fixed limit. However, itâs theoretically possible that a plugin could unsubscribe â if the data feed supports that â to free up the slots for new tickers during a scan. But, that would be gaming the limits and its unlikely to be tolerated or officially supported.
All our plugins indeed unsubscribe âoldestâ symbols once limit is hit, but it is not really reliable and as @TraderX noted it is not officially supported by data vendors. It is highly recommended to stay within limits.
Hello Term,
I found this thread that you started very interesting and your last post - the summary of issues for future reference - most useful. The same applies to the comments by Tomasz, Milosz and Sean as well. I consider developing a RT system myself and these issues have popped up on my list too. The exchange here helped me to organize my thoughts.
I am wondering whether you continued with your project. If you did, could you give an update now (a year later) on what happened and what were the most important findings for you? What did you do (which data feed, how many symbols etc.) and what challenges, external sources (codes, developers etc.) have you come across during the process?
Cheers,
My solution was to separate scanning and trade signal generation in separate modules. For the scanner / watchlist creation I use IBKR API or Trade Ideas. Symbols which appear on scanner are sent to empty Amibroker database. So there are only symbols from the scanner. This way I limit number of symbols for real time data feed.