I thought an open single Forum thread would be ideal as opposed to some private communication.
Is there a way WM_CLOSE event trickles down PLUGINAPI int Notify( ... ); or to listen to it?
AB must be implementing WNDPROC somefn() callback function as it does save/cleanup etc in response to user clicking the Exit button.
Currently, as it Stands there are 2 plugin unloading scenarios. (3rd, shutdown is mentioned below)
Either DB_Unloaded event is fired where plugin can cleanup, in which case Release() is "not" called, OR
User clicks an Exit Appl button(Exit or X etc) and AB directly calls Release() and in this case "DB_Unloaded" notify is "not" Sent.
While Debug confirms this up to 6.1x64, higher versions by way of OutputDebugString() indicate the same but debugging itself is not possible.
This is a discussion, if things are not exactly explained or incorrect, kindly skip the sarcasm and constructively post feedback. Will do my best in response.
Here, is a whodunit challenge scenario:
While there is plugin -> RMB -> Shutdown, it explicitly tells the Data plugin to Release some resources.
Now, if there is an open COM DispatchDriver and then User Exits without plugin->shutdown, AB COM reference count will not be 0 and therefore just destroy UI and become a BG process instead of clean exit. (known standard behaviour)
It may also be noted that, when AB goes to BG process, it neither Calls DB_Unloaded event but more importantly, PLUGIN Release() is not called. So plugin is not informed that certain Resources have to be released.
So AB is doing its job by becoming BG, which is the effect but this time its because of its own process.
Would like to hear some thoughts, Thanks. Ofc Dr. Tomasz insights have been extremely useful.
So, right now the performance penalty is DispatchConnect(), do some quick work and explicitly DispatchRelease() thereby always keeping internal Reference count 0.
In stress case its quite high otherwise 1000 RTD ticks can be processed in like 70-90ms (i7 3.1 GHz laptop cpu) I have further optimised Connect-release but still looking to listen to WM_CLOSE.
Notify() is already called on database disconnection (unload/switching to other plugin). There is also Release() called later. If the plugin is MFC based (extension dll) there is also ExitInstance called. Last but not least, DllMain() with fdwReason == DLL_PROCESS_DETACH is called as last resort.
Use the functions listed above. Listening to WM_CLOSE is not error-proof because application can ignore WM_CLOSE (if for example there is unsaved document and application asks the user to save document and user chooses "Cancel"), also your plugin will be unloaded when user switches database to one that doesn't use your plugin.
That was the challenge, I am calling COM DispatchRelease() in Release(), so while I was expecting Release() to be called, it is not the case. AB straight away becomes background process.
Now I will test ExitInstance first as it is based on QT.dll and is compiled with _AFXDLL
Thanks
00000002 17:09:55 [17108] DBG: InitInstance() print
00000003 17:09:55 [17108] DBG: ExitInstance() print
00000004 17:09:55 [17108] DBG: InitInstance() print
00000005 17:09:55 [17108] DBG: Init() begin WS thread
00000006 17:09:55 [17108] DBG: Init() done
00000008 17:09:55 [17108] DBG: Notify() DB Load event
AB EXITED HERE with Click Exit
NO more Debug strings printed, AB now BG process
I spent quite some time already but my DllMain() version seems like the function is not calling the DLL entry point, so working to fix it. Trying different function definitions right now
Release(), ExitInstance() and DllMain() ARE CALLED if everything is correct in your plugin. You can try plugin without COM.
Problem that you are observing is caused by unreleased COM objects in your code. COM objects are reference counted and will prevent the the entire program (or your DLL) from exiting if reference count does not drop to zero.
The problem is that the code doesn't arrive to the point where they are called because COM object that you are using inside your code is blocking. You might have some destructor or something that just hangs on COM object destroy or you might have incorrect reference count (more AddRefs than Releases) and your code hangs on unreleased COM handle and your code never arrives to the point when the above mentioned functions were called.
As mentioned in opening post,
When user Exits without clicking shutdown, the challenge is for plugin not knowing that it should shutdown.
The reason I stated above is some kind of Notify() to tell plugin to shutdown.
That is there is performance penalty of COM DispatchConnect() do something and DispatchRelease()
Hence the idea of WM_CLOSE.
The issue is known, but the idea was to telling plugin to Release() just like when user will click Shutdown button.
Maybe if you could once again just see the OP with the scenarios. (not meant in a bad way)
Just to write in short sentence: Plugin is aware that COM object if not released will not cleanly exit. So trying to intercept APPL_EXIT incase RMB_SHUTDOWN not clicked.
My guess (I don't see your code), is that you are calling AddRef too many times inside your plugin. If you had matched AddRef/Release call pairs, you would never have any problem.
So init once in OnProcTimer() and "expect" PLUGINAPI Release() to call say ReleaseCOM()
So with method, under Load testing it is fast because we initialize the COM pointers once.
Now, plugin->Shutdown() Does call ReleaseCOM()
There is ONLY one scenario where a USER could click EXIT_AB before clicking Plugin_Shudown.
Approach 2:
The COM ptrs even though are global, do this
COM_Init()
Do quick COM related Task, say GetTickerList() or AddStock()
COM_Release()
=========
So in Approach #2, I always close it asap so there is no question of having any Active COM connection.
If user directly Exits AB, its ok.
The only thing is practically there is no problem because we dont need to lookup/Add so frequently.
Approach #1: would work "as long as" user clicked plugin->shutdown and then exited. Its a scenario.
So the thing here was if WM_CLOSE or something similar is Notified, COM will be released. Then whether WM_CLOSE happens or not, it doesnt matter.
If user decides to cancel, he can always reconnect the plugin.
Actually AddRef is matched and its only 1 open if we keep the pointers global. But in 1 open global state, there is this single possibility of termination without shutdown happening.
As I called the topic earlier, its a Design Discussion and what is optimal approach. Thanks
So I expected PLUGIN Release() to be called regardless of anything if user has clicked exit,
but it turns out that if something like a COM pointer was active, AB is destroying UI and going to Background process.
Only after practically playing around it becomes known. So your insight would give clear picture as to what is happening.
This is somewhat academic discussion and guessing game since you have presented no code at all so blind guesses is all that can be offered to you but You should look at the destructors. Some of the destructors in your code hang on unreleased COM. Release would be called if your code did not hang on destructor. (Destructor is apparently called earlier)
Documentation / Report:
Issue: Data Plugin menu (RMB) does not appear for a(some) Symbol(s)
Reason/Solution: Symbol info -> property -> Use only Local DB was Set: YES
.......................
Yesterday was extremely tough, wrote lot of if/else to handle this and later it turned out to be the above describe cause. Hope time spent here (these posts) serve as a future guide.
In months of testing, all symbols added either via UI import, Ole Import or by Plugin have always set "use only Local DB: No" by AB
Just one question for Tomasz,
Q: In all cases so far, Symbol Info -> default "use only local DB" is No, but is there a scenario it is set to Yes? Bcos all the code I reviewed does not go anywhere near that setting. Thanks.
In my case, weeks of adding/removing (10K+) symbols never encountered this.
A little more clarity on this para here: It also works if you import the data for symbols that are NOT present in the database. In this case newly imported symbols are marked by ASCII importer as "use only local database for this symbol" (See Information window for details), so they are EXCLUDED from the real-time update. How to use AmiBroker with Real-Time data plugins
Unable to edit above post: But for a developer, plugin context menu will not appear because Notify() Event is "Not" sent for symbols marked True "use only local DB" property. The menu is drawn in response to the Event.
Also found explanation here: Finding relevant posts has become hard
Yes, IF database was configured with external plugin AND symbol wasn't present already in the database AND format definition had $AUTOADD 1 so symbol was added during import.
And Yes, Notify(), GetQuotesEx() or any other plugin function is NOT sent for symbols that are marked as "Use only local database for this symbol"
When plotting chart or running Analysis, if one or more quotations have a "duplicate" or identical timestamp, what policy does AB use to select a quote. (As in first or last etc )
I am trying to refine the backfill logic. While inside the plugin, pQuotes Array is totally under plugin control.
Clever use of AB Import seems to work fine, but same time stamp will create duplicates. Older quotes greater than Array size are eventually overwritten(discarded) in plugin. Thanks.
Duplicate time stamps are not allowed. You must differentiate them inside plug-in by assigning unique time stamp. For example you can increase microsecond by one or use tickid.
Import does not create duplicates. If quote with given time stamp already exists, that bar is updated instead of inserting new one.
Yes, unique timestamp is the way, and I am trying to achieve the same
It is just that I can see multiple entries in Quote Editor
"But" in the chart/Analysis always "one" quote but with a *hybrid construction of OHLC and sum of Vol
Base time: 1min, bars supplied 1min, and periodicity also 1min.
Here 1 quote is from RTD bar and the 724.8 bar is ASCII imported. (17:20:00)
Quote Editor:
Explore/Chart: First identical bar open, (Highest high), (Lowest Low), last identical Close
and sum of V, like this
The approach to take is clear now.
Q: So even if bar timestamps are separated by sub-second, the construction of 1 minute bar will still follow the approach marked(*), i.e., as shown in Explore image?
you should NOT be ASCII importing data to plugin-driven database (unless plugin is written specifically to manage that)
if you are using plugin it is PLUGIN RESPONSIBILITY to ensure uniqueness of timestamps (including removing duplicates!)
if you are using 1-minute interval for chart / analysis , then obviously data are compressed to 1-minute, meaning all ticks belonging to given minute are accumulated (and OHLC V are calculated respectively.
the above data compression applies to ANY interval higher than tick
Generally when you drive database with the plugin, plugin has TOTAL CONTROL and can write anything (including nonsense) to data array. It is responsibility of plugin author to write plugin correctly and to ensure that timestamps are unique and data are good. If author chooses to write nonsense to data array, the Quote Editor screenshot would just show that nonsense created/allowed by the plugin.