This is just an exploratory question as I would write and contribute the code if it's considered a worthwhile project.
Would there be any overall speed-up value in having a translation utility which could take some AFL code and convert it to C++ to be used as a plugin, including all of the necessary marshaling code, or would QuickAFL tend to defeat the purpose?
You should not write plug in without experience. Also there would be no speed gains translating AFL array functions to C/C++ code.
Plugins may be considered if very slow AFL looping code is involved or if creating data plug ins.
First you should investigate in your AFL whether existing loop code can be rewritten via array processing.
Also investigate whether multiple calls of same code are in there (-> instead storing to single variable).
Let me ask a more specific question.
If we can assume that the fastest IB delivers its snapshot transaction data to IB.dll is four price transactions per second (250ms) and, given any other overhead time between actual execution of the AFL, what is approx the slowest time (in ms) the script can run and still be able to respond to every price transaction without having a real-time overrun? 100, 150, 200 ms?
I'm just about ready to find out as I'm hooking up the trading part of the code now, but I know that my average script running time is 40ms and it would be nice to know if anyone has found out the answer to this through their actual implementation. This is in not HFT. I just don't want to miss a price transaction coming through (when I only get 4 per sec) where a comparison needs to be made which could trigger a trade.
I think it is great idea. I would love to have all my custom functions in a dll rather than a text file that would have to include in various formulas that write & use. I've been thinking converting them into dll, but haven't had the enough time to focus on it.
Let me re-iterate, it is bad idea:
you should not worry about AFL because it is FASTEST part of the chain.
Your Internet connection is likely to bring 300ms round trip delay (round trip ping) and IB does not deliver every transaction but accumulated updates every 200-300ms. This accounts for 500+ ms. A properly written AFL formula runs in 5-50 ms. That is a small fraction of the remaining time.
As Donald Knuth said "premature optimization is the root of all evil", you should not write DLL to get rid of maybe 2ms. Instead focus on main problem - the data source. For tick-by-tick feed use IQFeed or eSignal.
You are right that it is not a good idea. However, people have real or perceived need for creating dll. Just recently there was a conversation on this board that when the text file containing afl becomes too big it slows its execution. I have experienced that my self. I am sure you remember from a few years back on Yahoo board conversation about creating a utility to convert afl files to dll (either locally or people upload their codes to AB site and convert them there). You even asked if people were willing to pay for this service, if so how much. Subsequent to that conversation I stop converting my afl code to dll, thinking that it was going to happen. Obviously my conclusion was incorrect as this utility never became public. At the time, one of your concerns was people creating bogus black boxes and selling them to your user base. If you ever decide to create such utility you can make it so that the dll that I create on my machine would only work with AB that is registered to me.
With multiple core machines DLLs run slower (single thread), not faster than AFL (multi thread). There is no real reason for DLLs other than hiding code (which is also illusion because if I wanted I can reverse engineer any DLL to AFL). The main interest for the tool you mentioned was hiding code and distributing black boxes. I don't want AmiBroker to be abused by people wanting to make quick buck on selling black boxes to people who are naive enough to believe that they have any value. I don't want this forum to deteriorate to "DLL exchange". There is nothing to gain and so much to lose with DLLs. The fact that AFLs are open, human-readable text files is a key to success of AmiBroker as a trading system development platform for the creative people, not a selling machine for get-rich-quick types.
I would rather invest time in getting AFL even faster than enabling black pandora box of DLLs.
I hear what you say but I personally think it would be great to have the option (for legit reasons) for one to be able to protect their code (not as a DLL but directly from AFL). Be interesting to know what others think on this?
Tomasz, AB is your baby. You do what you want with it. At the time that there was a conversation about afl to dll conversion utility on the Yahoo board, there was a conversation about using Python from within AB. You did not like the idea and didn't develop the tool. A few years later a user developed it. The same will be with this tool. Someone will develop it and you won't have control over it. As good as talented that these developers maybe, they will not have the intimate knowledge that you have of AB. My humble advice is if you develop these tools, you control them. For me having the luxury of my functions appear in the formula editor similar to builtin function is valuable, even at the cost of loosing some speed. I am EOD portfolio trader, a few second slower explores & backtests are inconsequential.
@QuantTrader - I ensure you that I CAN control what DLLs are loaded and which are not. And I could easily block loading of DLLs. So if such tool appears without my approval I can easily block DLLs produced by unapproved tool.
As for AmiPy Python plugin - it was created WITH my prior approval. Many years passed since I initially spoke about Python and in the meantime things like Tensorflow appeared and I simply changed mind about usefulness of Python plugin due to development of many machine learning modules for Python. And not only I approved its creation but also helped with it.
DLL interface was designed for the purpose of extending AFL (adding more functions). Using it for replacing AFL is NOT intended use of it.
AmiPy Python plugin extends AFL functionality. It is 100% inline with intended use of DLLs. AFL translation to DLL does the opposite thing, it only hides the code and provides no functional gain. Once DLLs get "common" this opens up huge can of worms (literally, there could be VIRUSES, TROJANS and WORMS inside DLLs).
Once you start "DLL exchange" no one would ever care about safety and people would copy/paste unknown plugins all over the place including malicious ones. That would be disaster and potential liability.
I totally agree, the AFL code in the DLL plugin. Makes AmiBroker very slow, I have experienced it. Thanks Mr. Tomasz