AB Pro can run 32 threads, any one have experience on run AB Pro on AMD 2950X or 2990WX? Thanks for sharing.
I have an i9 7980XE (18 cores, 36 threads), but AmiBroker isn't why I have it. The extra 17 cores are useless in AmiBroker due to my use of Norgate plug-in functions in almost everything I do. (I believe single-threaded processing is a limitation of all plug-in functions, not anything peculiar to Norgate plug-ins.)
You are mistaken. The fact that you are using Norgate does not mean that it does not run in 32 threads. It runs that. The only thing is that actual Norgate "AFL" plugin function calls are serialized (so they are called from multiple threads but Norgate plugin handles one call at given time). This is done because 3rd party plugins are not written in re-entrancy safe way. Re-entrancy is different concept from multi-threading.
But formulas run in multiple threads REGARDLESS if you use plugins or not.
As to AMD CPUs, they work just fine. I got one Ryzen testing machine and it "just works". There is no reason to treat them any different than Intel.
Does AmiBroker somehow test for reentrancy of plug-in functions and serialize calls to those that fail the tests, or is serialization of all plug-in function calls enforced across the board?
You can't test for that. Re-entrancy bugs exhibit in all kinds of bizzarre rare/non-reproducible ways (like racing conditions etc) plus some AFL plugins were written before multithreading was available. So serialization is enforced for AFL plugins. It is not done for GetExtraDataEx function that can be exposed by DATA plugins and available via AFL GetExtraData. Yet, there is not a single plugin that uses GetExtraDataEx successfully because re-entrancy safe and multithreading-safe code is tricky and requires truly experienced programmers
Besides the impact of serialization is really overestimated. If your code does anything else than just calling Norgate functions, one thread at a time will be serving Norgate function call and all the others will do all the rest of work in parallel without interruption.
2 Likes
Have you considered some sort of plug-in developer accreditation system? Perhaps something as simple as an online exam on multi-threaded programming would work. I have quite a bit of confidence in the programming competence of the Norgate team.
No, I am really not into "online exams" 
Norgate is free to use GetExtraDataEx (in fact this API was provided on their request) which does not do serialization.
It is not (only) about competence. People can be competent yet still struggle with hidden / unknown issues with underlying libraries. For example in Windows XP there was bizarre multithreading/serialization bug in Metafile playback to 2 different DCs using GDI calls. Almost no-one on Earth known about this bug because back then (10+ years ago) apparently very few people ever attempted multithreaded playback of metafiles, only me and the other guy who reported it to MS and was pretty much ignored by them ('won't fix' resolution for Win XP and Vista, but years after MS fixed it in Windows7). Such little unknown things decide that even competent programmers may fail to deliver working solution because of some underlying OS/library/subsystem problem.
Data plugins by nature work with shared resources (like disk). No matter how many threads you use at the end of the day you still need to access shared resource and this shared resource is usually limiting factor, not CPU speed which is orders of magnitude faster even in single thread.
2 Likes
Tomasz,
I want to upgrade from 5820K (6 core) to a faster CPU. I do want to upgrade to 2990WX. My holdup is 2990WX is ABOUT 100% more expensive than 2950X, but getting only 25% more performance than 2950X. Did you perform benchmark comparison of AB on 2950X and 2990WX? If not, do you have link to benchmark comparison of the two CPU using AB type (number crunching) application? Thanks.
Some benchmarking is here:
http://www.amibroker.com/kb/2017/10/06/limits-of-multithreading/
It is not for AMD, but discusses the fact that you might NOT be limited by CPU. Depending on what are you doing you may be limited by for example data access speed, not CPU. So please read the article and check the timings you get on your current hardware and pay attention to what component is taking most. Improving workflow may give you better results than faster CPU.
1 Like
In my opinion upgrading now (especially to such expensive CPU), would be a bad move - because in a few months AMD is going to release their 3rd generation of ZEN processors. There's a high chance that comparable AMD or Intel processors will be much, much cheaper in a few months time, because rumored specifications and prices of the upcoming ZEN processors are impressive. So let's hope they meet the expectations
See here:
3 Likes
@learnbusell, If I ask you I want to go from LA to SF by car and I don't know if I should buy a 911 or a cargo van, wouldn't you ask if I am hauling something or am I driving to SF with my wife with a couple of pieces of luggage? What I am saying you need to correctly define your problem before attempting to find a solution.
- What instruments are trading
- Are these instruments traded, backtested in intra-day or EOD
- What is you trading style (portfolio trader or a single instrument
- How many bars of data are you using for backtesting/optimization
- What kind if mathematics are you using in your modeling
and so on and so forth.
1 Like
Even with AMD 2950X, a new box total cost could be $2500 and up. I think it is prudent to wait till Windows scheduling issue of AMD Threadripper chips get cleanup and wait for gen 3 for Threadripper to be released.
Thanks for all the opinions.
AMD has announced that its CEO will give the 2019 Computex (May 27) keynote address. The company is expected to unveil its next-generation of high-end products at the event. The topic of the keynote will be “The Next Generation of High Performance Computing.”
That sounds interesting and it's going to happen soon (6 weeks). I'm waiting eagerly and going to upgrade my system too
By the way, let's not forget, that AmiBroker is probably the fastest and the least demanding (when it comes to system requirements) technical analysis software on the market. Let me quote Tomasz:
So if I were you, I would think twice, if you really need a CPU with 32 cores (64 threads) 
That seems like overkill. Besides for the reasons mentioned in Tomasz's article --> http://www.amibroker.com/kb/2017/10/06/limits-of-multithreading/, in most cases you won't be able to fully utilize that power.
Hmmm...
I guess a Dual AMD EPYC 7601 setup is out of the question then. 