Maximum Number of Cores

[jerdchen]

Continuing the discussion from What is the maximum number of CPU cores AmiBroker supports for multi-threaded optimization?:

Is it Possible to use 128 Threads with Amibroker ? I have a new Machine withthe newest Threadripper with 64 Cores.

[Pietro]

Amazing ! In my mind Windows is the limiting factor and your memory setup. There is a sysinternal tool vmmap to have a view on broker.exe consuming the stack, heap ...

https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/3
https://devblogs.microsoft.com/oldnewthing/20050729-14/?p=34773

for me it could work if you are reading the db in analysis/optimization process, running several explorations parallel. But if you are doing any file i/o (fput ...) i think you will fail and have to #pragma.

It could/would be very helpful for us, you publish some stats in comparison your "lucky old" machine with this new monster

best regards,
Peter

[Tomasz]

Current maximum is 64 threads per single Analysis window.
But you can run two or more Analysis windows in parallel and then they will use N * 64 threads where N is number of analysis windows.
Plus don't forget that Charts use in separate threads too.

Unless you are doing some really super heavy math in your formula you are not likely to be limited by CPU speed but by RAM bandwidth.

I can easily increase the limit in the program from 64 threads to 128 threads, but I really doubt if you would see any gain, as a) you are likely to be limited by RAM bandwidth and b) you only have 64 true cores. Hyperthreading (SMT) does not count as "real core".

See KB article:

Limits of multithreading

Also as @Pietro wrote, Windows OS by itself limits the number of threads to 64 per group and I am afraid that this would impact real performance too and the program might not be able to run more than 64-threads even if it asked for.

You might need to check yourself. Run two Analysis windows and run Microsoft Process Explorer, it will display all threads in process Properties / Threads window.

[jerdchen]

Hi Peter,

Thank you very much for the hints and tips.
I wasn't aware that the W10 Pro can "only" process 64 threads in parallel.

Your link here was particularly interesting.
--> https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/3

The "old" computer is an AMD Ryzen 9 7950 X, 64 GB RAM (DDR 4) fast SSD Gen 4, 16 core. My trading calculator.

The new computer is a Threadripper, 7980 X, 128 GB RAM (DDR 5!!!!), fast SSD Gen 5 (!!!), 64 core.

I will post comparisons here shortly.

Best regards,

Gerd

[jerdchen]

Hello Tomasz,

Thank you very much for the answer.

As I wrote above, I wasn't aware that the W10 Pro can "only" process 64 threads in parallel.

Further investigations are currently difficult. The AMD Threadripper runs in our house (Lucerne / Switzerland). But I'm currently in our holiday apartment (Ticino/Switzerland).
Access to the new computer via RDP / VPN connection.
Unfortunately I cannot access the task manager via VPN. As well
I have to install Process Explorer first. Both would certainly be interesting.

I will post some results here shortly.

But I was able to test one thing via VPN:
I have two instances of AB open. Identical APX / AFL files are then opened in both instances. I then ran identical optimizations in parallel in both instances.

The computer needs (almost) the same time for just one instance of AB as for 2 instances. The time difference is marginal.

It remains to be said that the Threadripper is extremely fast. The 64 threads certainly play a role here. But above all, the fast DDR 5 RAM memory and the fast SSD Gen 5 should have a very positive effect. I use EOD Norgate data (Current & Past). For example, if you test rankings on SP 900 or R1000... well... then you quickly add up to 2500 titles.

Best regards,

Gerd