Data access speed, not CPU, is often limiting factor, was: AmiBroker perf

Hello,

So I decided to move forward with the previous thread discussions and upgraded my desktop, I moved from a 4th Gen i7 (8 Core no threading) to the Ryzen (16 Core 32 Thread). In addition, to support the new CPU added all the latest parts, a top-end motherboard, new RAM, and M.2 SSD HDs.

And ... disappointingly, I barely note any performance improvements, in fact, might even be slower. I have a number of strategies that I am familiar with back-testing and optimization and comparing the speed across each machine. During the test, it registers the 8 threads vs 32 threads in the analysis window.

I notice CPU usage barely goes above 5%, see stats below:

image|665x500

Anyone has anyone seen similar or contradictory results?

Understand there is a decreasing performance with cores vs threads, however, even ignoring the threads, the processor is 5 years newer and has double cores. I would expect at a min it should half the time is taken with the old CPU.

I noticed decent improvement when I overclock the clock speeds but really should not need to do this.The only part I can think of is RAM, as both machines have 16GB but doubt 32GB would make any difference in this use case.

Any thoughts on how I could improve raw hardware performance (non-code options)?

1 Like

Unfortunately your post does not include crucial information required to help you. Please follow this advice: How to ask a good question

CPU speed is irrelevant if your data can't be delivered any faster. If you don't get improvement over previous setup it just means that in the OLD SETUP you were already waiting for data.

Are you using 3rd party data source? For example Norgate?

If you observe what you described it typically means that you are WAITING FOR DATA.

FIRST THING is to read existing documentation
http://www.amibroker.com/guide/x_multithreading.html

And then CAREFULLY READ THIS:

The screenshot that you have included is NOT ENOUGH to give you the answer. It just shows that only small portion of symbols (1021 out of 33686) in in the cache, which may indicate wrong settings.

As I wrote many many times on this forum CPU can only be busy IF your formulas are complex. If your formulas are trivial, CPU will WAIT FOR DATA, especially if they are delivered by 3rd party source that are typically much slower than AmiBroker native database.

2 Likes

This year I’ve tried four different CPUs: 10900K, 5800X, 5950X and 11900K in quest for better Excel performance. And surprisingly only 11900K was able to produce speed what I was looking for

Ryzen 9 5950X was the main disappointment because most Windows applications are using only few cores and CPU load is about 5-10% in most scenarios. Also I didn’t like high temperatures in light work

I am still working on database before transferring it to AmiBroker, so can not give you feedback in AB experience with this processors. But seems like something bottlenecking your performance. You can try with local database for example and compare results

Thanks Tomasz, this makes sense, I have read the info on threading.

I am using Norgate Premium data. The waiting for data option, appears likely, as the CPU utilization is so low. Is there any steps I can take to improve this, or is this Norgate limits?

Per other threads I have set the cache to half RAM, so 8GB, didn't make much difference, and also put settings to load all 32000 symbols to cache.

Thanks Ector, I was wondering whether there might be something in the AMD vs Intel. I do some testing with a seperate local DB vs Norgate, and update with performance.

Regarding AMD CPUs they are good:

Regarding Norgate, see previously mentioned thread:

Regarding helping you, I need to re-iterate this again, please follow this advice: How to ask a good question

without FORMULA, without description on WHAT you are doing (precisely, not vaguely - tell exactly if you are running backtest or optimization, on how many symbols, how many data bars, how many optimization steps, etc, ALL details), without screenshot from "INFO" tab, one can not give you any answers.

1 Like

@topmarx, looking at your post's linked image, I see that you also have many static variables.

As you surely know (and is also clearly explained in the multithreading section of the manual), if in your formula you have a section of code included in the following conditional:

if (Status("stocknum") == 0)
{
  // StaticVarSet(...)
}

all the code inside it is executed in single-thread mode, so this too could be a factor to take care of when comparing performance.

1 Like

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.