This question pertains to AmiBroker 6.20.1, Standard Edition (32-bit) running on a Windows 10 Professional (64-bit) system with 16GB of Physical Memory. AmiBroker is configured to allow a maximum of 512MB for In-memory cache.
When I run a backtest, the value for Free Virtual Memory in the Performance Monitor declines as expected as the backtest creates Static Variables. However, when the backtest is complete, even after I've removed all Static Variables, the Free Virtual Memory does not return to anywhere near its value before starting the backtest, even allowing for memory consumed by database cache.
Here's a fairly extreme example:
Free Virtual Memory
Before Backtest 3,700MB
After Backtest 570MB
(all Static Variables are removed)
After flushing Database Cache 850MB
The amount of virtual memory consumed during the backtest is not of concern. The test deliberately creates a large number of Static Variables.
But, how do I reclaim the virtual memory after the backtest is complete and the Static Variables are removed? (The only solution that I've found, so far, is to restart AmiBroker.)
Precisely how do you "remove static variables"?
Memory is allocated for various things, not only for variables. It is allocated for ~~~EQUITY, it is allocated for cache, it is allocated for result list (yes this List view with trades also needs memory) and for dozens of different things.
Also, even if you call free() in C runtime library, it does NOT return memory to the system, because typically C runtime library has its OWN sub-allocator that gets large memory blocks from the system and does NOT return it but re-uses it for subsequent reallocations. That is especially true for small allocations and especially true for 32-bit compiler. So if you are doing zillions of small allocations you would see that. See https://en.wikipedia.org/wiki/C_dynamic_memory_allocation
The memory is not "lost" in any way, it is just saved to be re-used, to prevent memory fragmentation.
I remove memory after running the backtest by executing a small utiltiy AFL exploration file that contains only the StaticVarRemove("*") command. The Performance Monitor shows that the utility does reduce the count of static variables to 0, and frees a modest block of memory.
Alhough I expected, as you said, that memory is not lost in any way but just saved for re-use, I have enountered situations where running multiple projects successively will reduce available virtual memory to such a low point that AmiBroker issues a low memory warning and then crashes (often, but not always).
Is there some action that I can take either programmatically, or manually, within an open AmiBroker session to "reclaim" memory to avoid these crashes? The only solution that I've found, so far, is to restart AmiBroker after encountering a low memory warning.
If you are using that much variables, you should use 64-bit version instead. First it can use memory beyond 4GB and secondly 64-bit C runtime does not implement malloc the same way as 32-bit (64-bit uses new "low fragmentation" heap in Windows 64 and calls HeapAlloc). As for 32-bit version the memory handling is done by 'old' Microsoft C runtime (MSVCRT.DLL) that you can't change (its part of OS starting from Windows XP).