So, a posting to a news group from some guy in 2001 (5 years old)
and a
paper written in 1995 (11 years old) that has references to papers as
old as 1964, none of which say that measurements of RAM will behave as
I’ve demonstrated today. Then more usenet postings (still no C code
that demonstrates this magic), and finally a recent article from
linuxjournal describing the Linux memory, but nothing that really says
memory will stay around at 80M levels even 20-30 seconds after all ram
has been supposedly freed.
Run your code under a process trace program such as ktrace (Mac OS X).
You should be able to easily grep the resulting dump for system calls
that actually return memory to the OS. Just because free is called
doesn’t mean that the memory allocator has actually notified the kernel
that it no longer needs that huge hunk of memory.
I’m guessing you’ll look for calls to munmap or something similar.
Run your code under a process trace program such as ktrace (Mac OS X). You
should be able to easily grep the resulting dump for system calls that actually return memory to the OS. Just because free is called doesn’t mean
that the memory allocator has actually notified the kernel that it no longer
needs that huge hunk of memory.
In these graphs the red line represents the amount of memory Ruby is
allocating
from the heap. The blue-green regions represent where those allocations
are
coming from in byte offsets from the base of the heap. These graphs no
longer
have the brk() line indicating the end of the heap because (on Linux
anyway) it
exactly tracks the highest allocated heap memory address.
This visually demonstrates that the heap is becoming highly fragmented
with
both scripts, albeit worse with ‘mutex.rb’.
I’m guessing you’ll look for calls to munmap or something similar.
At least on Linux, the allocation pattern used by these scripts never
causes
the heap implementation to get system memory via m/unmap() (as you can
see
visually in my graphs by how the amount ruby allocates using malloc()
never
goes beyond the size of the heap, even during peak allocation periods).
On Aug 29, 2006, at 11:13 PM, Marshall T. Vandegrift wrote:
At least on Linux, the allocation pattern used by these scripts
never causes
the heap implementation to get system memory via m/unmap() (as you
can see
visually in my graphs by how the amount ruby allocates using malloc
() never
goes beyond the size of the heap, even during peak allocation
periods).
Right, so brk/sbrk, but in the environment you are testing, the
process never
actually gives memory back to the kernel. So even if the memory
isn’t used
(i.e. it isn’t part of the working set of the process), from the
point of view
of the kernel it is allocated to the process and so occupies various
tables and
swap/paging space within the kernel.