top, you liar!

aka: "Your server memory usage probably isn't what you think it is"

I learned a really valuable lesson in understanding the server today: "free memory" to the system means "wasted memory" (I think I got it from here). I'll elaborate...

The long of it

The problem was, we just upgraded our server from 2GB RAM to 4GB, and moved a whole host of sites off mod_python to mod_wsgi. One of the (supposed) benefits of wsgi is reduction in memory usage, in that a complete python interpreter is not supposed to be loaded with wsgi, as is the case with mod_python. What we had running before was a system with ~20 django sites, usually 6 child apache processes at roughly 200Mb each. I was getting a little nervous about memory usage, we weren't anywhere near capacity on our machine, but were using a tonne (Canadian that is) of memory.

So we upped to 4GB

...And recently switched our sites to WSGI. I should be happy, but a quick 'top' shows me memory usage is at like 96% (or so)!!! How can that be? I run 'ps aux', do some quick math, and yes, we should be using less memory, what's the deal?

Well, as it turns out, Linux (and therefore derivatives of) like to use free (aka wasted) memory to cache disk transactions. So to gain a (more) accurate representation of what is actually being used, try a command like 'free' instead.

free -m

aaron@videostation:~$ free -m
             total       used       free     shared    buffers     cached
Mem:          2023       1963         60          0         22       1094
-/+ buffers/cache:        846       1176
Swap:         5930        207       5722

The important bit is the line '-/+ buffers/cache', where the real figures are, used is not 1963, its 846 (less the cached stuff), and free is really 1176, not 60.

Still enough to give me a good palpatation early in the day. Oh well, off to the next hard learned lesson.

Aaron