top, you liar! Post on Feb 27, 2009

aka: "Your server memory usage probably isn't what you think it is"

I learned a really valuable lesson in understanding the server today: "free memory" to the system means "wasted memory" (I think I got it from here). I'll elaborate...

The long of it

The problem was, we just upgraded our server from 2GB RAM to 4GB, and moved a whole host of sites off mod_python to mod_wsgi. One of the (supposed) benefits of wsgi is reduction in memory usage, in that a complete python interpreter is not supposed to be loaded with wsgi, as is the case with mod_python. What we had running before was a system with ~20 django sites, usually 6 child apache processes at roughly 200Mb each. I was getting a little nervous about memory usage, we weren't anywhere near capacity on our machine, but were using a tonne (Canadian that is) of memory.

So we upped to 4GB

...And recently switched our sites to WSGI. I should be happy, but a quick 'top' shows me memory usage is at like 96% (or so)!!! How can that be? I run 'ps aux', do some quick math, and yes, we should be using less memory, what's the deal?

Well, as it turns out, Linux (and therefore derivatives of) like to use free (aka wasted) memory to cache disk transactions. So to gain a (more) accurate representation of what is actually being used, try a command like 'free' instead.

free -m

aaron@videostation:~$ free -m
             total       used       free     shared    buffers     cached
Mem:          2023       1963         60          0         22       1094
-/+ buffers/cache:        846       1176
Swap:         5930        207       5722

The important bit is the line '-/+ buffers/cache', where the real figures are, used is not 1963, its 846 (less the cached stuff), and free is really 1176, not 60.

Still enough to give me a good palpatation early in the day. Oh well, off to the next hard learned lesson.

Aaron

Delete svn directories Post on Jan 09, 2009

Every once in a while we find the need to hack out a bunch of .svn directories, a good example is borrowing a javascript plugin (with 500 nested folders) from a project already committed to a subversion repository. It can be (and really, is) quite painful to go through and remove all those little .svn folders so you can potentially commit the newly-coveted javascript morsel into your new project (if this sounds hackish, it probably is, there must be a better way).

My new favorite hack

rm -rf `find . -type d -name .svn`

Found it here.

Cheers

Windows vs Linux viruses Link on Dec 22, 2008

http://www.theregister.co.uk/2003/10/06/linux_vs_windows_viruses/