Capacity planning - memory (Debian)

First thing, you may want to disable unused Apache modules :

On my gentoo, with only modssl, modauth_pgsql :

> ps aux
[...]
root      1543  0.0  5.4  6500 3268 ?        Ss   02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1549  0.0  6.3 14932 3780 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1550  0.0  6.4 15012 3836 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1551  0.0  6.4 15012 3836 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1559  0.0  6.3 14932 3780 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1561  0.0  6.4 15012 3836 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1562  0.0  6.4 15012 3836 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1563  0.0  6.4 15012 3836 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1565  0.0  6.3 14932 3780 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1566  0.0  6.3 14932 3780 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
apache    1567  0.0  6.3 14932 3780 ?        S    02:36   0:00 /usr/sbin/apache2 -k start -D SSL -D AUTH_PGSQL
[...]

The only LoadModule statements in my main apache2.conf are :

LoadModule access_module                 modules/mod_access.so
LoadModule auth_module                   modules/mod_auth.so
LoadModule log_config_module             modules/mod_log_config.so
LoadModule env_module                    modules/mod_env.so
LoadModule setenvif_module               modules/mod_setenvif.so
LoadModule mime_module                   modules/mod_mime.so
LoadModule cgi_module                    modules/mod_cgi.so
LoadModule dir_module                    modules/mod_dir.so
LoadModule alias_module                  modules/mod_alias.so

One method you could use is to use free, check the 'xxxx' value

in "-/+ buffers/cache: xxxx"

before launching apache2 and after (let it handle some requests).

You can see shared memory segments and their size using the command "ipcs -m". The "nattch" column shows the number of processes using the shared memory at that current moment, I believe. It doesn't tell you what processes each segment belongs to though, so the next thing to do is get some relevant process IDs by using "ipcs -m -i ", where is the shmid you want to look at. I believe that "cpid" is the process that created it. I have no clue what "lpid" is - perhaps the last process to access it?

In any case, you can then translate the PID to a name (if it's still running) by doing something like "ps aux | grep ".

There's probably some better means of doing it than this but that's all I know.

Thanks to you both. I definitely need to review my loaded modules, I'm certain I can get a savings there.

The ipcs command is really interesting (and an education in itself). I wish the numbers lined up better with top or ps, but at least I have a better idea of what's going on with memory.

Thanks!

Please note that ipcs only tells what is allocated using shm calls. This is mostly used by different processes needing a common memory pool to exchange/reuse data. Apache2 configured to use threads doesn't use any for example.

The amount of memory used by an app is difficult to determine because of the memory saving mechanisms buit in Unix :

• copy on write for forked processes,

• shared libraries.

I believe the most practical way is to substract the free output values giving free memory (couting buffers and cache in free memory) before and after an application is launched.

@gyver:

The amount of memory used by an app is difficult to determine because of the memory saving mechanisms buit in Unix
That's certainly been proven to me over the last couple of days. :?

> I believe the most practical way is to substract the free output values giving free memory (couting buffers and cache in free memory) before and after an application is launched.
That seems to be the only way. My concern though is that on my box where Apache lives with mysql, I can't measure how much both of them grow with use over hours or days.

Since you can't tell how much either is using, it will be hard to tell, in a memory crunch, which is the culprit.

What I really need to do, I think, is keep an eye on CPU utilization and swap stats - at least I'll know if I need to get more memory or move a service to a different box.

That's the price I guess for keeping all of your services on one box (not really a good idea).

Thanks again!

ged

Are you sure you are running apache2 and not apache 1.3? The latter uses processes and not threads.

Correct, I am using Apache 1.3. Does that change the discussion?

I.e. am I any better or worse off in trying to calculate memory or CPU usage?

There is also vmstat which–at the very least--will tell you what has happened on your system. If you run it as "vmstat -s" it will give you some useful statistics. Using vmstat, your favorite statistic graphing software (MRTG, Cacti, etc.) and a simple little script to feed the info (especially pages swapped in and pages swapped out) to the graphing app, you can get a good, long term view of what your box is doing with it's swap. If it's high and swapping lots of data all the time, then it's time to investigate what is causing the swapping and think about migrating.

There is, of course, a man page for vmstat that will give you much more info than I can.

I hope this was at least somewhat useful.

--James

PS -> If you don't yet have a favorite graphing tool, Cacti is very powerful and not too hard to set up. I steer away from MRTG now because the new systems (most of which are based on the RRD tools) are much easier to configure. The linux administration world, IMHO, owes a great debt of gratitude to the author of MRTG/RRDTools.

One nice tool is to use pmap, available from:

http://procps.sourceforge.net/

I think that's already a part of the default Debian Sarge (and probably Woody) releases?

For instance, you can use pmap like this:

# pmap -x `ps -C apache -o ppid|tail -1`

Keep in mind that some are shared libs which is loaded once, made available to multiple processes automatically through some virtual memory address mapping… and some of memory usage could be deleted stuff that's still charged against the total memory size of the process.

But pmap -x generally breaks it down such that you can see exactly what is using how much.

-Dan