Memory usage optimization for a newbie

Found this advices on a previous topic:

1. /etc/php.ini – set memory_limit to something lower, like "16M" (note that if it's too low, Wordpress won't run)

2. Use MySQL settings for "small" systems, see the settings in /usr/share/mysql/my-small.cnf

3. Try turning off "KeepAlive" in Apache, or keep it on but set the KeepAliveTimeout to a lower value like 5 (the default is 15 seconds). This is set in /etc/httpd.conf

I don't know if that helped me, but I set KeepAliveTimeout to 5, (on apache2 it's in /etc/apache2/apache2.conf) and memory_limit in php.ini on 16mb (is this ok?) and now I have 297mb free.

I don't understand the 2. one…

The amount of RAM in use is not directly proportional to the number of concurrent visitors. Even if you had zero visitors, you'd still use dozens of megabytes of RAM. That's simply because the various server processes (Apache, MySQL, etc.) need to reside in memory all the time, waiting to serve the first visitor at a microsecond's notice.

IMHO, there isn't much point trying to get your memory usage below current levels. As I said above, just because you're using 80M to serve 1 user, it doesn't mean you'll need 800M to serve 10 users. With proper configuration, memory usage will increase to a certain extent and then flatten out.

Make the most out of the RAM you paid for. Using excessively minimal settings will only hurt performance.

1. Pay attention to the "key_buffer" setting in /etc/mysql/my.cnf

If this server is only used for PHP sites, you can safely assign up to 1/4 of your RAM to "key_buffer".

2. Pay attention to the "MaxClients" setting in /etc/apache2/apache2.conf

Try 15. That'll probably keep Apache's total memory usage under control even with a lot of traffic.

3. Install munin.

That'll allow you to monitor memory usage and various other statistics over time.

@hybinet:

Make the most out of the RAM you paid for.

Thats exactly what I want to do :)

1. what do you mean by "if this server is only used for PHP sites"? Yes, I have sites with php/mysql and some static html/css sites. I will not have sites that use Ruby, or Pearl or something else. Is that what you meant?

2. MaxClients - it was 24, I changed to 15

3. found this tutorial about instaling and using munin. Do I need to install both client and server, or do I need just the client?

Thanks for the advices.

@AndrijaM:

3. found this tutorial about instaling and using munin. Do I need to install both client and server, or do I need just the client?

You need both. The client collects data and the server aggregates and presents it.

@AndrijaM:

what do you mean by "if this server is only used for PHP sites"? Yes, I have sites with php/mysql and some static html/css sites. I will not have sites that use Ruby, or Pearl or something else. Is that what you meant?

That's what I meant. You have Apache with PHP, you have MySQL, and you have nothing else, right? No email server with aggressive virus and spam filtering, no Rails, no Django, no resource-hungry control panel, nothing else to eat up your precious RAM. (Postfix listening only on localhost and used only for sending emails from PHP is okay.)

Thanks ayman.

@hybinet:

You have Apache with PHP, you have MySQL, and you have nothing else, right?

Right, lamp is all I need, no email servers at all, BUT I have phpMyAdmin installed, is that ok? Is there any other alternative for it?

PHPMyAdmin is just a PHP script. It uses no more RAM than any other commonly used PHP script, such as Wordpress.

What about sftp server? I will need to open at least 20 sftp accounts for my clients (btw. is there a way to do it without sftp server?).

What's an SFTP server? You mean transferring files using SSH, right?

If all 20 users are logged on at the same time and doing something crazy, you might have a problem. If it's just one or two users uploading files from time to time, the SSH server (sshd) that's already running in the background will take care of them just ok. There's no such thing as a separate SFTP server, it's built into the sshd.

But do you really want to give system accounts to all of your clients? At the very least, search the web on how to create chrooted user accounts that can only access SFTP.

Well, I have another hosting with another provider, it's shared hosting and to upload files on the server I need to use FTP.

When I signed up with Linode, I found out that ftp is an old and unsecure way of transfering data, so they told me in support to use sftp, which I'm using with Filezilla.

I will host my clients sites on my linode, so I need to provide them a way to transfer files to their web site. Putty or ssh is out of the question for them, as thay are not so tech savy. I need to provide them with Filezilla or something similar.

A friend told me that I need sftp server for that. Thats why I asked :)

As I said, there's no such thing as an SFTP server. It's included in SSH. If you're connecting to your server with PuTTY to administer it, you can also connect to it with FileZilla to transfer files. Try it. Select "SFTP" in FileZilla'a connection manager, and enter the username and password just like you would with plain old FTP.

@hybinet:

Select "SFTP" in FileZilla'a connection manager, and enter the username and password just like you would with plain old FTP.

Just out of interest, if you are running virtual hosts where users have no system accounts, how do you deal with letting users upload web content to their web directory.

I've seen virtual ftp setups, but can you do this sort of thing with SFTP, and how?

There is 2 methods you can use SFTP which is using SSH.. or you can setup FTPS which is FTP over SSL. SFTP comes with SSH and you just need to create a user on the system and manage the keys if you wish. FTPS you setup your desired FTP server and enable the SSL portion if it supports it. My personal ftp server recommendation is vsftpd http://vsftpd.beasts.org/

@hybinet:

… Select "SFTP" in FileZilla'a connection manager, and enter the username and password just like you would with plain old FTP.

Yes, thats exactly what I'm doing, but as kangaby asked I need to find a way to "deal with letting users upload web content to their web directory".

I guess I'll have to try vsftpd, as Schultz recommends.

But, back to the topic and memory optimization, I've heard someting about caching, seems to me it's related to memory optimization. Wikipedia says:

"Web caching is the caching of web documents (e.g., HTML pages, images) to reduce bandwidth usage, server load, and perceived lag"

I would guess if it reduces server load, then it also reduces memory that server uses, am I right?

Caching uses more RAM. The idea is that unused RAM is wasted. It's better to use it to cache data so that you don't need to hit the disk (or if you're caching precomputed data, don't need to hit the CPU).

@AndrijaM:

I need to find a way to "deal with letting users upload web content to their web directory".

Why not just create an account for each user and make each of them own their own files? It's pretty easy these days to chroot user accounts and give them only SFTP access (no shell), since recent versions of the openSSH server include that functionality by default.

@AndrijaM:

I've heard someting about caching

apt-get install php-apc && /etc/init.d/apache2 restart

That will make Apache use 10-20MB more RAM, but your PHP scripts will run nearly twice as fast.

Also don't forget the various caching plugins that are available for popular PHP programs (e.g. wp-super-cache for Wordpress). These plugins usually don't cause increased memory usage, as they use static files instead of RAM.

@hybinet:

It's pretty easy these days to chroot user accounts and give them only SFTP access

That seems like the best option for me, I'll have to try that (not only VPS but also a linux newbie :) ).

After installed php-apc, now when I run free -m +- buffers/cache is now 288 free, and before that it was 265, so it looks like I got 20 mb ram back :) Anyway, everything works fine, my sites are loading fast, all the trouble setting everyting up is now paying off, linode is really great!

Couple more questions:

• I see that many people are using lighttpd and not apache. Whats the difference or in what situations is better to have it then apache?

• linux installation, I've chose Ubuntu 9.10, is there a difference in memory usage with some other version?

You got 20MB back is because you restarted Apache. Make a few visits to your site and the memory usage will return to where it was, plus a few more MB for the APC cache.

1) Apache with modphp handles both static (html, css, images) and dynamic (PHP) requests using the same pool of processes. lighttpd (and nginx, which has recently become more popular) separate them. Static files are served by a preconfigured number of worker processes, and memory usage stays nearly constant whether you have 100 visitors or 10000 visitors. PHP requests are passed to a separate pool of FastCGI processes, which runs with its own set of privileges and keeps memory usage nearly constant. You can see why people say lighttpd and nginx are good for memory-constrained VPS's. The drawback? If you want to use modrewrite (pretty URLs) with your favorite CMS, you'll have to put the rules into the server configuration itself, rather than in a convenient .htaccess file. Works fine if it's just your site, but not so good when you have clients constantly complaining about their .htaccess files not working.

2) As long as you stick to 32-bit, memory usage difference between different distributions is generally negligible (a few MB).

Thank you!