Recommendations (other than Apache)
Does anyone have any recommendations as to what to set the PHP memory limit at? Any other considerations? I've installed APC on the server. I haven't done a whole lot to MySQL.
I use it and it uses next to no RAM for each PHP process spawned and is much faster than Apache.
If MySQL is using most of your RAM, then you should take a look at your my.cnf settings and see what things you can trim down.
top sorted by mem% 4714 web24 20 0 118m 70m 41m S 7 14.2 0:12.35 php-cgi 5698 web24 20 0 113m 65m 41m S 0 13.2 0:13.40 php-cgi 5861 web24 20 0 109m 60m 40m S 0 12.2 0:08.36 php-cgi 6260 web24 20 0 103m 44m 30m S 0 9.0 0:02.16 php-cgi 2197 mysql 20 0 116m 21m 4568 S 0 4.3 0:05.39 mysqld 4265 root 20 0 109m 20m 16m S 0 4.2 0:00.27 apache2 4712 www-data 20 0 111m 14m 6832 S 0 2.8 0:01.94 apache2 4348 www-data 20 0 111m 11m 5136 S 0 2.4 0:01.93 apache2 5551 web33 20 0 93164 9924 6912 S 0 1.9 0:00.03 php-cgi 5725 www-data 20 0 111m 9104 2308 S 2 1.8 0:01.50 apache2 5859 www-data 20 0 111m 8816 2288 S 0 1.7 0:01.02 apache2 5917 www-data 20 0 110m 8744 2280 S 1 1.7 0:00.94 apache2 6259 www-data 20 0 110m 8456 2200 S 1 1.7 0:00.43 apache2
512MB of RAM can handle quite a bit of web load, but you're right that your mail stuff (particularly SpamAssassin it seems) takes up a good chunk of RAM. You don't necessarily have to use two linodes for this, as the costs scale linearly; 1x1024 linodes costs the same as 2x512, and the 1x1024 might be easier to manage. That said, there's something to be said for decoupling services for resiliency purposes, particularly if they're in different datacenters.
There is certain efficiencies to be gained by using Litespeed, Lighttpd, or nginx instead of Apache, but by switching Apache to mpm_worker and fcgi (or equivalent), you've already gotten most of the advantage.
In terms of a PHP memory limit, that depends on your PHP app. Back in the day, the default limit was 16MB, and most everything ran fine with that. These days, the default is higher, and some apps require it go higher still. I don't think there's often occasion to have it higher than 64MB, though, and I think 32 might be the default? I don't remember anymore.
Since you seem to be running one FastCGI family (parent process + one or more children) per website, you should take into account the worst-case scenario where each of those processes use to the maximum permitted amount of memory. Sometimes you simply can't afford to give more than 32MB to each process.
I have Moodle 2.2 installed and that is the biggest memory hog by far on the system and just double checking they say to decrease the PHP memory limit from 128 to 40.