Sudden PHP ERROR & Slow Application
I have been experiencing an issue lately with my servers/application, the development team is unable to find the root cause. No significant changes were made to the application or servers. The issue appeared suddenly.
1 web server; Fedora 21, PHP, 2GB RAM
1 database server; Fedora 21, MySQL, 2GB RAM
Server version: Apache/2.4.10 (Fedora)
PHP 5.6.7 (cli) (built: Mar 20 2015 06:12:07) Zend Engine v2.6.0, Copyright © 1998-2015 Zend Technologies with Xdebug v2.3.1, Copyright (c) 2002-2015, by Derick Rethans
1) The servers are used for a native iphone application
2) An SSL was installed on Feb 17th
3) The servers were working effectively, last app testing was Feb 19th
4) No significant changes were made to the application or server after Feb 19th
1) After the Linode Xen Upgrade on Mar 7th & 8th, the server response slowed significantly
2) I am not saying that the Linode Xen upgrade is the cause but it is the only item remaining after troubleshooting
3) The extreme slowness was first noticed on Mar 11th
Issue 1: Upload Images
The app allows a user to upload a maximum of 10 images per post. The user can upload images from their Camera Roll, Google Images, or Take a Photo.
Use Case 1:
If you upload images from your Camera Roll or Google Images, the image size is usually 1MB or lower. In this case all 10 images are uploaded with no issues. However, the upload can take approximately 20-30 seconds. Sometimes it can go to 1 minute.
Use Case 2:
1) If you upload images from Take Photo (iPhone), a single image size is on average 5MB. The app seems to work with uploading 3-4 images without issue, but it takes more than 1 minute for the upload to be successful.
2) If you upload 5 images the system seems to terminate the connection. This is where you can check the Error Log and find a PHP Error. However, there are times where the app just crashes with no error.
3) If you upload 6 or more images the app crashes and no errors are reported in the error log.
Issue 2: Slow Downloads
The app downloads other user posts to your Home Page, these are users you are following. The app also downloads your own posts to your Profile Page. Both the Home Page and Profile Page are slow. The pages can take anywhere from 30-60 seconds to load.
This PHP Error occurs only once in a while when the application does not crash but seems to have the connection terminated.
When the app does crash there is no Errors in the Error Log
When the app does crash, you can just open the app, then try to post again (posting more than 5 images usually crashes the app)
[Tue Mar 31 20:29:43.744128 2015] [:error] [pid 23957] [client 22.214.171.124:4894] PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 39868997 bytes) in Unknown on line 0
Based on the PHP Error above, I have tried to test different configurations within the php.ini file with no success.
The available memory on the server under load is 1.7GB (total of 2GB)
The available CPU on the server under load is 97-98%
The load testing was conducted with JMETER
PHP.INI CONFIG TESTS
memory limit = 128M change to 256M, 512M, 1024M, and -1 (-1 was used for a test only, the default of 128M was re-entered as the issue remained)
post max size = 100M change to 512M
upload max filesize = 2M change to 5M, then 10M
max file uploads = 20 change to 50
enable realpathcachesize = 16k (these are disabled by default, I enabled and changed to 64k)
enable realpathcachettl = 120
Please let me know if you have any suggestions. I have tried to work with Linode support as well. I have provided the Linode support team with outputs for different performance metrics/commands and they have concluded that the RAM and CPU are sufficient.
Also, does anyone know how to check if APC or OpCache installed with PHP? I would like to know if either is installed and then perhaps install OpCache to see if there would be any benefits.
I appreciate the feedback. However, when I check the php.ini and phpinfo() page there is no module for OpCache. Also, if I look at the php.d and php-zts directories there is no .ini file for OpCache. Let me know if you have any thoughts.
I am using PHP 5.6.7.
I appreciate the additional details. I executed the command find / -name opcache.so and no results were returned. Also, if I look at the phpinfo page in the web browser there is no OpCache module found. I agree, it doesn't appear that OpCache is installed. I will look into installing after a few more rounds of testing.
I used Fedora because the development team was much more comfortable with the OS, but in future will change to CentOS or Ubuntu. I appreciate the effort and thank you for your support.
Do you rescale the images being uploaded or re-process them in any way? That could be the reason you need so much memory.