Storing Images on Web Server & Horizontal Scaling Challenge
I'm opening this topic on the forum to gather suggestions for my current and future infrastructure. Please feel free to provide any suggestions.
Background/Plan
1) The Linode servers are used for a native iPhone application, later I will expand to Android devices
2) The architecture that I want to build would be based on horizontal and vertical scaling
Current Architecture/Infrastructure
1 web server; 2GB RAM, Fedora 21, Apache 2.4.10, PHP 5.6.7
1 database server; 2GB RAM, Fedora 21, MySQL Version: 5.6.23 MySQL Community Server (GPL)
Future Architecture/Infrastructure
add 1 load balancer for web servers
add 1 additional web server and continue to add web servers as needed (horizontal scaling)
the web servers will have more horizontal scaling than vertical
the database server will have vertical scaling
later I will add a replication server and start to look at assigning the replication server as read-only and primary server as write-only
Problem
I initially wanted images to be stored inside the database; making access centralized, and backup a bit easier
However, the dev team decided to store the images on the web server (local file system)
The DB server holds a reference URL i.e.: /Images/[userid]/[postid] (userid and postid are numbers specific to the user and post)
The challenge occurs when I add another web server
My assumption is that I would need to have files replicated from Web Server 1 to Web Server 2 continuously and for every Web Server added
Questions
1) Is it possible to have Apache use another server for storing the images (i.e.: a file server)?
This would make it easier to scale the web servers without having to worry about coping the images to each server. It also allows me to vertically scale the File Server and if possible created a CDN or replicated File Server.
Please let me know if you have any suggestions for possible solutions to storing images on the file system and having multiple web servers.
Thanks,
Ray
2 Replies
1) Use a network file system such as NFS or GlusterFS to host images
2) Store them in the database as you originally planned
3) Use a crude rsync job to replicate images (really easy to set up but not the most reliable)
4) Use a CDN like Amazon S3/CloudFront (or another provider).
Each has it's own merits and problems.
I will look into the NFS or GlusterFS option.
I know there are downsides to the DB storage option as well but I would have preferred it for the simple fact that it is more centralized. The DB option is in the past now, as I think dev would require too much time to refactor the the code. Perhaps at a later time I can revisit it.
I had looked at Amazon initially for complete hosting and their cost was just too much. But I will look at their S3/CloudFront options individually and see if it can fit the budget.
I appreciate the help, please let me know if you have any other suggestions.
Thanks again,
Ray