Keeping images ready to deploy

Actually, don't immediately discount option #1. It is faster than you'd think to install a bunch of packages and spray configuration templates across the filesystem. It is more work up front, since you not only have to have a working configuration but you also have to write code to reproduce it, but it requires a hell of a lot less space and leverages the existing Linode master images.

If you want to go this route, take a look at Chef or Puppet too.

I have a "clean" image that I keep around that has a few things that I need to get started. I can clone the disk to another Linode relatively easily and be up and running in ~15 minutes. The only down side to this is that I can't use the full amount of disk space on one of my 'nodes :( For one image it works alright, but if you plan on having multiple images I'd seriously take a look at creating StackScripts or storing your images on your local machine and copying them over as you need them.

Thanks for the ideas, guys. I'll try some of these things and see how it goes…if I come to any interesting conclusions I'll post back here!

I've run some tests, and here is what I found.

The image I was building included starting with a fresh Ubuntu 10.10, installing updates, installing and configuring apache2, install build tools, building ruby from source, and installing and configuring bundler, passenger, and rails. Then deploying my app using capistrano.

I didn't use stackscripts yet, as it wasn't clear how to run those from a different user than root…so I basically just logged in, created a new user, logged in as that new user, and ran my scripts.

So for the script:

5:23 start boot, set up user

5:25 start main_setup.sh script

5:36 start capistrano deployment

5:38 app deployed and operational

So about 16 minutes start to finish. The bulk of the time is spent compiling ruby, pretty much pegged at 100% of one CPU.

I then saved the above image, and cloned it to a new node. The clone process:

6:06 start clone process for 4GB image

6:09 booted, hostname reconfigured, start capistrano deployment

6:11 app deployed and operational

So about 5 minutes start to finish to clone to a new node.

Note that I sized the disk image to only 4GB - if it was larger, as it might well need to be in production, the clone would take longer. Or I could clone a smaller image, then resize it larger prior to boot on the destination - I didn't try that.

Anyway, just wanted to post back here in case anyone else was wondering about this kind of thing!

@gregr:

I didn't use stackscripts yet, as it wasn't clear how to run those from a different user than root…so I basically just logged in, created a new user, logged in as that new user, and ran my scripts.

Stack scripts are run automatically on the building of a linode as the root user, if you wanted to run them as another use you can use sudo inside a script to execute the commands. However you're probably going to want to run as root anyway since installing software etc requires it.

@gregr:

So about 16 minutes start to finish. The bulk of the time is spent compiling ruby, pretty much pegged at 100% of one CPU.

I then saved the above image, and cloned it to a new node. The clone process:

6:06 start clone process for 4GB image

6:09 booted, hostname reconfigured, start capistrano deployment

6:11 app deployed and operational

So about 5 minutes start to finish to clone to a new node.

Note that I sized the disk image to only 4GB - if it was larger, as it might well need to be in production, the clone would take longer. Or I could clone a smaller image, then resize it larger prior to boot on the destination - I didn't try that.

for the first option, you can try:

• speedup your ruby compile, with extra options for make like -j4 (4 compiles at the same time, ideal for the 4 cores linode)

• produce an already ruby binary install, try to produce a rpm/deb/tbz2/etc for your distro and deploy this

for the second option, maybe you can rsync your filesystem, instead of clone the disk, to rsync to a full disk.

@gehena:

for the first option, you can try:

• speedup your ruby compile, with extra options for make like -j4 (4 compiles at the same time, ideal for the 4 cores linode)

• produce an already ruby binary install, try to produce a rpm/deb/tbz2/etc for your distro and deploy this

Mmm…good thoughts; I'll have to look at what the makefile actually does and see if I can make something like -j4 work.

I've also thought about using rvm instead of building from source; somehow it just feels wrong to use rvm on a production server, but maybe I can get over that… :)