Backup times

We've deployed some improvements over the past 6 hours that should improve initial back up times.

Keep in mind that the initial backup is going to take a while. The incrementals should be fairly quick, depending on how much has changed since the previous backup.

Also, even if a backup fails, all of the data that was successfully backed up is used as a starting point for subsequent backups. The failed just means that at least one file failed to be backed up during…

-Chris

@caker:

We've deployed some improvements over the past 6 hours that should improve initial back up times.

Chris, any clue how much time max/min would be a backup of say 1Gb?

The first backup failed for me as well:

Backup Log

Type Started Finished Duration Status

snapshot 2009-04-03 14:47:52 2009-04-03 19:33:21 4 hours, 45 minutes, 29 seconds failed

Is this the way it is supposed to be? Fair enough, its a beta!

Trying the second backup now.

I've been fiddling with stuff all day today – including taking the backend down for a few minutes, which likely caused the backups to miss files or fail completely. Subsequent snapshots will fill in the holes.

-Chris

While I was surprised at how long it took, my first backup yesterday was successful. I didn't expect it to take just 10 minutes, but not as long as it took.

If it was relatively short (even half an hour to an hour), I wouldn't say it's necessary, but the length of time it takes will make the requirement for a status component more essential (a couple of us in IRC were wondering is it working? is it stuck?). for what's it's worth it took just under 4 hours for 1.4gb.

• If anyone else is basing time on previous migrations/upgrades, I believe the reason that this will also take significantly longer is that the file system is checking and copy every single file individually, rather than the whole node block-by-block.

Edit: * As this was "day zero" many people were testing the service out at once, so it will be interesting to see how long the initial backup pans out over time.

My first try failed. The second try succeeded:

snapshot 2009-04-03 17:48:27 2009-04-03 20:23:08 2 hours, 34 minutes, 41 seconds successful

The backup was a snapshot of my 18432 MB Centos linode.

I also got an excessive i/o email

Your Linode, linode18355, has exceeded the notification threshold (300) for disk io rate by averaging 2461.50 for the last 2 hours.

My guess is everyone is trying out the service.

@MarkJ:

for what's it's worth it took just under 4 hours for 1.4gb.

• If anyone else is basing time on previous migrations/upgrades, I believe the reason that this will also take significantly longer is that the file system is checking and copy every single file individually, rather than the whole node block-by-block.

Doing a block by block backup should actually take alot longer. If it takes 4 hours to do 1.4Gb, imagine how long it would take to backup 12GB (linode 360) block by block.

Of course, the speed should improve as caker tweaks stuff :-).

My first backup (on-demand snapshot) was successful :)

snapshot     2009-04-03 18:06:40     2009-04-03 19:31:40     1 hour, 25 minutes, 0 seconds      successful

This is on a Linode 360.

I'm doing a backup on request. Just a question, the backup… backup only my currently mounted disk, so the one that my linode it's actually using, or even all the others? I have some "cloned" disk, that i used as backup, but with the new features maybe i can delete them to have faster backup.

Another thing, backup use the space i have on my linode?

Thanks, Backup on linode are a great feature!

I think it grabs all profiles and images–as long as they were made using the standard tools and the filesystem is a supported type. I also don't think it matters if the linode is up or down at the time of the backup. All good things to test, just to be sure ;)

The backups are made to a separate backup server and don't use your own space. That's part of what you're paying for!

When you restore, the restored disk images are only as big as needed for the data they contain. So a 10Gb image with only 1Gb of files will restore as a 1Gb disk, which certainly helps when restoring (especially to a smaller linode). :)