| View previous topic :: View next topic |
| Author |
Message |
caker
Linode.com Staff
Joined: 15 Apr 2003
Posts: 2715
Location: Galloway, NJ
|
 Posted: Thu Apr 02, 2009 11:31 pm Post subject: Linode Backup Service (beta) |
 |
|
Linode Backup Service (beta)
If you're participating in the beta and discover an issue, please open a new thread.
We want you to have backups. Hell, I want backups. But they're tedious to configure and monitor. We've all heard the stories (and possibly experienced them) when you need a backup long after configuring them you realize they haven't been running successfully for months. So, our goal for the Linode Backup Service was ambitious: create a reliable, redundant and highly available system, that's easy to use (set-it-and-forget-it), affordable, and Just Works.
Description
The Linode Backup System is designed to be an easy to use, reliable and redundant on-site backup solution for your Linode. It performs backups without causing any interruption of your running system, and is seamlessly integrated into the Linode Manager.
Backing Up
There are four backup slots: Three of the slots are executed and rotated automatically: a daily backup, a 2-7 day old backup, and an 8-14 day old backup. The fourth backup slot is a user-initiated snapshot and remains in place until another user-initiated snapshot is taken.
You can configure the time upon which the automatic backups are initiated from a list of 2 hour windows -- you'll want to perform any database dumps before this window. You can also configure which day of the week to consider for the weeklies.
Restoring
You can restore a backup to any of the Linodes attached to your account, even if it does not have backups enabled. Currently only a full restore is possible.
Redundancy
We built a custom distributed and scalable storage engine which will replicate your files across at least two storage nodes. Even your backups are backed up!
Features and Limitations
The backup system must be able to mount your disk images on the host. If you've used fdisk on your images to create partitions, or created encrypted volumes, or done anything other than use our deployment or disk image creation tools, we won't be able to back up the data. The backup system operates on files, not at the block level.
A failed backup will never rotate out a good one. If a backup fails on the day of a weekly backup, the next oldest backup will be used for that weekly slot.
Files that have been modified, but are the same length and without any metadata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.
Currently, only ext2/3 volumes can be backed up. This limitation will be removed in an upcoming release.
ACLs are correctly backed up and restored.
Pricing Structure (tentative)
Linode 360: $5.00/mo
Linode 540: $7.50/mo
Linode 720: $10.00/mo
Linode 1080: $15.00/mo
Linode 1440: $20.00/mo
Linode 2880: $40.00/mo
Backup Service Beta
How do I participate in the Backup Service Beta?
Participating in the beta is free of charge.
We've only deployed the storage hardware in the Newark, NJ facility. So, for now, you must have a Linode in Newark to participate in the Backup Service Beta. Open a ticket under that Linode and request that we enable backups.
What happens at the end of the beta?
The backup service will be scheduled to be turned off automatically. However, we'll give everyone the chance to retain their backup service and will start charging for it.
What kind of risk is there? Is this thing going to work?
There's little risk to your existing data. However, I wouldn't rely on our backup service as your only form of backup just yet. During the beta period, backups may be inconsistent or incomplete, or we may need to wipe all of the backup data and start fresh. In other words, keep your own backups and don't rely on this thing working. You've been warned!
What about the other facilities?
Atlanta and Dallas sometime later this month, and Fremont early May.
-Chris
Last edited by caker on Mon Mar 29, 2010 2:44 pm; edited 2 times in total |
|
| Back to top |
|
 |
Mr Nod
Junior Member
Joined: 17 Mar 2009
Posts: 39
|
| Posted: Fri Apr 03, 2009 3:13 am Post subject: |
|
|
Yay Caker you've made my day - thanks a lot !  |
|
| Back to top |
|
|
saman007uk
Senior Member
Joined: 16 Oct 2004
Posts: 138
|
| Posted: Fri Apr 03, 2009 5:00 am Post subject: |
|
|
Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption.
Also, will the backups be available outside of the linode network - i.e. can we access them from home? |
|
| Back to top |
|
|
phvt
Senior Member
Joined: 28 Feb 2005
Posts: 74
|
| Posted: Fri Apr 03, 2009 5:56 am Post subject: |
|
|
Awesome, thanks! Looking forward to checking it out! And I'm glad it's in Newark  |
|
| Back to top |
|
|
phvt
Senior Member
Joined: 28 Feb 2005
Posts: 74
|
| Posted: Fri Apr 03, 2009 6:20 am Post subject: Re: Linode Backup Service (beta) |
|
|
| caker wrote: |
Files that have been modified, but are the same length and without any metedata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.
|
This may not work very well for things like MySQL InnoDB tables, which allocate chunks of disk. I realize a database is not the best example, but let's say it's a big, mostly read-only database.
| saman007uk wrote: | | Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption. |
It doesn't sound like it's an LVM-based snapshot, and instead is running rdiff/rsync or something simpler. The original post says we'll need to dump databases to a file prior to the backup window. |
|
| Back to top |
|
|
jcr
Senior Member
Joined: 14 May 2007
Posts: 71
|
| Posted: Fri Apr 03, 2009 8:29 am Post subject: |
|
|
Great idea!
Any idea of how long a backup should take? |
|
| Back to top |
|
|
Mr Nod
Junior Member
Joined: 17 Mar 2009
Posts: 39
|
| Posted: Fri Apr 03, 2009 8:56 am Post subject: |
|
|
| jcr wrote: | Great idea!
Any idea of how long a backup should take? |
I'm backing-up my Linode 360 atm. So far it's taken 1.25 hours. |
|
| Back to top |
|
|
caker
Linode.com Staff
Joined: 15 Apr 2003
Posts: 2715
Location: Galloway, NJ
|
| Posted: Fri Apr 03, 2009 10:34 am Post subject: Re: Linode Backup Service (beta) |
|
|
| phvt wrote: | | caker wrote: |
Files that have been modified, but are the same length and without any metedata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.
|
This may not work very well for things like MySQL InnoDB tables, which allocate chunks of disk. I realize a database is not the best example, but let's say it's a big, mostly read-only database.
|
This will work fine because the mtime changes every time a change is made.
-Chris |
|
| Back to top |
|
|
caker
Linode.com Staff
Joined: 15 Apr 2003
Posts: 2715
Location: Galloway, NJ
|
| Posted: Fri Apr 03, 2009 10:37 am Post subject: |
|
|
| phvt wrote: | | It doesn't sound like it's an LVM-based snapshot, and instead is running rdiff/rsync or something simpler. The original post says we'll need to dump databases to a file prior to the backup window. |
I beg to differ . This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.
| saman007uk wrote: | | Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption. |
Doing a snapshot does NOT always mean you're doing to get your mysql DB files in a consistent state. It can depend on which type of table engine you're using, for instance...
So dump your DBs to a file before the backup window just like you would with any other backup configuration.
-Chris |
|
| Back to top |
|
|
alexsh1
Junior Member
Joined: 22 Sep 2008
Posts: 48
Location: London, UK
|
| Posted: Fri Apr 03, 2009 10:49 am Post subject: |
|
|
| caker wrote: |
I beg to differ . This is a very sophisticated system which *does* involve LVM snapshots on the host. |
I like "very sophisticated" here! |
|
| Back to top |
|
|
dbb
Junior Member
Joined: 12 Aug 2008
Posts: 36
|
| Posted: Fri Apr 03, 2009 10:55 am Post subject: |
|
|
| Just making it clear: ext4 disk images are not currently supported, correct? |
|
| Back to top |
|
|
phvt
Senior Member
Joined: 28 Feb 2005
Posts: 74
|
| Posted: Fri Apr 03, 2009 11:01 am Post subject: |
|
|
| caker wrote: |
I beg to differ . This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.
|
My understanding is that LVM snapshots can be made of running InnoDB tables and restored successfully. The InnoDB tables will be in "crashed" state, but the data will recover automatically.
There's also the option to mysqldump --single-transaction with InnoDB and get a consistent backup without blocking applications.
http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html#option_mysqldump_single-transaction
Other MySQL storage formats (like the default MyISAM) don't have this ability.
By "simple" I didn't mean simplistic, I meant like K.I.S.S.! |
|
| Back to top |
|
|
xoff00
Joined: 03 Apr 2009
Posts: 1
|
| Posted: Fri Apr 03, 2009 11:22 am Post subject: one-time backups and getting the backup offline |
|
|
I'm somewhat more interested in being able to just do a backup on demand (I backup my important files to AWS daily) and grab a copy of that backup for my own archives (i.e. burn a dvd set).
The first is taken care of by the user-selectable backup, but will the second be available? I.E. Will I ever be able to grab a disk image?
...Chris |
|
| Back to top |
|
|
caker
Linode.com Staff
Joined: 15 Apr 2003
Posts: 2715
Location: Galloway, NJ
|
| Posted: Fri Apr 03, 2009 11:22 am Post subject: |
|
|
| phvt wrote: | | caker wrote: |
I beg to differ . This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.
|
My understanding is that LVM snapshots can be made of running InnoDB tables and restored successfully. The InnoDB tables will be in "crashed" state, but the data will recover automatically. |
Yeah, but there's a potential snag when involving a virtualization layer:
We're issuing the LVM snapshot on the host -- not from within your Linode's kernel. Performing a snapshot flushes dirty buffers, but since that's being done on the host your Linode kernel is unaware that this is happening, so it still has dirty buffers that were never written to disk.
This may be moot because of journaling (both at the filesystem and the db level), but, speaking for myself, I'd dump my DBs to a file...
-Chris |
|
| Back to top |
|
|
caker
Linode.com Staff
Joined: 15 Apr 2003
Posts: 2715
Location: Galloway, NJ
|
| Posted: Fri Apr 03, 2009 11:24 am Post subject: Re: one-time backups and getting the backup offline |
|
|
| xoff00 wrote: | | The first is taken care of by the user-selectable backup, but will the second be available? I.E. Will I ever be able to grab a disk image |
Restoring restores to disk images, which you can then mount and pull down on your own with whatever (rsync, scp, tarball, etc).
-Chris |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
RSS 2.0 | Additional RSS options
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
