Talk:Backups with s3sync
From LinodeWiki
[edit] S3sync semi-automated bash scripts for data offloading and restoration.
Please send me your feedback : hareem.haque@gmail.com
[edit] Semi Automated Uploads to S3 bucket. Data offloading
#!/bin/sh
#This script first asks for a users input. And then it will sync content of a specific directory
#with a S3 bucket.
#Specify all the data related to your AWS Account.
#Please remove the "" prior to entering your credentials.
# PLEASE NOTE THAT I HAVE NEVER TESTED THE SCRIPT WITH AN EU BUCKET.
export AWS_ACCESS_KEY_ID="YOUR ACCESS KEY HERE"
export AWS_SECRET_ACCESS_KEY="YOUR SECRET KEY HERE"
#Enter all required info below. Make sure to double check everything that you specify.
echo -n "What directory would you like to offload to S3 ? "
read bkupdir
echo -n "Please specify your bucket ?"
read Bucket
echo -n "Specify your prefix? "
read prefix
echo -n "Specify log name for this process. Please specify a time input "
read mlog
echo -n "Specify log dir path. (plese specify a path to your log directory - Please inclde a /): "
read ldir
#Script will ask you all the required questions.
#Please make sure that you specify the path to your s3sync folder and that is has proper permissons.
#S3SYNCDIR = path to s3sync folder. This is where s3cmd.rb is located.
#I have not used the cache control. I don't use it cause caching slows my EC2 AMI.
BACKUPDIR=$bkupdir
SYNCDIR=/path/to/your/s3sync/folder/
S3BUCKET=$Bucket
S3PREFIX=$prefix
S3STORE=${S3BUCKET}:${S3PREFIX}
# move to the ruby sync direcotry where the .yml file is
# Also you can replace $RANDOM with anything you like.
cd ${SYNCDIR}
./s3sync.rb -r -v -p --cache-control="no-cache" ${BACKUPDIR} ${S3STORE} > $ldir/$mlog-$RANDOM.log
[edit] Semi Automated Downloads from S3 bucket. Data restoration
#!/bin/sh
# I have not tested this script with the EU buckets.
#This script will restore data from one bucket/prefix to your specifide folder.
# Please make sure that you credentials are correct. And remove ""
export AWS_ACCESS_KEY_ID="YOUR ACCESS KEY"
export AWS_SECRET_ACCESS_KEY="YOUR SECRET KEY"
#Enter all required info below. Make sure to double check everything that you specify.
echo -n "What directory would you like to restore your data to ? "
read bkupdir
echo -n "Please specify your bucket ?"
read Bucket
echo -n "Specify your prefix? "
read prefix
echo -n "Specify log name for this process. Please specify a time input "
read mlog
echo -n "Specify log dir path. (plese specify a path to your log directory- Please include a / ): "
read ldir
#Please double check everything.
# Please specify the correct path to your s3sync folder.
BACKUPDIR=$bkupdir
SYNCDIR=/path/to/your/s3sync/folder/
S3BUCKET=$Bucket
S3PREFIX=$prefix
S3STORE=${S3BUCKET}:${S3PREFIX}
# move to the ruby sync direcotry where the .yml file is
cd ${SYNCDIR}
./s3sync.rb -r -v -p --cache-control="no-cache" ${S3STORE} ${BACKUPDIR} $ldir/$mlog-$RANDOM.log
