Click here to Skip to main content
15,867,771 members
Articles / Database Development / NoSQL
Tip/Trick

Automating backup for MongoDB using CRON and S3CMD

Rate me:
Please Sign up or sign in to vote.
5.00/5 (4 votes)
19 Feb 2013CPOL 49.4K   4   6
This paper attempts to pass all the steps to create an automated backup for a MongoDB Server on a Linux Server where all backups are sent to Amazon AWS S3 service scheduled through crontab.

Table of Contents

Introduction

The tip below describes how to create an automatic backup routine for a MongoDB server in a Linux environment using the Cron scheduler and Amazon S3 to store all backup data.

Background

To complete the steps, you must have:

  • MongoDB 2.2.3 or higher configured in a Linux environment (preferably in an instance on Amazon EC2
  • Amazon S3 account to store all backup data 

Install and Configure S3CMD

S3cmd is a program that allows you to perform all operations in an Amazon S3 using the shell

To install and configure:

Shell
sudo su
sudo yum --enablerepo epel install s3cmd
s3cmd --configure

To test s3cmd, display all buckets in your Amazon S3 account.

Shell
s3cmd ls

Backup Script 

Create a new file shell, named as mongodb_to_s3_backup.sh using any Linux editor (I use vi)

Shell
vi mongodb_to_s3_backup.sh
Bash
#!/bin/bash
 
#Force file syncronization and lock writes
mongo admin --eval "printjson(db.fsyncLock())"
 
MONGODUMP_PATH="/usr/bin/mongodump"
MONGO_HOST="SERVER_IP_HERE" #replace with your server ip
MONGO_PORT="27017"
MONGO_DATABASE="dbname_here" #replace with your database name
 
TIMESTAMP=`date +%F-%H%M`
S3_BUCKET_NAME="bucketname_here" #replace with your bucket name on Amazon S3
S3_BUCKET_PATH="mongodb-backups"
 
# Create backup
$MONGODUMP_PATH -h $MONGO_HOST:$MONGO_PORT -d $MONGO_DATABASE
 
# Add timestamp to backup
mv dump mongodb-$HOSTNAME-$TIMESTAMP
tar cf mongodb-$HOSTNAME-$TIMESTAMP.tar mongodb-$HOSTNAME-$TIMESTAMP
 
# Upload to S3
s3cmd put mongodb-$HOSTNAME-$TIMESTAMP.tar 
   s3://$S3_BUCKET_NAME/$S3_BUCKET_PATH/mongodb-$HOSTNAME-$TIMESTAMP.tar
 
#Unlock database writes
mongo admin --eval "printjson(db.fsyncUnlock())"

Test the script and verify your Amazon S3 bucket.

Shell
bash mongodb_to_s3_backup.sh

Backup Automatization with CRON

To schedule the mongodb_to_s3_backup.sh use the cron scheduler.

Shell
sudo su
crontab -e

Each line (without comments) represents a schedule

Shell
#every day at 01h 00m
00 01 * * * /bin/bash /home/ec2-user/mongodb_to_s3_backup.sh

Just this. Any questions or problems, please, ask me and I'll help you. Be happy ;]

All codes are in my gist account at: https://gist.github.com/lazarofl/4961746

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Architect America-NET Ltda.
Brazil Brazil
Lazaro is a brazilian software architect, live in São Paulo, focusing on web development, NoSql and agile practices for startups.

In his personal life he is a part time runner, speaker, amateur astronomer, gamer and an absorbing knowledge ambulant.

Comments and Discussions

 
QuestionRestore MongoDB Pin
Member 113602197-Jan-15 18:42
Member 113602197-Jan-15 18:42 
QuestionProblem with script (a few suggestions) Pin
Member 1033540714-Oct-13 5:36
Member 1033540714-Oct-13 5:36 
AnswerRe: Problem with script (a few suggestions) Pin
Lazaro Lima22-Nov-13 1:21
Lazaro Lima22-Nov-13 1:21 
QuestionWell explained documented stuff Pin
Bijendra Singh3-Jul-13 12:18
Bijendra Singh3-Jul-13 12:18 
QuestionNot an aritcle. Pin
Dave Kreskowiak19-Feb-13 6:25
mveDave Kreskowiak19-Feb-13 6:25 
AnswerRe: Not an aritcle. Pin
Lazaro Lima19-Feb-13 6:34
Lazaro Lima19-Feb-13 6:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.