First of all I can tell you that we will explain it for an architecture with two mongodb nodes in sharding, one of them with the role of router and the other with the role of config server.
In the tests that have been done with mongodump and mongorestore commands, the restoration of mongod data files wasn’t satisfactory and didn’t ensure the integrity of the data because in some cases sharding configuration was loosed and in another cases some data was corrupted even using –oplog option which allow to write the most recent written data while backup is being done. In a few tests restoration crashed when there weere 2.000.000 of entries in database.
Mongodb version used in these tests was 2.0. Filesystem used in data partitions was ext4 above LVM.
The best and most reliable way to backing up mongod data files in sharding (but you have to remember it is a complete backup) is doing a LVM snapshot, is blocking write operations and then making a snapshot backup in a remote directory. Doing this you only block database write operations during LVM snapshot is being made.
This is the diagram of the architecture:
In each server we have to have the following partitions:
- 1 lvm with root partition (/) mounted
- 1 lvm with mongodb data partition mounted (/var/lib/mongodb)
- Free space for making LVM snapshot (at least the same size as the LVM data partition)
If you want to know how resize a LVM you can check this post Resizing Logical Volumes in Linux (LVM).
Once you meet this requirements we will be able to connect to mongodb console:
#mongo -port 30000
and block write operations on the database:
mongo1$ use admin
mongo1$ db.runCommand({fsync:1,lock:1})
then we have to clone the LVM volume where we keep mongod data:
mongo1$ lvcreate -L40G -s -n mongosnapshot /dev/mapper/lvmcondatosmnongo
the next step is unblock write operations on the database:
mongo1$ db.$cmd.sys.unlock.findOne()
and finally we can make the backup on this LVM clone mounting it in a directory of our choice:
#mkdir -p /mnt/backup/mongonapshot #mount /dev/mapper/mongosnapshot /mnt/backup/mongosnapshot
In this step you have to choose between making a raw backup or a filesystem backup. I recommend you making a filesystem backup because you will be able to restore the backup faster and optimize the disk space. If you make a raw backup you copy empty blocks too.
Filesystem backup: #tar -pczf /backups/mongo.tar.gz /mnt/backup/mongosnapshot Raw backup: #dd if=/mnt/backup/mongosnapshot of=/backups/mongo.dd
We already have the backup!
Final step is restore the mongod data files so we have to stop all services in each node and uncompress the backup:
# /etc/init.d/mongodb stop #tar zxvf backups/mongo.tar.gz /var/lib/
or restore the clone image:
#dd if=/backups/mongo.dd of=/var/lib/mongodb
References:
http://www.howtoforge.com/linux_lvm_snapshots_p1
http://www.mongodb.org/display/DOCS/Backups
http://equivocation.org/node/103
http://www.tcpdump.com/kb/os/linux/lvm-resizing-guide/shrink.html