My first post of 2016 is going to be a quick overview of how to use Data Containers and back them up.
Getting Started
Most of you are probably using a line such as -v /local/path/on/server:/var/lib/postgresql/data to store your data persistently. There is nothing wrong with this but I discovered when I needed to migrate everything from one docker server to another it was problematic so I learned about using Data Containers to store my data.
Data Containers
Data containers are very easy to use and I’ll go over it quickly.
Before doing anything else you need to create the Data Container. In this case, we’re going to use Postgres.
This command only needs to be run once in order to create the data container
docker run -d -v /var/lib/postgresql/data --name data_postgres busybox
Now we’re going to get Postgres running using Graham Gilbert’s Postgres image and link to this data container.
docker run -d --name="postgres-sal" \
--volumes-from data_postgres \
-v /var/lib/postgresql/data \
-e DB_NAME=sal \
-e DB_USER=admin \
-e DB_PASS=password \
--restart="always" \
grahamgilbert/postgres
You’ll see on Line 2 that we’ve got a –volumes-from command, this links your Postgres Container with the Postgres Data Container. On line 3 you see that we’ve removed /local/path/on/server. This is because docker will automatically store the data in /var/lib/postgresql/data on data_postgres.
Now say you want to upgrade your postgres container. It’s as simple as running
docker pull grahamgilbert/postgres
docker postgres-sal stop
docker rm -f postgres-sal
docker run -d --name="postgres-sal" \
--volumes-from data_postgres \
-v /var/lib/postgresql/data \
-e DB_NAME=sal \
-e DB_USER=admin \
-e DB_PASS=password \
--restart="always" \
grahamgilbert/postgres
Now you’ll be running the latest version of the Docker image without losing any data.
Backing Up Your Data Container
Backing up the data is a very simple process. One command will create a temporary docker container with access to your data container, create a compressed file, and save it on your docker host’s storage.
Here’s the script that I use:
#!/bin/sh
YEAR_DIR=$(date +%Y)
MONTH_DIR=$(date +%m)
DAY_DIR=$(date +%d)
BACKUP_PATH="/root/backups/${YEAR_DIR}/${MONTH_DIR}/${DAY_DIR}"
mkdir -p ${BACKUP_PATH}
#Backup Postgres Database
docker run --volumes-from data_postgres -v ${BACKUP_PATH}:/backup busybox tar -zcvf /backup/postgres-$(date +%Y-%m-%d-%H%M).tar.gz /var/lib/postgresql/data
Restoring the Data
Restoring the data is a simple process. First we need to remove the Postgres Container and Postgres Data Container
docker stop postgres-sal
docker rm -f postgres-sal data_postgres
Now we need to create a fresh data container, restore the data, and get Postgres up and running again.
cd /directory/to/backup
docker run -d -v /var/lib/postgresql/data --name data_postgres busybox
docker run --volumes-from data_postgres -v $(pwd):/backup busybox tar xvfz /backup/backup-filename-here.tar.gz
docker run -d --name="postgres-sal" \
--volumes-from data_postgres \
-v /var/lib/postgresql/data \
-e DB_NAME=sal \
-e DB_USER=admin \
-e DB_PASS=password \
--restart="always" \
grahamgilbert/postgres
Now you have Postgres up and running with your backup data.
Additional Notes
I use s3cmd to back up the tar.gz files to a private S3 bucket. Feel free to back up your backups however you would like.
If you are using Data Containers, ensure that you DO NOT run
docker rm $(docker ps -a -q)or any other command that deletes all of your containers. Doing this will delete your data and you’ll have to use your backup in order to get your stuff back.