Backup and restore data

This article describes the process of saving backup copies, restoring and transferring system data between different installations.

MongoDB

MongoDB is a NoSQL database that serves as the primary storage for user data, objects and links, their properties, state history and incidents.

Backup

To export data from the MongoDB to an archive, create and execute the following script:

$sudo /opt/saymon-scripts/mongodump.sh

Script example (you need to set your own HOST, LOGIN and PASS variables):

#!/bin/bash
BACKUP=/backups/mongo/mongodump-$(date +%F-%H-%M).gz
HOST='10.130.0.10'
LOGIN='saymon'
PASS='5tr43ewQ21'

# Outputs a log message, prefixed with current date-time.
log() {
    echo "[`date`] $1"
}

# Create MongoDB backup.
mongodump -h"$HOST" -u"$LOGIN" -p"$PASS" --db saymon --gzip --archive > "$BACKUP"

log "MongoDB stored to $BACKUP"

By default the backup copy is created in the /backups/mongo folder in the gz format.

To automatically and periodically create new backups and delete the old ones, add the corresponding commands into the crontab file. For example:

$sudo crontab -e

0 */2 * * * /opt/saymon-scripts/mongodump.sh
10 */2 * * * ls -tr /backups/mongo | xargs realpath | head -n -6 | xargs rm

Restoring

In order to restore or import data into MongoDB on a new system installation, execute the following script:

$sudo /opt/saymon-scripts/mongorestore.sh

Script example (you need to set your own HOST, LOGIN and PASS variables):

#!/bin/bash
HOST='10.130.0.10'
LOGIN='saymon'
PASS='5tr43ewQ21'

# Outputs a log message, prefixed with current date-time.
log() {
    echo "[`date`] $1"
}

# Directory for MongoDB backups.
DIR=/backups/mongo

# Ask user to specify dump file.
read  -p "Please specify backup file, e.g. mongodump-2021-04-15-14-10.gz: " BACKUP

# Check if BACKUP exists, remove DIR if ndeeded.
if [[ "$BACKUP" = */backups/mongo/* ]]; then
  BACKUP=$(sed 's/\/backups\/mongo\///' <<< "$BACKUP")
fi

if [ -f $DIR/$BACKUP ]; then
  log "Backup file $DIR/$BACKUP found!"
else
  log "Backup file $DIR/$BACKUP not found!"
  exit 1
fi

# Create MongoDB backup.
mongorestore -h"$HOST" -u"$LOGIN" -p"$PASS" --db saymon --drop --noIndexRestore --gzip --archive="$DIR/$BACKUP"
redis-cli -h 10.130.0.10 -a '$6$9UG8HCr0nqj$bVMZfvDKTobl' flushall

log "MongoDB, database saymon, restored from $DIR/$BACKUP"

OpenTSDB

OpenTSDB — is a time series database, where we store metrics that are used to plot graphs.

Backup

In order to export data from OpenTSDB to an archive on the existing installation, create and execute the following script:

sudo /opt/saymon-scripts/hbasedump.sh

OpenTSDB directory might be different depending on the server installation. Ensure that OpenTSDB data is located in the directory specified in the SOURCE variable before creating the script.

OpenTSDB data might be located in the following directories:

  • /var/lib/hbase

  • /data

  • /var/lib/docker/volumes/<volume_id>/_data/hbase-root/hbase

You can try to find the hbase folder with the following command:

$ sudo find / -type d -name 'hbase'

Script example:

#!/bin/bash
SOURCE=/var/lib/hbase
BACKUP=/backups/hbase/hbasedump-$(date +%F-%H-%M).tar.gz

# Outputs a log message, prefixed with current date-time.
log() {
    echo "[`date`] $1"
}

# Create HBase backup.
tar zcvf "$BACKUP" "$SOURCE"

log "HBase stored to $BACKUP"

By default the backup copy is created in the /backups/hbase folder in the .tar.gz format.

Backups directory must exist before executing the script.

To automatically and periodically create new backups and delete the old ones, add the corresponding commands into the crontab file. For example:

$sudo crontab -e

0 */2 * * * /opt/saymon-scripts/hbasedump.sh
10 */2 * * * ls -tr /backups/hbase | xargs realpath | head -n -6 | xargs rm

Restoring

In order to restore or import data into OpenTSDB on a new system installation, create and execute the following script:

sudo /opt/saymon-scripts/hbaserestore.sh

OpenTSDB directory might be different depending on the server installation. Ensure that OpenTSDB data directory is correctly specified in the DEST.

Text of the script:

#!/bin/bash
#
# Outputs a log message, prefixed with current date-time.
log() {
    echo "[`date`] $1"
}

# Directory for OpenTSDB backups.
DIR=/backups/hbase

# Where to restore OpenTSDB data.
DEST=/var/lib/hbase

# Ask user to specify dump file.
read  -p "Please specify backup file, e.g. hbasedump-2021-04-15-14-31.tar.gz: " BACKUP

# Check if BACKUP exists, remove DIR if ndeeded.
if [[ "$BACKUP" = */backups/hbase/* ]]; then
  BACKUP=$(sed 's/\/backups\/hbase\///' <<< "$BACKUP")
fi

if [ -f $DIR/$BACKUP ]; then
  log "Backup file $DIR/$BACKUP found!"
else
  log "Backup file $DIR/$BACKUP not found!"
  exit 1
fi

# Restore from BACKUP.
docker stop opentsdb
rm -rf "$DEST"
mkdir -p "$DEST" && tar xvf $DIR/$BACKUP -C /
docker start opentsdb

log "HBase restored from $DIR/$BACKUP"

Configuration files

To export configuration, copy the /etc/saymon directory which contains server configuration files.

To import configuration, put the backed up configuration files in the /etc/saymon directory.