Yc-Ma Blog

ONgDB集群数据增量备份方案

 
  • Yc-Ma
  • Jul 16, 2021

Here’s the table of contents:

  1. 修改集群配置
  2. 远程全量热备命令
  3. 远程增量热备命令
  4. 数据恢复脚本
  5. 参考文档

修改集群配置

# Enable online backups to be taken from this database.
dbms.backup.enabled=true
# By default the backup service will only listen on localhost.
# To enable remote backups you will have to bind to an external
# network interface (e.g. 0.0.0.0 for all interfaces).
# The protocol running varies depending on deployment. In a Causal Clustering environment this is the
# same protocol that runs on causal_clustering.transaction_listen_address.
dbms.backup.address=0.0.0.0:6362
causal_clustering.transaction_listen_address=:6000

远程全量热备命令

每周从集群拉取全量数据备份

#!/usr/bin/env bash
###################  backup shell
###################  delay 1s/1m/1h/1d
DELAY=7d
echo backup shell execute delay is:${DELAY}
###################  execute clean log
while [ 1 ]; do
   DATE=$(date "+%Y-%m-%d")
   echo backup date:${DATE}
   echo backup time:$(date "+%Y-%m-%d %H:%M:%S")
   ./bin/neo4j-admin backup --protocol=catchup --from=10.20.13.200:6000 --backup-dir=./data/backup/ --name=graph.db-backup
   sleep ${DELAY}
done
################### background execution this backup shell
# START_BACKUP_SERVER:
# nohup ./ongdb-backup.sh > ongdb-backup.file 2>&1 &
# STOP_BACKUP_SERVER:
# kill -9 `ps -ef|grep ongdb-backup.sh|grep -v grep|awk '{print $2}'`

远程增量热备命令

neo4j-home> export HEAP_SIZE=2G
neo4j-home> bin/neo4j-admin backup --from=10.20.13.200:6000 --backup-dir=./data/backup/ --name=graph.db-backup --fallback-to-full=true --check-consistency=true --pagecache=4G
Destination is not empty, doing incremental backup...
Backup complete.

每天从集群拉取增量数据备份

#!/usr/bin/env bash
###################  backup shell
###################  delay 1s/1m/1h/1d
DELAY=1d
echo backup shell execute delay is:${DELAY}
###################  execute clean log
while [ 1 ]; do
   DATE=$(date "+%Y-%m-%d")
   echo backup date:${DATE}
   echo backup time:$(date "+%Y-%m-%d %H:%M:%S")
   ./bin/neo4j-admin backup --protocol=catchup --from=10.20.13.200:6000 --backup-dir=./data/backup/ --name=graph.db-backup --fallback-to-full=true --check-consistency=true --pagecache=4G
   sleep ${DELAY}
done
################### background execution this backup shell
# START_BACKUP_SERVER:
# nohup ./ongdb-backup-incremental.sh > ongdb-backup.file 2>&1 &
# STOP_BACKUP_SERVER:
# kill -9 `ps -ef|grep ongdb-backup-incremental.sh|grep -v grep|awk '{print $2}'`

数据恢复脚本

bin/neo4j-admin restore --from=/mnt/backup/graph.db-backup-2020-10-28 --database=graph.db --force=false
ongdb@ip-10-20-0-157:~/ongdb-enterprise-3.5.17$ bin/neo4j-admin restore
Missing argument 'from'
usage: neo4j-admin restore --from=<backup-directory> [--database=<name>]
                           [--force[=<true|false>]]
environment variables:
    NEO4J_CONF    Path to directory which contains neo4j.conf.
    NEO4J_DEBUG   Set to anything to enable debug output.
    NEO4J_HOME    Neo4j home directory.
    HEAP_SIZE     Set JVM maximum heap size during command execution.
                  Takes a number and a unit, for example 512m.
Restore a backed up database.
options:
  --from=<backup-directory>   Path to backup to restore from.
  --database=<name>           Name of database. [default:graph.db]
  --force=<true|false>        If an existing database should be replaced.
                              [default:false]

参考文档

https://neo4j.com/docs/operations-manual/3.5/backup/
https://neo4j.com/docs/operations-manual/3.5/backup/performing/#backup-performing-commands
Search