DevOps Fest 2020. Николай Маржан. Consistent backups of multi-shard MongoDB
- 1. Percona Backup for MongoDB Solution for consistent backups of MongoDB sharded cluster
- 2. The Main Problem Consistency If there are write operations during the dump operation, the dump will not reflect a single moment in time. Changes made to the database during the backup process can affect the output of the backup. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 2
- 3. Single-Shard Backups Use “mongodump --oplog” option Creates a oplog file as part of the mongodump output. Which contains oplog entries that occur during the mongodump operation. This file provides an consistent snapshot of the state of a mongod instance. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 3
- 5. Multi-Shard Backups Backups created with mongodump do not maintain the atomicity guarantees of transactions across shards. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 5
- 6. Multi-Shard Backups 6 Start NOConsistency ShardB mongodump ShardB oplog ShardC mongodump ShardA oplog ShardC oplog ShardA mongodump
- 7. Why? With mongodump it is basically impossible to stop oplog exactly on the same transaction on all shards simultaneously without stop writes. © 2019 Percona 7
- 8. Multi-Shard Backups 8 Start Consistencyattheend ShardB mongodump ShardB oplog ShardC mongodump ShardA oplog ShardC oplog ShardA mongodump
- 9. Percona Backup for MongoDB Solution for consistent backups of MongoDB sharded cluster
- 10. Agent ● Agent must be attached on localhost of each mongod instance (including secondaries and config server instances). ● Agent connects and works only with/via MongoDB. No need to handle additional security aspects: open ports, TLS/SSL, authorization etc. ● Agent detects if it is a good candidate to do the backup or restore operations and coordinates with the other instances to accomplish the requested actions. © 2019 Percona10
- 11. CLI ● Administrators observe and control the backups or restores with a pbm CLI command that they can run from any host with the access to the MongoDB cluster. ● CLI connects and works only with/via MongoDB. Not needed to handle agent addresses, TLS/SSL, authorization etc. © 2019 Percona11
- 12. How to start using it? ● Create user If authentication is enabled ● Run agent locally for each mongod instance ● Configure location of remote storage ● Run “pbm backup” command © 2019 Percona12
- 14. Current state ● MongoDB 3.6, 4.0, 4.2 versions are supported. ● Backup and restore via S3-protocol. Officially supported object stores: AWS S3, GCP GCS (interoperability), MinIO. ● Backup and restore from locally mounted NAS/SAN. ● Take backup from Secondary or hidden nodes if possible. ● Klaus Post S2 compression (extension of Snappy) enabled by default for oplogs and logical backups. Gzip and LZ4 are supported also. © 2019 Percona14 Percona Solutions
- 15. Roadmap ● Recovery to any point-in-time (PITR) ● Centralize logs ● Encryption support ● Enhanced selection of backup source © 2019 Percona15 Percona Solutions
- 16. We are Hiring! Speak to me ;) © 2019 Percona16
- 17. Q & A Mykola Marzhan Director of Server Engineering