Database Dumps and Backups
Download as PPTX, PDF1 like1,159 views
The document outlines the importance and methods of database backups and recovery strategies, emphasizing the need for safeguarding critical business data and ensuring quick recovery from failures. It discusses various backup methods such as physical, online, offline, and logical backups, alongside tools like pg_basebackup, barman, and pgbackrest. The conclusion highlights the necessity of understanding recovery requirements to devise an effective backup strategy.
![© Copyright EnterpriseDB Corporation, 2020. All rights reserved.
46
# backup_config_file
[global section]
backup location
database user
…
[server_name section]
hostname
data directory
…
BART/BARMAN/pgBackRest
Common Architecture/Setup](https://image.slidesharecdn.com/databasedumpsandbackups-210224162258/85/Database-Dumps-and-Backups-46-320.jpg)
Database Dumps and Backups
- 1. Database Dumps and Backups Karen Jex February 2021
- 2. Who am I?
- 3. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 3 Agenda • Why Take Backups? • Why do we need to Recover? • What are our Recovery Requirements? • Backup Methods • Backup Strategy • Backup and Recovery Tools
- 4. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 4 Why Take Backups? • Safeguard critical business data • Recover from database failure • Recovery strategy
- 5. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 5 Practice your disaster recovery • The only way to be sure that a backup can be restored is by restoring it • If you need to restore a database, it will be an emergency
- 6. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 6 Why do we need to Recover? • Database server crash • Storage array failure • Batch process incorrectly modified data • Human error (or willful destruction) • Corrupted data file • Creating development/test databases
- 7. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 7 What are our Recovery Requirements? • Maximum Permitted Data Loss (RPO) • Maximum Time to Recover (RTO) • Backup Retention
- 8. Backups aren’t the whole story
- 9. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 9 Physical Database Backup • Offline • Online • Continuous archiving • Storage snapshots Backup Methods Logical Database Backup
- 10. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 10 Physical Database Backups
- 11. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 11 Physical Database Backups Advantages Disadvantages • Generally faster to backup/restore from than logical backup • Allows PITR (if WALs are archived) • Full or incremental backups • Lack of flexibility (backup/restore entire database cluster)
- 12. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 12 Offline Backups (Physical Database Backups) • Shut down database cluster • Copy database files • Start up database cluster
- 13. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 13 Advantages Disadvantages • Any file copy tool can be used • No need to back up associated WAL files • Database unavailable during the backup • PITR is not possible Offline Backups (Physical Database Backups)
- 14. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 14 • WAL = write-ahead log • wal_level : minimal, replica (default) or logical • archive_mode: off, on or always • archive_command WAL files
- 15. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 15 • Tell PostgreSQL you’re starting a backup • Copy the database files • Copy the WAL files • Tell PostgreSQL you’ve finished the backup Online Backups (Physical Database Backups)
- 16. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 16 Advantages Disadvantages • Database available during backup • WAL files must be backed up • WAL parameters must be set correctly Online Backups (Physical Database Backups)
- 17. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 17 • All WAL files are backed up • Allows PITR Continuous Archiving (Physical Database Backups)
- 18. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 18 Storage Snapshots
- 19. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 19 Storage Snapshots • Consistent snapshot of the filesystem • WAL files must be backed up • Perform CHECKPOINT just before the snapshot • Checkpoints of FS containing WAL and each tablespace must be simultaneous • TEST
- 20. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 20 Logical Database Backups my_db.dump
- 21. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 21 Logical Database Backups Advantages Disadvantages • Flexibility • Cross-version compatibility • Slower restore than from physical backup • No PITR
- 22. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 22 Comparison of Backup Methods Allows point in time recovery Database available during backup Allows backup of individual objects Offline Backup Online Backup ✔ Continuous archiving ✔ ✔ Logical Backup/Dump ✔ ✔
- 23. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 23 Backup Strategy • Backup method • Frequency of backups • Full or incremental backups • Retention period
- 25. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 25 • Point in time recovery (PITR) • Central backup architecture • Scheduling • Backup Catalogue • Management of Backup and WAL files Requirements of a Backup/Recovery Tool • WAL archiving • Monitoring and alerting • Compression • Incremental Backups • Backup/restore of individual objects
- 26. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 26 Requirements of a Backup/Recovery Tool Point in time recovery (PITR) Sun Mon Tue Wed Thu Fri Sat Backup 03:00 Batch 05:00 Backup 03:00 Backup 03:00 Batch 05:00 OLTP 09:00 - 18:00 OLTP 09:00 - 18:00 OLTP 09:00 - 18:00 OLTP 09:00 - 18:00 OLTP 09:00 - 18:00 Batch 22:00 Batch 22:00
- 27. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 27 Requirements of a Backup/Recovery Tool Central Backup Architecture
- 28. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 28 Requirements of a Backup/Recovery Tool Scheduling Backup Catalogue
- 29. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 29 Requirements of a Backup/Recovery Tool Management of Backup and WAL files WAL Archiving
- 30. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 30 Requirements of a Backup/Recovery Tool Monitoring and Alerting Backup/Restore selected objects
- 31. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 31 Requirements of a Backup/Recovery Tool Compression Incremental Backups
- 33. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 33 Backup and Recovery Tools Physical Backups Logical Backups • pg_basebackup • BART • Barman • pgBackRest • pg_dump • pg_dumpall • pg_restore
- 34. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 34 No Backup Tool?
- 35. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 35 Custom Scripts https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-BACKUP
- 36. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 36 pg_basebackup 1/2 https://www.postgresql.org/docs/current/app-pgbasebackup.html • Does not affect other clients to the database • Can be used for point-in-time recovery • makes a binary copy of the database cluster files • Puts system in and out of backup mode • Backs up entire database cluster • does not manage a backup library
- 37. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 37 pg_basebackup 2/2 # pg_basebackup -h mydbserver -z -D /my/backup/location/data
- 38. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 38 BART (EDB Backup and Recovery Tool) 1/2 EDB BART 2.6.1 User Guide
- 39. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 39 BART 2/2 # bart backup -s <server_name> # bart restore -s <server_name> # bart show-backups -s <server_name>
- 40. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 40 Barman 1/2 http://docs.pgbarman.org ● Remote backup and restore of multiple servers ● backup catalogs ● incremental backup ● retention policies ● archiving and compression of WAL files and backups ● rsync or PostgreSQL protocol
- 41. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 41 Barman 2/2 # barman backup <server_name> # barman recover <server_name> # barman list-backup <server_name>
- 42. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 42 pgBackRest 1/2 https://pgbackrest.org/ • Parallel Backup & Restore • Local or Remote Operation • Full, Incremental, & Differential Backups • Backup Rotation & Archive Expiration • Delta Restore • Parallel, Asynchronous WAL Push & Get • Tablespace & Link Support • Encryption • Limit restore to specific databases
- 43. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 43 pgBackRest 2/2 # pgbackrest stanza=<server_name> backup # pgbackrest stanza=<server_name> restore # pgbackrest stanza=<server_name> info
- 44. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 44 BART/BARMAN/pgBackRest Common Architecture/Setup
- 45. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 45 • Create DB superuser for backups • Create OS user for backups • Allow DB connections from the backup server (pg_hba.conf ) • Allow passwordless connections between DB server and backup server • Allow passwordless connection to DB from backup server BART/BARMAN/pgBackRest Common Architecture/Setup
- 46. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 46 # backup_config_file [global section] backup location database user … [server_name section] hostname data directory … BART/BARMAN/pgBackRest Common Architecture/Setup
- 47. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 47 BART/BARMAN/pgBackRest Common Architecture/Setup /backup_home server_name archived_wals backups …
- 48. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 48 pg_dump https://www.postgresql.org/docs/current/app-pgdump.html
- 49. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 49 pg_dump https://www.postgresql.org/docs/current/app-pgdump.html
- 50. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 50 pg_dump -F (--format) c (custom) d (directory) t (tar) -n schema -t table
- 51. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 51 pg_dumpall https://www.postgresql.org/docs/current/app-pg-dumpall.html
- 52. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 52 pg_restore https://www.postgresql.org/docs/current/app-pgrestore.html
- 53. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 53 Comparison of Backup and Recovery Tools PITR Scheduling Management of backup files WAL archiving Monitoring Centralised architecture BART ✔ via PEM ✔ ✔ via PEM ✔ Barman ✔ ✔ ✔ ✔ ✔ ✔ pgBackRest ✔ ✔ ✔ ✔ pg_dump pg_dumpall
- 54. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 54 Comparison of Backup and Recovery Tools Compression Incremental backups Specific database / schema / table Global objects Cross-version compatibility BART ✔ ✔ ✔ Barman ✔ ✔ ✔ pgBackRest ✔ ✔ Specific database ✔ pg_dump ✔ ✔ ✔ pg_dumpall ✔ ✔ ✔
- 55. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 55 Conclusions • There is a vast array of different backup/export methods and tools available but they all serve one main purpose, and that is to support your recovery strategy. • It is important to determine your recovery requirements before implementing the backup strategy that fits your specific needs. • Your backup strategy may involve one or more of the methods and tools that we have discussed.
- 56. © Copyright EnterpriseDB Corporation, 2020. All rights reserved. 56 Questions