Replication with postgres and Getting started with Logical Replication
- 2. About Me • 15+ years working with data • Avid Volunteer • Avid Picture Taker Twitter: @SQLAsylum Email: [email protected]
- 3. RDBMS types of Replication ● Master-Slave ● Logical/Object ● Standby/Mirror ● Merge/Rack/Master-Master ● Logship/replay
- 4. Master / Slave / Mirror Pros: ● Failover if synchronous ● Asynchronous or Synchronous ● Everything Duplicated ● Can split Reads/Writes Cons: ● 2 points of failure instead of one in synchronous ● Everything Duplicated ● Can delay/slow down writes Use Cases: HA, Splitting Reads/Writes in a real time scenario Master Slave App Begin Transaction Transaction complete Commit to slave(wait for ack) Ack Synchronous
- 5. Merge/Rack/Multi-Master Pros: ● Can write/read from anywhere ● Highly available ● SCALE! Cons: ● Postgres doesn’t have this(out of the box) ● Very difficult to get right ● Conflict resolution needs to be understood Use Cases: Highly available highly scalable needs. Master Master App Merge App
- 6. Log Ship Replay Pros: ● Offsite/Wan ● Disaster Recovery Cons: ● Delayed ● Slow ● Management Use cases: Disaster Recovery, offsite backup Master Master App Wal/Logs Wal/Logs App only knows about Master Copy/ftp Files
- 7. Logical /Object Replication Pros: ● Certain objects ● Destination can be changed ● Different schemas Cons: ● Breaking changes ● Reads the WAL log excessively(v10&V11) ● Management/updating Use cases: Data Warehouse, reporting. Master/ Publisher Master/ Subscriber App Table_a Table_b Table_c Table_d Asynchronous Reads WAL Log App only knows about Master Different indexes/columns can be added to subscriber
- 8. Logical Replication ● Per table basis ● Different Indexes/Schema ● Changes can be made ● Can limit based on DDL(Insert, update, delete) ● Reads the Wal Log (a lot) ● Setup/Snapshots can be painful Image taken from: https://severalnines.com/blog/postgresql-streaming-replication-deep-dive
- 9. When to Use Logical ● Consolodiate databases/systems to one ● Upgrade/migration paths ● Different set of users/schema to get data to. ● Different performance footprint(different indexes) Limits ● Tables must have the same full qualified name between publication and subscription. ● Tables must have primary key or unique key ● Does not replicate schema ,Sequences, Truncate stmnt or Blobs
- 10. Setup/Create 1. Create A publication. 2. Create tables on Subscription. 3. Copy Data from publisher to subscriber(optional depending on size). 4. Create a Subscription.
- 11. Monitoring ● Pg_stat_replication ● Pg_publication ● Pg_subscription ● pg_replication_slots
- 12. Summary ● Replication is needed for read slaves/reporting needs. ● More changes mean more nightmares/work. ● Be aware of Snapshotting/Reloading for large tables.
- 13. About Me • 15+ years working with data • Avid Volunteer • Avid Picture Taker Twitter: @SQLAsylum Email: [email protected] Resources https://severalnines.com/blog/postgresql-streaming-replication-deep-dive https://severalnines.com/blog/overview-logical-replication-postgresql