MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committed, and Repeatable
3 likes2,681 views
The document discusses the concept of read isolation in databases, particularly in the context of RDBMS and MongoDB. It explains various isolation levels and challenges such as dirty reads, non-repeatable reads, and phantom reads, providing examples to illustrate each case. The document also outlines how MongoDB handles read isolation through transactions, read concerns, and write concerns, emphasizing the importance of snapshot isolation for reliable transactions.
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committed, and Repeatable
- 1. Jay Runkel, Master Solutions Architect Read Isolation - Making your reads clean, committed, and repeatable jayrunkel
- 3. We are all going to a Philadelphia Phillies game!
- 4. In San Francisco, on Aug 8th
- 5. We all want to sit in Section 126
- 6. One person in each row places order for each row
- 7. Let’s all book at the same time Ready... Set... Go...
- 8. There will likely be collisions 1.Many simultaneous reads and writes 2.Likely that: § Some ticket purchase attempts will fail (conflicts) § System may become sluggish § Users may perceive system as erratic as they try to book seats that “disappear”
- 9. We could solve this by using a queue A lot people would wait...
- 10. Agenda 1. Isolation 2. RDBMS Isolation Levels 3. MongoDB Isolation Levels
- 11. Operations and Transactions Definitions for today: • Operation – any database query (insert, update, select/find, delete) • Transaction – A sequence of operations contained in a start transaction/commit/rollback block
- 12. Part 1 - Isolation
- 13. It’s the I in ACID Atomicity Consistency Isolation Durability
- 14. Databases process many concurrent operations These operations may access the same records and conflict with one another
- 15. Safest Approach Execute all operations/transactions sequentially One at a time • All other operations/transactions wait • No conflicts • Performance sucks Baseball Example: Form a line
- 17. Isolation vs. Concurrency Concurrency Isolation Read Uncommitted Read Committed Repeatable Reads Serializable We will define these levels in a second A little more background information first
- 18. Part 2 - Isolation Levels
- 19. (lack of) Isolation Challenges Dirty Reads Read data from a record that has been modified by another running transaction and not yet committed. Phantom Reads Only committed data is read But, new records may be added (or deleted) by another transaction that commits after the current transaction has started. Non-Repeatable Reads Only committed data is read Reading the same record more than once within a transaction may return different results (the record is updated by a transaction that commits between the reads)
- 20. Time Transaction 1 Transaction 2 0 Starts 1 Starts Update Seat 99 2 Read Seat 99 <irrelevant operation> 3 Commits <irrelevant operation> 4 Commits Dirty Read Transaction 1 returns the results transaction 2 ROLLBACK!
- 21. Time Transaction 1 Transaction 2 0 Starts 1 Select all seats in section Starts 2 Returning seats Reserve seat 28 3 Returning seats Reserve seat 29 4 Returning seats Free seat 27 5 Return seat 28 Commits 6 Commits Phantom Reads Some or all of the changes made by transaction 2 show up in the results of transaction 1
- 22. Time Transaction 1 Transaction 2 0 Starts 1 Read Seat 29 Starts 2 <irrelevant operation> Reserve Seat 29 3 <irrelevant operation> Commits 4 Read Seat 29 5 Commits Non-Repeatable Read The second read in transaction 1 sees the changes made by transaction 2
- 23. RDBMS Isolation Levels Dirty Reads Non-Repeatable Reads Phantom Reads Read Uncommitted yes yes yes Read Committed no yes yes Repeatable Reads no no yes Serializable no no no
- 24. Most RDBMS Don’t Offer All Levels Dirty Reads Non-Repeatable Reads Phantom Reads Read Uncommitted yes yes yes Read Committed no yes yes Repeatable Reads no no yes Serializable no no no Oracle
- 25. Part 3 - MongoDB Read Isolation
- 26. Read Isolation in MongoDB is determined by Transactions (or not) ReadConcern WriteConcern (sometimes)
- 27. A quick detour on read and write concern
- 28. ReadConcern Controls consistency and isolation level of reads # of nodes (in a replica set) that have “committed” the write § Default is “local” db.test.find({x:100}).readConcern(“local”)
- 29. ReadConcern local Node 1 Node 2 Node 3 Application Driver {x: 101} {x: 100} {x : 100}{x: 100} write {x: 101} Application Driver {x: 101} db.test.find({x : {$gt : 100}}) .readConcern(“local”)
- 30. Node 1 Node 2 Node 3 Application Driver {x: 101} {x: 101} {x : 100}{x: 100} write Application Driver {x: 100} ReadConcern: Majority db.test.find({x : {$gt : 100}}) .readConcern(“majority”)
- 31. Write Concern • Intelligent write receipt/confirmation – Specifies the the number of nodes that must have written the write to disk – Default number is 1 db.test.insert({x:100},{writeConcern:{w:2}})
- 32. ReadConcerns ReadConcern Description Available Local value, doesn’t account for chunk migration Local Local value, accounts for chunk migration Majority Write has been acknowledged by a majority nodes in the replica set Linearizable not available with transactions majority plus confirms with secondaries that primary can commit majority writes Snapshot transactions only transaction operations are guaranteed to have read from a snapshot of majority-committed data.
- 33. Node 1 Node 2 Node 3 Application Driver Write Concern 1 {x: 100} {x: 100} {x : 100}{x: 100} writeack db.test.insert({x:100}, {writeConcern:{w:1}})
- 34. Node 1 Node 2 Node 3 Application Driver Write Concern Majority {x: 100} {x: 100} {x : 100}{x: 100} writeack db.test.insert({x:100}, {writeConcern:{w:”majority”}})
- 35. End of Detour
- 36. Read Isolation Without Transactions
- 37. Available/Local ReadConcern → Read Uncommitted Read current version in memory • Possibly before acknowledgement sent • May be rolled back during a failover • Dirty reads Default isolation level for MongoDB • Standalone, replica set, sharded clusters
- 38. Dirty Reads - Reading Rolled Back Data Node 1 Node 2 Node 3 Application Driver {x: 101} {x: 100} {x : 100}{x: 100} write {x: 101} Application Driver {x: 101} Node 1 (Primary)
- 39. Dirty Reads - Reading Rolled Back Data Node 2 Node 3 Application Driver {x: 101} {x : 100}{x: 100} Application Driver {x: 101} {x: 100} {x: 100}
- 40. Multi-Document Reads with Local/Available No point-in-time read operations • Writes made after the start of the read operation may be reflected in results • Inserts, updates, deletes • Multiple versions of the same document may be returned • Matching documents may be missed
- 41. Time Process 1 Process 2 0 Starts 1 Select all seats in section Starts 2 Returning seats Select seat 29 3 Returning seats Select seat 30 4 Returning seats Free seat 27 5 Returning seats Free seat 28 6 Completes Select seat 31 7 Completes Phantom Reads (No Transactions) Some of the changes made by transaction 2 show up in the results of transaction 1
- 42. Majority - Without Transactions No Dirty Reads • Can’t read data that could be rolled back Phantom reads are still possible • Documents read by a query returning multiple documents can still be affected by other write operations
- 43. Read Isolation With Transactions
- 44. Isolation With Multi-Document Transactions “local” and “majority” ReadConcerns are upgraded to “snapshot” isolation This is true in MongoDB 4.0 § May not be true in future versions § Code should specify the required isolation
- 45. Snapshot Isolation With WriteConcern: majority • Operations are guaranteed to have read from a snapshot of majority-committed data • No dirty or phantom reads With WriteConcern:1 • Read from local snapshot • Dirty reads are possible • More performant, less durable
- 46. Snapshot Isolation – WriteConcern: 1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 Si = Snapshot at time i
- 47. Time Transaction 1 Transaction 2 0 s1.startTransaction(wc:1) 1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:1) 2 update({_id: 2}, {$set : {y: 2}}) find() 3 update({_id: 3}, {$set : {z: 2}}) ... 4 Commit (primary) Commit(primary) 5 Commit(majority) 6 Commit (majority) 7 Multi-Document Transaction, RC: snapshot, WC:1 The find in transaction 2 returns the state of the documents before the start of transaction 1
- 48. Multi-Document Transaction, RC: snapshot, WC:1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
- 49. Time Transaction 1 Transaction 2 0 s1.startTransaction(wc:1) 1 update({_id: 1}, {$set : {x: 2}}) 2 update({_id: 2}, {$set : {y: 2}}) 3 update({_id: 3}, {$set : {z: 2}}) 4 Commit (primary) 5 Acknowledgement s2.startTransaction(wc:1) 6 Commit(majority) find() 7 ... 8 Commit Multi-Document Transaction, RC: snapshot, WC:1 The find in transaction 2 returns the state of the documents after (primary commit) of transaction 1
- 50. Multi-Document Transaction, RC: snapshot, WC:1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
- 51. Snapshot Isolation – writeConcern: majority {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:maj}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2
- 52. Time Transaction 1 Transaction 2 0 s1.startTransaction(wc:majority) 1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:majority) 2 update({_id: 2}, {$set : {y: 2}}) find() 3 update({_id: 3}, {$set : {z: 2}}) ... 4 Commit (primary) Commit(primary) 5 6 Commit(majority) 7 Acknowledgement Multi-Document Trans., RC: snapshot, WC:maj The find in transaction 2 returns the state of the documents before the start of transaction 1
- 53. Time Transaction 1 Transaction 2 0 s1.startTransaction(wc:majority) 1 update({_id: 1}, {$set : {x: 2}}) 2 update({_id: 2}, {$set : {y: 2}}) 3 update({_id: 3}, {$set : {z: 2}}) 4 Commit (primary) 5 s2.startTransaction(wc:majority) 6 Commit(majority) find() 7 Acknowledgement ... 8 Commit Multi-Document Trans., RC: snapshot, WC:maj The find in transaction 2 returns the state of the documents before the start of transaction 1
- 54. Multi-Document Transaction, RC: snapshot, WC:maj {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:maj}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
- 55. Review ReadConcern Transactions WriteConcern Isolation Level Available No ReadUncommitted With sharding, may miss or return duplicate documents Local No ReadUncommitted Majority No ReadCommitted Linearizable No ReadCommitted Guarantees connection to “real” primary Snapshot Yes 1 No phantom reads Reads are repeatable Dirty reads are possible Snapshot Yes Majority No dirty or phantom reads Reads are repeatableConcurrency Isolation
- 56. ????Questions