Webinar Back to Basics 3 - Introduzione ai Replica Set
Download as PPTX, PDF•1 like•773 views
Il documento fornisce un'introduzione ai replica set in MongoDB, spiegando la loro importanza per la durabilità e l'affidabilità dei dati. Viene discusso il ciclo di vita di un replica set, incluse le operazioni di inizializzazione e failover, nonché come scrivere codice per gestire i driver delle applicazioni. Inoltre, il documento affronta le preoccupazioni riguardanti la consistenza dei dati e offre panoramiche su query e operazioni CRUD in un contesto di replica set.
![Vista Client Side
Secondary
host2
Secondary
host3
Primary
host1
Mongo
Client
Monitor
Thread 1
Monitor
Thread 2
{ ismaster : False,
secondary: True,
hosts : [ host1, host2, host3 ] }](https://image.slidesharecdn.com/b2b-webinar3-italian-replicasets-1-170620080428/85/Webinar-Back-to-Basics-3-Introduzione-ai-Replica-Set-37-320.jpg)
![Il Comando ismaster
>>> pprint.pprint( db.command( "ismaster"
))
{u'hosts': [u'JD10Gen-old.local:27017',
u'JD10Gen-old.local:27018',
u'JD10Gen-old.local:27019'],
u'ismaster' : False,
u'secondary': True,
u'setName' : u'replset',
…}](https://image.slidesharecdn.com/b2b-webinar3-italian-replicasets-1-170620080428/85/Webinar-Back-to-Basics-3-Introduzione-ai-Replica-Set-38-320.jpg)
![Cosa significa? - Inserts
def insert_with_recovery( collection, doc ) :
doc[ "_id" ] = ObjectId()
try:
collection.insert_one( doc )
except pymongo.errors.ConnectionFailure, e:
logging.info( "Connection error: %s" % e )
collection.insert_one( doc )
except DuplicateKeyError:
pass](https://image.slidesharecdn.com/b2b-webinar3-italian-replicasets-1-170620080428/85/Webinar-Back-to-Basics-3-Introduzione-ai-Replica-Set-57-320.jpg)
Webinar Back to Basics 3 - Introduzione ai Replica Set
- 2. Back to Basics 2017 : Webinar 3 Introduzione ai Replica Set Massimo Brignoli Principal Solutions Architect MongoDB @massimobrignoli [email protected] V1.0
- 3. Riassunto della Parti 1 e 2 • Perché esistono i NoSQL • I Tipi di database NoSQL • Le funzionalità chiave di MongoDB • Come installare MongoDB • Come eseguire le operazioni CRUD di base • Come creare gli indici • Come usare la funzione explain() • MongoDB Compass e MongoDB Atlas
- 4. Agenda • Durabilità dei Dati • L’approccio di MongoDB: i Replica Set • Il ciclo di vita di un Replica Set • Come scrivere codice quando si usa un replica set
- 5. Replica Sets • Replica set – da 2 a 50 repliche • Replica sets crea un self-healing ‘shard’ • Data center awareness • Replica sets indirizza: • Alta affidabilità • Durabilità e Consistenza • Manutenzione (e.g., HW swaps) • Disaster Recovery Application Driver Primary Secondary Secondary Replication
- 6. Replica Sets – Workload Isolation • Replica sets abilitano la separazione dei workload • Esempio: workload operazionale sul nodo primario e il workload analitico sui nodi secondari eCommerce Application MongoDB Primary In-memory Storage Engine MongoDB Secondary WiredTiger Storage Engine User Data Sessions, Cart, Recommendations MongoDB Secondary WiredTiger Storage Engine Persisted User Data
- 7. Node 1 Node 2 Node 3 Replica Set - Creazione
- 8. Node 1 (Primary) Node 2 Node 3 Replica Set - Inizializzazione rs.initiate()
- 9. Node 1 (Primary) Node 2 (Secondary) Node 3 Replication Heartbeat Replica Set – Secondo nodo rs.initiate() rs.add(“nodo2”)
- 10. Node 1 (Primary) Node 2 (Secondary) Node 3 (Secondary) Replication Replication Heartbeat Replica Set –Terzo nodo rs.initiate() rs.add(“nodo2”) rs.add(“nodo3”)
- 11. Node 2 (Secondary) Node 3 (Secondary) Heartbeat Primary Election Node 1 (Primary) Replica Set – Failure Primary
- 12. Node 1 (Primary) Node 2 (Primary) Node 3 (Secondary) Heartbeat Replication Replica Set - Failover
- 13. Node 2 (Primary) Node 3 (Secondary) Heartbeat Replication Node 1 (Recovery) Replication Replica Set - Recovery
- 14. Node 2 (Primary) Node 3 (Secondary) Heartbeat Replication Node 1 (Secondary) Replication Replica Set - Recovered
- 15. Node 1 (Primary) Node 2 (Secondary) Node 3 (Secondary) Application Driver Read and write Strong Consistency
- 16. Node 1 (Primary) Node 2 (Secondary) Node 3 (Secondary) Application Driver Eventual Consistency Read Read Read and write
- 17. Improved Tunable Consistency • maxStalenessMS • Decide come e quando reindirizzare le query alle repliche secondarie • Legge dalle repliche solo se sono all’interno di una finestra di consistenza definita • Migliora la qualità dei dati mentre si scalano le letture sui secondari • readConcern “linearizable” per la consistenza più forte • Assicura che un nodo sia il primario al momentEnsure that a o della lettura • Assicura che I dati ritornati non saranno mai rollbackati se un altro nodo è eletto come primario
- 19. Oplog
- 20. Rollback
- 21. Durabilità – writeConcern Può perdere dati in caso di caduta del primario senza nessuna notifica.
- 22. Durabilità – writeConcern In caso di caduta del Primary garantisce la non perdita di dati, in quanto già replicati su un altro nodo.
- 23. Overview
- 24. Overview
- 25. Insert arriva al primario
- 26. Secondario fa tail dell’oplog
- 27. Secondario prepara un batch
- 28. Prefetch Workers
- 29. Partiziona i batch per collection
- 30. Applica le entry
- 32. Informa il Primary dell’optime
- 33. Primary notifica il client
- 34. Driver Responsibilities https://github.com/mongodb/mongo-python-driver Driver Authentication & Security Python<->BSON Error handling & Recovery Wire Protocol Topology Management Connection Pool
- 35. Inizializzare MongoClient c = MongoClient( "host1, host2", replicaSet="replset" )
- 36. Vista Client Side Secondary host2 Secondary host3 Primary host1 Mongo Client MongoClient( "host1, host2", replicaSet="replset" )
- 37. Vista Client Side Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 { ismaster : False, secondary: True, hosts : [ host1, host2, host3 ] }
- 38. Il Comando ismaster >>> pprint.pprint( db.command( "ismaster" )) {u'hosts': [u'JD10Gen-old.local:27017', u'JD10Gen-old.local:27018', u'JD10Gen-old.local:27019'], u'ismaster' : False, u'secondary': True, u'setName' : u'replset', …}
- 40. Vista Client Side Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2
- 41. Vista Client Side Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3
- 42. Vista Client Side Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 43. Facciamo una insert c = MongoClient( "host1, host2", replicaSet="replset" ) client.db.col.insert_one( { "a" : "b" } )
- 44. Insert si blocca Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert
- 45. Risposta ismaster da Host 1 Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert ismaster
- 46. Ora la scrittura può procedere Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert Insert
- 47. Più tardi, il server host3 risponde Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 48. Steady State Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 49. La vita Interviene Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖
- 50. Il Monitor può non essersi accordo Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert ConnectionFailure
- 51. Quindi riprova Secondary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 52. Controlla il Primary Secondary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 53. Host 2 è il Primary Primary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 54. Steady State Primary host2 Secondary host3 Secondary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 55. Cosa significa? - Connect import pymongo client = pymongo.MongoClient() try: client.admin.command( "ismaster" ) except pymongo.errors.ConnectionFailure, e : print( "Cannot connect: %s" % e )
- 56. Cosa Significa? - Queries import pymongo def find_with_recovery( collection, query ) : try: return collection.find_one( query ) except pymongo.errors.ConnectionFailure, e : logging.info( "Connection failure : %s" e ) return collection.find_one( query )
- 57. Cosa significa? - Inserts def insert_with_recovery( collection, doc ) : doc[ "_id" ] = ObjectId() try: collection.insert_one( doc ) except pymongo.errors.ConnectionFailure, e: logging.info( "Connection error: %s" % e ) collection.insert_one( doc ) except DuplicateKeyError: pass
- 58. Cosa Significa? - Updates collection.update( { "_id" : 1 }, { "$inc" : { "counter" : 1 }})
- 59. Approfondimenti • Jess Jiryu Davis ha una versione approfondita di questo talk https://emptysqua.re/blog/server-discovery-and-monitoring-in-mongodb- drivers/ • Specifiche del server discovery e monitoring https://github.com/mongodb/specifications/blob/master/source/server- discovery-and-monitoring/server-discovery-and-monitoring.rst
- 60. Ultimo Webinar : Sharding • Come costruire un cluster altamente scalabile e performante • Come rimuovere i colli di bottiglia in scrittura • Come scegliere la chiave di partizionamento Martedì, 27 Giugno 2017, ore 11:00.
- 61. Q&A
Editor's Notes
- #3: Who I am, how long have I been at MongoDB.
- #6: High Availability – Ensure application availability during many types of failures Meet stringent SLAs with fast-failover algorithm Under 2 seconds to detect and recover from replica set primary failure Disaster Recovery – Address the RTO and RPO goals for business continuity Maintenance – Perform upgrades and other maintenance operations with no application downtime Secondaries can be used for a variety of applications – failover, hot backup, rolling upgrades, data locality and privacy and workload isolation
- #35: Present a native language interface - converts python types to BSON objects Convert the JSON query language into commands for the database Convert JSON data into BSON data and vice-versa Handles interfacing to different MongoDB topologies Helps recover from server side outages/network errors Manages the client side connection pool The pymongo driver code is on Github (Apache License)
- #37: Calls i
- #38: Calls i
- #40: State machine, full set of states defined in spec.
- #41: Calls i
- #42: Calls i
- #43: Calls i
- #45: Needs a primary to complete a write.
- #46: Needs a primary to complete a write.
- #47: Needs a primary to complete a write.
- #48: Needs a primary to complete a write.
- #49: Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #50: Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #51: Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #52: Primary is marked as unknown Wakes up all monitor threads to check for a primary.
- #53: Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #54: Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #55: Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #57: Try once. This will accomdate elections. Other errore should be propagated.
- #58: Try once. This will accomdate elections. Other errore should be propagated.
- #59: Can you afford to over or under count. Operations need to be idempotent. Turn an update into a write of a document, cf EventSourcing. Then aggregate on the server.