Training Slides: 352 - Tungsten Replicator for MongoDB & Kafka
0 likes68 views
This document provides an overview of replicating data from MySQL to MongoDB and Kafka using Tungsten Replicator. It reviews the replicator workflow, prerequisites for MongoDB and Kafka targets, and sample configurations. The presenter demonstrates Tungsten Replicator's ability to extract data from MySQL, convert rows to messages with metadata, and apply the messages to MongoDB collections or Kafka topics. Requirements like user accounts, ports, and Zookeeper configuration are discussed.
![Extractor Config
[defaults]
user=tungsten
install-directory=/opt/continuent
mysql-allow-intensive-checks=true
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-ext-2
replication-user=tungsten
replication-password=secret
replication-port=3306
enable-heterogeneous-service=true](https://image.slidesharecdn.com/352-tungsten-replicator-for-mongodb-kafka-200910193944/85/Training-Slides-352-Tungsten-Replicator-for-MongoDB-Kafka-9-320.jpg)
![Applier Configs
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=false
rmi-ssl=false
thl-ssl=false
rmi-authentication=false
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=mongodb
replication-user=dbadmin
replication-password=Secret123
replication-port=27017
svc-applier-filters=dropstatementdata
replication-host=atlasendpoint.mongodb.net
property=replicator.applier.dbms.connectString=mongodb+srv://$
{replicator.global.db.user}:${replicator.global.db.password}@$
{replicator.global.db.host}/?retryWrites=true&w=majority
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=mongodb
replication-user=dbadmin
replication-password=Secret123
replication-port=27017
svc-applier-filters=dropstatementdata
MongoDB AtlasMongoDB](https://image.slidesharecdn.com/352-tungsten-replicator-for-mongodb-kafka-200910193944/85/Training-Slides-352-Tungsten-Replicator-for-MongoDB-Kafka-10-320.jpg)
![Applier Configs
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=kafka
replication-user=root
replication-password=null
replication-port=9092
property=replicator.applier.dbms.zookeeperString=localhost:2181
property=replicator.applier.dbms.requireacks=1
Kafka](https://image.slidesharecdn.com/352-tungsten-replicator-for-mongodb-kafka-200910193944/85/Training-Slides-352-Tungsten-Replicator-for-MongoDB-Kafka-11-320.jpg)
Training Slides: 352 - Tungsten Replicator for MongoDB & Kafka
- 1. The MySQL Availability Company Tungsten Replicator Master Class Advanced: Working with MongoDB and Kafka Chris Parker, Customer Success Director, EMEA & APAC
- 2. Topics In this short course, we will • Review replicator flow • Explore MongoDB and Kafka specific pre-requisites • Review configurations • Demo
- 4. How MongoDB Replication Works THL THL Source Replicator: Extractor Target Replicator: Applier THL = Events + Metadata DBMS logs Download transactions via network Mongo Applier Lib Extractor Options MySQL Binary Logging 2 1
- 5. How Kafka Replication Works THL THL Source Replicator: Extractor Target Replicator: Applier THL = Events + Metadata DBMS logs Download transactions via network Kafka Applier (Native)Extractor Options MySQL Binary Logging 2 1 Zookeeper
- 6. What Tungsten Replicator Does to Apply into Kafka • Takes an incoming row and converts it to a message • Message consists of metadata: • Schema name, table name • Sequence number • Commit timestamp • Operation Type • Embedded Message Content
- 7. Object Mapping • Mongo • MySQL Database à Database • Table à Collection • Row à Document • Kafka • MySQL Schema & Table à Topic (schemaname_tablename) • Row à Message
- 8. Prerequisites • Review online documentation • https://docs.continuent.com • Download the Prerequisite Checklist • Extractor/Applier Hosts • OS User • /etc/hosts • sudoers • Ruby • Java • Network • Review Port Requirements • MySQL • my.cnf settings • User accounts • MongoDB • User Account • If using Atlas, export certificates • Add replicator to Atlas Whitelist • Kafka • Configure zookeeper
- 12. Demonstration
- 13. Summary What we have learnt today • Reviewed replicator flow • Explored Kafka and Mongo specific pre-requisites • Reviewed configurations
- 14. Congratulations! • Thank you for following the Tungsten Replicator Master Class! • Make sure you complete all the tests to receive your digital certificate! • Re-watch all the training videos as many times as you want! • How did we do? • We’d love to hear your feedback (The good and the bad!) • Email: [email protected] • Any topics we didn’t cover? Let us know!
- 15. THANK YOU FOR LISTENING continuent.com The MySQL Availability Company Chris Parker, Customer Success Director, EMEA & APAC