Scaling your web app with MySQL replication
9 likes3,121 views
This document discusses using MySQL replication to scale web applications. It explains how replication allows distributing read operations across multiple read-only slave servers to improve performance and handle increased load. Replication also enables high availability by allowing failover to a slave server if the master fails. The document provides an overview of setting up and managing MySQL replication and compares the advantages of using replication over simply scaling up a single database server.
![3 ENABLE THE MASTER
Master
Configuration file
[mysqld]
log-bin=mysql-bin
server-id=1
19
Thursday, 30 September 2010 19](https://image.slidesharecdn.com/scalingwithreplication-100930054006-phpapp01/85/Scaling-your-web-app-with-MySQL-replication-19-320.jpg)
![8 ENABLE THE SLAVE
Slave 1
Configuration file
[mysqld]
server-id=2
relay-log=mysql-relay
read-only
# optional:
log-bin=mysql-bin 24
Thursday, 30 September 2010 24](https://image.slidesharecdn.com/scalingwithreplication-100930054006-phpapp01/85/Scaling-your-web-app-with-MySQL-replication-24-320.jpg)
![7 ENABLE THE NEW SLAVE
Slave 2
Configuration file
uni que!
[mysqld]
mus t be
server-id=3
relay-log=mysql-relay
read-only
# optional:
log-bin=mysql-bin 38
Thursday, 30 September 2010 38](https://image.slidesharecdn.com/scalingwithreplication-100930054006-phpapp01/85/Scaling-your-web-app-with-MySQL-replication-38-320.jpg)
Scaling your web app with MySQL replication
- 1. Scaling your web app with MySQL replication Giuseppe Maxia MySQL Community Team Lead This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. 1 Thursday, 30 September 2010 1
- 2. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 2 Thursday, 30 September 2010 2
- 3. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 3 Thursday, 30 September 2010 3
- 4. database server a simple web application scheme r/w requests web server clients 4 Thursday, 30 September 2010 4
- 5. database server scaling web requests r/w requests web servers load balancer clients 5 Thursday, 30 September 2010 5
- 6. database load on a simple web application r e write a d 85% 15% 6 Thursday, 30 September 2010 6
- 7. write read 20% 80% database load on a successful web application 7 Thursday, 30 September 2010 7
- 8. database server scaling up means buying a bigger ✘ r/w requests database server web servers load balancer clients 8 Thursday, 30 September 2010 8
- 9. write read 20% 80% the bigger database server will eventually have the same problem 9 Thursday, 30 September 2010 9
- 10. read/write master a web application read/only slaves scheme with replication load balancer R/W R/O web servers load balancer clients 10 Thursday, 30 September 2010 10
- 11. r e write a read d 85% 15% 100% read/write master read/only slaves database load with replication 11 Thursday, 30 September 2010 11
- 12. r e write a read d 85% 15% 100% read/write master read/only slaves scaling database load with replication 12 Thursday, 30 September 2010 12
- 13. Replication assessment without replication with replication database handling easy harder performance high lower (binary logs) Point in Time recovery none easy failover none possible write scaling none minimal backup with downtime without downtime read scaling none easy 13 Thursday, 30 September 2010 13
- 14. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 14 Thursday, 30 September 2010 14
- 15. client master transaction binary log reads slave IO thread relay log replication concepts SQL thread reads 15 Thursday, 30 September 2010 15
- 16. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 16 Thursday, 30 September 2010 16
- 17. 1 SHUT DOWN THE DATABASE SERVER Master 17 Thursday, 30 September 2010 17
- 18. 2 MAKE A BACKUP COPY Master 18 Thursday, 30 September 2010 18
- 19. 3 ENABLE THE MASTER Master Configuration file [mysqld] log-bin=mysql-bin server-id=1 19 Thursday, 30 September 2010 19
- 20. 4 RESTART THE MASTER Master 20 Thursday, 30 September 2010 20
- 21. 5 CREATE REPLICATION USER Master SQL command GRANT REPLICATION SLAVE ON *.* to 'slave_user@'10.10.100.%' IDENTIFIED BY 'slave_pass'; 21 Thursday, 30 September 2010 21
- 22. 6 INSTALL MySQL on the slave Slave 1 Make sure that: • You're using the same version of MySQL • You have the same directory structure • The server is not started yet 22 Thursday, 30 September 2010 22
- 23. 7 COPY THE MASTER DATA to the slave Slave 1 23 Thursday, 30 September 2010 23
- 24. 8 ENABLE THE SLAVE Slave 1 Configuration file [mysqld] server-id=2 relay-log=mysql-relay read-only # optional: log-bin=mysql-bin 24 Thursday, 30 September 2010 24
- 25. 9 START THE SLAVE SERVER Slave 1 25 Thursday, 30 September 2010 25
- 26. 10 INITIALIZE THE SLAVE Slave 1 SQL command SET MASTER TO MASTER_HOST=master_IP, MASTER_PORT=3306, MASTER_USER=slave_user, MASTER_PASSWORD='slave_pwd'; 26 Thursday, 30 September 2010 26
- 27. 11 START THE SLAVE SERVICE Slave 1 SQL command START SLAVE; 27 Thursday, 30 September 2010 27
- 28. 12 CHECK THE SLAVE Slave 1 SQL command SHOW SLAVE STATUS G ... Slave_IO_Running: Yes Slave_SQL_Running: Yes ... 28 Thursday, 30 September 2010 28
- 29. Troubleshooting • SHOW SLAVE STATUS says SLAVE_IO_RUNNING=No • Make sure that the slave host can connect to the master • Make sure that master and slave have different Server-id • Check the error log of both master and slave 29 Thursday, 30 September 2010 29
- 30. Testing the slave • Create a table in the master. • Make sure that the slave has replicated the table. • Insert data in the master • read that data in the slave 30 Thursday, 30 September 2010 30
- 31. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 31 Thursday, 30 September 2010 31
- 32. 1 NO NEED TO STOP THE MASTER! Master 32 Thursday, 30 September 2010 32
- 33. 2 STOP THE SLAVE Slave 1 SQL command STOP SLAVE IO_THREAD; # wait until the SQL_THREAD # has done everything STOP SLAVE SQL_THREAD; # STOP THE SERVER 33 Thursday, 30 September 2010 33
- 34. 3 MAKE A COPY OF THE DATA DIRECTORY Slave 1 34 Thursday, 30 September 2010 34
- 35. 4 RESTART THE SLAVE Slave 1 35 Thursday, 30 September 2010 35
- 36. 5 INSTALL MySQL on the new slave Slave 2 Make sure that: • You're using the same version of MySQL • You have the same directory structure • The server is not started yet 36 Thursday, 30 September 2010 36
- 37. 6 COPY THE old slave DATA on the slave Slave 2 37 Thursday, 30 September 2010 37
- 38. 7 ENABLE THE NEW SLAVE Slave 2 Configuration file uni que! [mysqld] mus t be server-id=3 relay-log=mysql-relay read-only # optional: log-bin=mysql-bin 38 Thursday, 30 September 2010 38
- 39. 8 START THE NEW SLAVE Slave 2 39 Thursday, 30 September 2010 39
- 40. 9 CHECK THE SLAVE Slave 2 SQL command SHOW SLAVE STATUS G ... Slave_IO_Running: Yes Slave_SQL_Running: Yes ... 40 Thursday, 30 September 2010 40
- 41. Why it works • No need to issue a CHANGE MASTER TO command. • Because we cloned the old slave • The new slave gets its parameters from the .info files in the data directory 41 Thursday, 30 September 2010 41
- 42. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 42 Thursday, 30 September 2010 42
- 43. From single server application r/w requests 43 Thursday, 30 September 2010 43
- 44. To replication-aware application read/write master read/only slaves R/W load balancer R/O 44 Thursday, 30 September 2010 44
- 45. Single server application $link = mysql_connect( $server_IP, 'mysql_user', 'mysql_password' ); $result = mysql_query( 'INSERT INTO table_name (x) VALUES (1)', $link ); $result = mysql_query( 'SELECT * FROM table_name WHERE x=1', $link ); 45 Thursday, 30 September 2010 45
- 46. Making an application aware of replication <<database handling>> db <<R/W database handling>> IP user db password IP connect user password connect <<read-only database <<R/W database handling>> handling>> read db db write IP IP user user password password read connect read write 46 Thursday, 30 September 2010 46
- 47. Using replication: the WRONG way No Write Yes statement? Connect to the Connect to next available the master slave Read from Write to the slave master R/W split by Stop statement 47 Thursday, 30 September 2010 47
- 48. Why statement split is wrong • Breaks transactions • High risk of inconsistency • Loses or corrupts data 48 Thursday, 30 September 2010 48
- 49. Using replication: the RIGHT way No Write Yes function? Connect to the Connect to next available the master slave Read from Read and write slave from master more Yes more queries? queries? Yes R/W split No No by function Stop 49 Thursday, 30 September 2010 49
- 50. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 50 Thursday, 30 September 2010 50
- 51. Managing replication •MONITORING • … or die 51 Thursday, 30 September 2010 51
- 52. Sample get slave status monitoring slave Running? No Yes master Get master binlog and position Same or later No Yes binlog/position? check table contents alert 52 Thursday, 30 September 2010 52
- 53. slave master GET table GET table GET table CRC table GET CRC CRC CRC same? No Yes monitoring contents Stop alert 53 Thursday, 30 September 2010 53
- 54. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 54 Thursday, 30 September 2010 54
- 55. Replacing Slave a slave crashe s No are there Yes more slaves? STOP the master STOP one slave add the first add another slave slave Stop 55 Thursday, 30 September 2010 55
- 56. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 56 Thursday, 30 September 2010 56
- 57. Replacing the Master master crashe s Let all slaves catch up with execution STOP replication in all slaves FIND the most make it the up to date slave master FIND which run missing connect all transactions are transactions slaves to the missing from to other new master other slaves slaves Stop 57 Thursday, 30 September 2010 57
- 58. Why replication From single server to replication How to set replication Table of Adding a slave contents Using replication Replacing a Managing slave replication Replacing the master Leveraging replication 58 Thursday, 30 September 2010 58
- 59. read/write load master balancing read/only slaves R/W load balancer R/O 59 Thursday, 30 September 2010 59
- 60. master backup slaves STOP SLAVE remove slave perform from load backup balancer attach slave START Let slave to load SLAVE catch up balancer 60 Thursday, 30 September 2010 60
- 61. master make summary tables slaves STOP SLAVE calculate remove slave summary from load tables balancer attach slave Let slave START SLAVE to load catch up balancer 61 Thursday, 30 September 2010 61
- 62. master Partitions innodb for heavy non partitioned statistics slave slave innodb innodb non partitioned partitioned by range slave MyISAM partitioned by range 62 Thursday, 30 September 2010 62
- 63. Simulating master multiple innodb non partitioned dimensions slave slave innodb non partitioned ARCHIVE partitioned by range (date) slave slave ARCHIVE partitioned by range ARCHIVE (location) partitioned by range (product) 63 Thursday, 30 September 2010 63
- 64. Semi-synchronous replication • Available in 5.5 and higher • Makes sure that at least one slave has copied the data. • Increases reliability 64 Thursday, 30 September 2010 64
- 65. client master transaction with regular commit replication execute slave returns binary log to client replication 65 Thursday, 30 September 2010 65
- 66. client transaction master with semi- commit synchronous replication execute slave returns binary log to client sends transaction relay log to slave gets acknowledgement 66 Thursday, 30 September 2010 66
- 67. READ MORE 67 Thursday, 30 September 2010 67
- 68. The MySQL online manual http://dev.mysql.com/doc 68 Thursday, 30 September 2010 68
- 69. High Performance MySQL 69 Thursday, 30 September 2010 69
- 70. MySQL High Availability 70 Thursday, 30 September 2010 70
- 71. Web Operations 71 Thursday, 30 September 2010 71
- 72. Cloud Application Architectures 72 Thursday, 30 September 2010 72
- 73. What we didn't cover (And are matter for more presentations) 73 Thursday, 30 September 2010 73
- 74. Partial replication • Replicating only one or more objects • Specialized slaves • Different storage engines • Different data structures 74 Thursday, 30 September 2010 74
- 75. Row-based replication • Available in 5.1 and higher • Makes your data more consistent. • Fixes many problems with statement-based replication 75 Thursday, 30 September 2010 75
- 76. Delayed replication • Available in 5.6 and higher • Protect replication against human mistakes and data corruption 76 Thursday, 30 September 2010 76
- 77. Tools • Monitoring • Testing and simulating • Repairing • Filtering, improving 77 Thursday, 30 September 2010 77
- 78. THANKS FOR YOUR ATTENTION This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. 78 Thursday, 30 September 2010 78