Testing mysql creatively in a sandbox
2 likes920 views
This document discusses using MySQL Sandbox, a tool that allows quickly installing and configuring multiple MySQL database servers for testing purposes using a simple command line interface. Key points: - MySQL Sandbox can install MySQL servers in seconds using a single command, much faster than manual installation. - It supports configurations like replication between multiple servers for testing replication scenarios. - Pre-packaged scripts are provided to easily start, stop, and manage sandboxes. Custom scripts can also be created to automate testing workflows. - Various options allow customizing server configurations like ports, data directories, and replication topologies in a reproducible manner.
More Related Content
Similar to Testing mysql creatively in a sandbox (20)
![[INSIGHT OUT 2011] B12 better my sql security and administration(ronald)](https://cdn.slidesharecdn.com/ss_thumbnails/insightout2011b12bettermysqlsecurityandadministrationronald-111114182222-phpapp01-thumbnail.jpg?width=560&fit=bounds)
Testing mysql creatively in a sandbox
- 1. Testing MySQL creatively in a Sandbox Giuseppe Maxia QA Director, Continuent, Inc http://mysqlsandbox.net This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/ licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. Tuesday, April 17, 12 1
- 2. about me - Giuseppe Maxia • a.k.a. The Data Charmer • QA Director at Continuent, Inc • Long time hacking with MySQL features • Formerly, community manager,db consultant, designer, coder. • A passion for QA and open source • Blogger • http://datacharmer.blogspot.com Tuesday, April 17, 12 2
- 3. MySQL Sandbox lightning presentation Tuesday, April 17, 12 3
- 4. I used to install a lot of MySQL databases for testing MANUALLY Tuesday, April 17, 12 4
- 5. Then, I decided to use Perl ... Tuesday, April 17, 12 5
- 6. DBA pop quiz Tuesday, April 17, 12 6
- 7. HOW MANY KEYSTROKES to install a MySQL server? Tuesday, April 17, 12 7
- 8. HOW MANY KEYSTROKES to install a MySQL server? 10 sb 5.0.83 1234567890 Tuesday, April 17, 12 7
- 9. HOW MANY KEYSTROKES to install 3 MySQL servers in replication? Tuesday, April 17, 12 8
- 10. HOW MANY KEYSTROKES to install 3 MySQL servers in replication? 11 sb r5.0.83 12345678901 Tuesday, April 17, 12 8
- 11. HOW LONG does it take to install a MySQL server? Tuesday, April 17, 12 9
- 12. HOW LONG does it take to install a MySQL server? < 5 seconds time sb 5.0.83 0m1.518s Tuesday, April 17, 12 9
- 13. HOW LONG does it take to install 3 MySQL servers in replication? Tuesday, April 17, 12 10
- 14. HOW LONG does it take to install 3 MySQL servers in replication? < 10 seconds sb 5.0.83 0m4.515s Tuesday, April 17, 12 10
- 15. MySQL Sandbox http://mysqlsandbox.net • Free software (Perl under GPL) • One (unix) host • Many database servers • Single or multiple sandboxes • Customized scripts to use the servers • Standard or circular replication • Installs IN SECONDS Tuesday, April 17, 12 11
- 16. overview MySQL MySQL server server Data DB1 Data DB1 DB2 DB3 DATA DIRECTORY DB2 DB3 PORT SOCKET Tuesday, April 17, 12 12
- 17. overview MySQL MySQL server server Data DB1 SAME Data DB1 DB2 DB3 DATA DB2 DB3 DIRECTORY? /var/lib/mysql /var/lib/mysql DATA CORRUPTION Tuesday, April 17, 12 13
- 18. overview MySQL MySQL server server SAME PORT or SOCKET? 3306 3306 /tmp/mysql.sock /tmp/mysql.sock DOES NOT START Tuesday, April 17, 12 14
- 19. The hard way Tuesday, April 17, 12 15
- 20. The hard way Read the manual Tuesday, April 17, 12 15
- 21. The hard way try to figure out Read the manual what to change Tuesday, April 17, 12 15
- 22. The hard way try to figure out Read the manual Install what to change Tuesday, April 17, 12 15
- 23. The easy way MySQL Sandbox $ make_sandbox /path/to/mysql-5.1.54_linux.tar.gz # it should work always Tuesday, April 17, 12 16
- 24. The easier way Prepare once Install many times # some $ make_sandbox 5.1.54 # preliminary # work Tuesday, April 17, 12 17
- 25. The easiest way Prepare once Install many times # some # preliminary $ sb 5.1.54 # work Tuesday, April 17, 12 18
- 26. MySQL Sandbox VERSION MySQL server Data DB1 DB2 DB3 $SANDBOX_HOME/msb_VERSION/data VERSION /tmp/mysql_VERSION.sock Tuesday, April 17, 12 19
- 27. MySQL Sandbox 5.1.54 MySQL server Data DB1 DB2 DB3 $SANDBOX_HOME/msb_5_1_54/data 5154 /tmp/mysql_5154.sock Tuesday, April 17, 12 20
- 28. MySQL Sandbox 5.5.9 MySQL server Data DB1 DB2 DB3 $SANDBOX_HOME/msb_5_5_09/data 5509 /tmp/mysql_5509.sock Tuesday, April 17, 12 21
- 29. Single Sandbox MySQL customized scripts server start stop restart status clear send_kill use Tuesday, April 17, 12 22
- 30. Multiple Sandbox MySQL customized scripts server start_all stop_all restart_all m n1 status_all s1 n2 clear_all s2 n3 send_kill_a ll use_all Tuesday, April 17, 12 23
- 31. Where do you get it •from CPAN sudo su - cpan MySQL::Sandbox •from launchpad http://launchpad.net/mysql-sandbox Tuesday, April 17, 12 24
- 32. The easy replication way MySQL Sandbox $ make_replication_sandbox /path/to/mysql-5.1.54_linux.tar.gz Prepare once Install many times # some $ make_replication_sandbox # preparation 5.1.54 Tuesday, April 17, 12 25
- 33. default architecture $HOME /sandboxes opt expanded tarballs $SANDBOX_HOME mysql $SANDBOX_BINARY installed sandboxes Tuesday, April 17, 12 26
- 34. default architecture $HOME /sandboxes opt msb_5_0_91 mysql msb_5_1_48 5.0.91 rsandbox_5_1_48 5.1.45 master 5.1.48 node1 5.5.4 node2 Tuesday, April 17, 12 27
- 35. Tuesday, April 17, 12 28
- 36. creating a single sandbox make_sandbox /path/to/mysql-X.X.XX-OS.tar.gz Tuesday, April 17, 12 29
- 37. using a single sandbox # after # make_sandbox # /path/to/mysql-X.X.XX-OS.tar.gz $ cd $SANDBOX_HOME/msb_X_X_XX $ ./use Tuesday, April 17, 12 30
- 38. creating a single sandbox with a specific options file make_sandbox /path/to/mysql-X.X.XX-OS.tar.gz --my_file=/path/to/my.cnf Tuesday, April 17, 12 31
- 39. easily create a sandbox after the first one The long way $ cd $HOME/opt/mysql # $SANDBOX_BINARY $ gunzip -c /path/to/mysql-5.1.34-osx10.5-x86.tar.gz | tar -xf - $ mv mysql-5.1.34-osx10.5-x86 5.1.34 $ make sandbox 5.1.34 Tuesday, April 17, 12 32
- 40. easily create a sandbox after the first one The short way $ make_sandbox --export_binaries path/to/mysql-5.1.34-osx10.5-x86.tar.gz Tuesday, April 17, 12 33
- 41. starting a single sandbox $ cd $SANDBOX_HOME/msb_X_X_XX $ ./start Tuesday, April 17, 12 34
- 42. starting a single sandbox with temporary options $ cd $SANDBOX_HOME/msb_X_X_XX $ ./start --option=value $ ./restart --option=value $ ./start --key-buffer=20000000 Tuesday, April 17, 12 35
- 43. creating a sandbox with custom port and directory $ make_sandbox 5.1.34 -- --sandbox_port=7800 --sandbox_directory=mickeymouse Tuesday, April 17, 12 36
- 44. creating a sandbox with automatic port checking $ make_sandbox 5.1.34 -- --check_port # if 5.1.34 is free # port=5134 # directory=msb_5_1_34 # else # port=5135 (or the first free) # directory=msb_5_1_34_a Tuesday, April 17, 12 37
- 45. create a replication sandbox $ make_replication_sandbox path/to/mysql-5.1.34-osx10.5-x86.tar.gz Tuesday, April 17, 12 38
- 46. create a circular replication sandbox $ make_replication_sandbox --circular=4 path/to/mysql-5.1.34-osx10.5-x86.tar.gz Tuesday, April 17, 12 39
- 47. changing port to an existing sandbox $ sbtool -o port -s /path/to/source/sandbox --new_port=XXXX Tuesday, April 17, 12 40
- 48. installing the innodb plugin $ sbtool -o plugin --plugin=innodb -s /path/to/source/sandbox Tuesday, April 17, 12 41
- 49. creating a replication sandbox with new base port $ make_replication_sandbox --replication_directory=newwdir --check_base_port 5.0.79 # Creates a replication directory under # $SANDBOX_HOME/newdir # The previous one is preserved. # No conflicts happen Tuesday, April 17, 12 42
- 50. creating a stand-alone master $ make_sandbox 5.5.16 -- --master # Creates a sandbox with binary log and # server-id enabled Tuesday, April 17, 12 43
- 51. creating a quick slave ~/sandboxes/rsandbox_5_5_16/m -e 'show variables like "port"' +---------------+-------+ | Variable_name | Value | +---------------+-------+ | port | 19771 | +---------------+-------+ $ make_sandbox 5.5.16 -- --slaveof='master_port=19771' # adds a slave to an existing master Tuesday, April 17, 12 44
- 52. MySQL Sandbox cookbook $ perldoc MySQL::Sandbox::Recipes Tuesday, April 17, 12 45
- 53. testing MySQL creatively Tuesday, April 17, 12 46
- 54. Breaking replication make_replication_sandbox 5.5.16 cd $HOME/sandboxes/rsandbox_5_5_16 ./m -e 'create table t1 (i int not null primary key)' test ./s1 -e 'insert into t1 values (1)' test ./m -e 'insert into t1 values (1)' test ./s1 -e 'show slave statusG' | grep 'Error|Running' Slave_IO_Running: Yes Slave_SQL_Running: No Last_Error: Error 'Duplicate entry '1' for key 'PRIMARY'' on query. Default database: 'test'. Query: 'insert into t1 values (1)' Last_IO_Error: Last_SQL_Error: Error 'Duplicate entry '1' for key 'PRIMARY'' on query. Default database: 'test'. Query: 'insert into t1 values (1)' Tuesday, April 17, 12 47
- 55. Fixing replication ./s1 -e 'delete from t1 where i = 1' test ./s1 -e 'start slave' ./s1 -e 'show slave statusG' | grep 'Error|Running' Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Error: Last_IO_Error: Last_SQL_Error: Tuesday, April 17, 12 48
- 56. Making a slave lag ./s1 -e 'stop slave SQL_THREAD' ./s2 -e 'stop slave SQL_THREAD' ./m < heavy_load_commands.sql # 1 hour later ./s1 -e 'start slave SQL_THREAD' ./s2 -e 'start slave SQL_THREAD' Tuesday, April 17, 12 49
- 57. Options for replication nodes --master_options = name Options passed to the master --slave_options = name Options passed to each slave --node_options = name Options passed to each node --one_slave_options = name Options passed to a specific slave with the format "N:options" Tuesday, April 17, 12 50
- 58. customizing sandboxes during installation (1) make_replication_sandbox --node_options=--high_performance --one_slave_options='1:-c read-only' 5.1.57 Tuesday, April 17, 12 51
- 59. customizing sandboxes during installation (2) ./use_all 'show variables like "%innodb%buffer%"' # master Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 # server: 1: Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 # server: 2: Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 Tuesday, April 17, 12 52
- 60. customizing sandboxes during installation (3) ./use_all 'show variables like "%read_only%"' # master Variable_name Value read_only OFF # server: 1: Variable_name Value read_only ON # server: 2: Variable_name Value read_only OFF Tuesday, April 17, 12 53
- 61. re-playing binary logs • 1) server become unusable for human error • 2) install a new server • 3) load the latest backup • 4) create a sandbox and copy the binary logs to its data directory • 5) make the new server a slave of the sandbox • 6) wait for completion • 7) reset slave • 8) remove the sandbox Tuesday, April 17, 12 54
- 62. Testing Percona Server make_sandbox --add_prefix=ps --export_binaries Percona-Server-5.5.21-osx10.7-.tar.gz # and later make_sandbox ps5.5.21 Tuesday, April 17, 12 55
- 63. DEMO Tuesday, April 17, 12 56
- 64. Participate! Tuesday, April 17, 12 57
- 65. THANKS http://mysqlsandbox.net This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. To view a copy of this license, visit http:// creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. Tuesday, April 17, 12 58