My First 100 days with a MySQL DBMS (WP)

COLLABORATE 15 – IOUG Forum
Database
1 | P a g e “My First 100 days with a MySQL DBMS”
White Paper
My First 100 days with a MySQL DBMS
René Antúnez, The Pythian Group
ABSTRACT
TARGET AUDIENCE
This document will benefit whoever is starting to use and administer a MySQL RDBMS, it covers the basic concepts and
management tips to be able to start as a MySQL Administrator.
EXECUTIVE SUMMARY
WHAT IS A MYSQL RDBMS?
MySQL is one, if not, the world’s most popular open source database and it is a main component of the LAMP
(Linux,Apache,MySQL,Php) open source software stack. Being an open source software, there exists a commercial site
(http://mysql.com) which includes links to webinars, whitepapers, and information on products and services and a developer
zone (http://dev.mysql.com/) includes links to manuals, worklogs, podcasts, labs, articles, news and Planet MySQL. Unlike
other RDBMS, MySQL uses a pluggable storage engine architecture, meaning that that a storage engine can be loaded and
unloaded from a running MySQL server
MYSQL ARCHITECTURE
As many other popular RDBMS’s the MySQL database system also operates client/server architecture. MySQL can be setup
to run multiple instances with different server ids on different sockets and port numbers
• Server MySQL : mysqld is the daemon program that manages databases and tables. Mostly installed using binary pre-
compiled for select Operating System. It is also available open source from different vendors such as Oracle, Percona
and Mariadb. Default port for mysql is 3306 and can be configured
• Client programs: These are programs are connecting to mysqld server by sending requests using network or socket
connection. While there are many clients to be used for mysql connection MySQL provides pre-compiled command
line utility called mysql.
With MySQL being the most popular open source DBMS in the world and with an estimated growth of 16 percent annually
until 2020,we can assume that sooner or later an Oracle DBA will be handling a MySQL database in their shop. This
beginner/intermediate-level session will take you through my journey of an Oracle DBA and my first 100 days of starting to
administer a MySQL database, show several demos and all the roadblocks and the success I had along this path.
Reader will be able to:
• What is the MySQL architecture and how does it relate to the Oracle Architecture.
• Gain an insight into how to administer a MySQL DBMS and what tools are available for a MySQL DBA.
• What are the main concerns regarding backups and replication with a MySQL DBMS.

Database
White Paper
MySQL consists of two-tier architecture while processing data to respond DML statements. MySQL’s architecture on storage
engine and SQL tiers are different than conventional RDBMS systems.
To understand the role of storage engines in MySQL you have to have a basic understanding of MySQL’s API architecture.
1) A query comes in (SQL interface)
2) Is parsed and the syntax is checked (Parser),
3) An execution plan is determined (optimizer)
4) Execution plan sent to the query processor who generates a queue of method calls (Handler calls to the API interface
of the various storage engines).
5) The storage engine handles the actual data/file/disk access.
How it’s done is left up to the individual storage engine. Because of the separation of the server from the storage engines, the
various storage engines can have differing capabilities.
• Connection Tier: The MySQL server (mysqld) accepts connections based on the protocol used and authenticates them
to allow access to Upper Tier. If connections established through this tier it’s passed to upper MySQL tier.
• Upper Tier: This tier has combined layers of SQL parser and optimizer. The server mysqld parses statements and uses
optimizers to help to determine best access method.
• Lower Tier: This tier is more of modular architecture than upper tier. At this tier MySQL has multiple storage engines
those manage own tables. Each storage engine is responsible for managing associated. So bottom line is storage
engines are pluggable to MySQL server. There are differences and restrictions between storage engines.
OPTION FILES
MySQL reads startup options from option files, better known as configuration files and these lists specific startup options
which allows the option set to persist after a server restart. One thing that we need to be aware of is that the option file by
default can be read from a number of locations, so be sure to look in the manual for a full listing of where MySQL looks for

Database
White Paper
the option files. In Linux you normally see them in /etc/my.cnf and /etc/mysql/my.cnf, in Windows INSTALLDIRmy.ini and
INSTALLDIRmy.cnf, be sure to read the latest manual for a full listing of where MySQL looks for the option files.
Example of a global option file.
To find out which configuration files mysqld is loading on Linux, you can run the following shell command:
DISK AND MEMORY SPACE USAGE
MySQL server uses disk space based on storage type selected. Top-level directory per database is created from mysql --datadir
configuration.
a) Database directories: Database directory created per database to hold data files associated with storage engine types.
This directory is created independent from storage engine type.
b) Table format files called (*.frm) files. This file keeps the definition (schema) of the table. Every table will have one .frm
file of its own.
c) Data and index files created for tables under same top-level directory used for database. MyISAM storage engine
creates one file per table for each data and index. So it will have three files per table under its database directory.
d) If using the InnoDB storage engine it uses tablespace to control data and index information for all of its tables. It is
the most important and complex one amongst other engines provided. It’s the only engine provides ACID
compliance as well as other advanced features like stored procedures and triggers.
[client]
port=3306
socket=/tmp/mysql.sock
[mysqld]
port=3306
socket=/tmp/mysql.sock
key_buffer_size=16M
max_allowed_packet=8M
[mysqldump]
quick
$strace -e stat64 mysqld --print-defaults
$lsof -p <mysqld_pid)

Database
White Paper
INSTALLING MYSQL
There are different methods of installing MySQL, but the easiest way to do it, is through a yum repository, below is an
example on a Linux 6.6
The generic Unix/Linux binary file layout of the installation is shown in the table below
Directory Contents
bin Client programs and mysqld server
data Log files and databases
docs Manual in Info format
man Unix manual pages
As root
[root@localhost ~]# groupadd mysql
[root@localhost ~]# useradd -r -g mysql mysql
[root@localhost ~]# yum localinstall mysql-community-release-el6-6.noarch.rpm
[root@localhost ~]# yum repolist enabled | grep "mysql.*-community.*"
[root@localhost ~]# yum repolist all | grep mysql
[root@localhost ~]# yum install mysql-community-server
[root@localhost ~]# service mysqld start
Starting mysqld:[ OK ]
[root@localhost ~]# service mysqld status
mysqld (pid 407) is running.
[root@localhost ~]# mysql_secure_installation
As local user
[mysql@localhost Desktop]$ mysql -u root -p
Enter password:
…
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE USER 'rene'@'%' IDENTIFIED BY 'oracle';
mysql> grant all privileges on *.* to 'rene'@'%' with grant option;
mysql> FLUSH PRIVILEGES;
mysql> exit

Database
White Paper
include Include (header) files
lib Libraries
scripts mysql_install_db
share Misc support files including error messages, sample option files and SQL for database
installation
sql-bench Benchmarks
MYSQL ENGINE TYPES
As we mentioned before, MySQL uses a pluggable storage engine architecture, meaning that that a storage engine can be
loaded and unloaded from a running MySQL server. Two of the most popular are Innodb and MyISAM . As of MySQL 5.5 and
later Innodb is the default storage engine in a fresh MySQL installation.
DIFFERENCES OF INNODB AND MYISAM
a) InnoDB DML operations (add, update and delete data) is ACID (atomic, consistent, isolated and durable) model
compatible.
b) InnoDB has foreign keys and relationship constraints while MyISAM does not
c) InnoDB has granular locking level from table to row-level lock for inserting and updating while MyISAM implements
table-level lock.
d) InnoDB has Stored Procedures and Triggers while MyISAM does not
e) InnoDB allows some online DDL (alter) operations without locking MyISAM does not
mysql> SHOW ENGINES;
+--------------------+---------+----------------------------------------------------------------+--------------+------+-----+
| Engine | Support | Comment | Transactions | XA |
Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+-----+
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL
|
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO
|
| MyISAM | YES | MyISAM storage engine | NO | NO | NO
|
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO
|
| CSV | YES | CSV storage engine | NO | NO | NO
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO
|
| ARCHIVE | YES | Archive storage engine | NO | NO | NO
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO
|
+--------------------+---------+----------------------------------------------------------------+--------------+------+-----+
9 rows in set (0.28 sec)

Database
White Paper
MYSQL TROUBLESHOOTING LOGS
As always logs are the best place to start troubleshooting MySQL has the following files:
• Error Log. - It contains information about errors that occur while the server is running (also server start and stop). If
mysqld notices a table that needs to be automatically checked or repaired, it writes a message to the error log. On some
operating systems it writes the stack trace if mysqld crashes.
• General Query Log. - This is a general record of what mysqld is doing (connect, disconnect, queries). It logs each
SQL statement received from clients and can be very useful when you suspect an error in a client and want to know
exactly what the client sent to mysqld. This log can grow quickly on a moderately busy server - so watch it carefully.
This can be enabled or disabled:
• Slow Query Log. – It is your first line of offense for tuning queries Ιt consists of "slow" SQL statements (as
indicated by its name). It logs SQL statements that took more than the parameter long_query_time in seconds and
required at least min_examined_row_limit rows to be examined. To disable or enable the slow query log, use the global
system variable slow_query_log. Set slow_query_log to 0 (or OFF) to disable the log or to 1 (or ON) to enable it.
OBTAINING EXECUTION PLAN INFORMATION
When you issue a query, the MySQL Query Optimizer tries to devise an optimal plan for query execution. The EXPLAIN
statement provides information about how MySQL executes statements. You can see information about the plan by prefixing
the query with EXPLAIN. EXPLAIN EXTENDED can be used to obtain additional execution plan information. As of
MySQL 5.7.3, the EXPLAIN statement is changed so that the effect of the EXTENDED keyword is always enabled.
The order of the tables given in the EXPLAIN output matters. Tables listed in the order that MySQL reads them while
processing and performing the nested-loop joins. MySQL reads a row from the first table, and then finds a matching row in
the second table, the third table, and so on. When all tables are processed, MySQL outputs the selected columns and
backtracks through the table list until a table is found for which there are more matching rows. The next row is read from this
table and the process continues with the next table.
If you detect that you have a problem with indexes not being used when you believe that they should be, run ANALYZE
TABLE to update table statistics, such as cardinality of keys, that can affect the choices the optimizer makes.
SET GLOBAL general_log = 'OFF';
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 10
log_queries_not_using_indexes = 1
mysql> explain extended select j1.c1 from j1, j2, j3 where j1.c1 = j2.c1 and j3.c1 = j1.c1;
+----+-------------+-------+------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+-------------+
| 1 | SIMPLE | j1 | ALL | NULL | NULL | NULL | NULL | 2 | |
| 1 | SIMPLE | j2 | ALL | NULL | NULL | NULL | NULL | 3 | Using where |
| 1 | SIMPLE | j3 | ALL | NULL | NULL | NULL | NULL | 4 | Using where |
+----+-------------+-------+------+---------------+------+---------+------+------+-------------+

Database
White Paper
LOGICAL AND PHYSICAL BACKUPS
Backups are always important when managing data. MySQL has several ways to do a backup.
LOGICAL BACKUP
The first way is doing a logical backup; this will save the logical structure and the data content of our database. This type of
backup is suitable for small databases. A logical backup does not include log and/or configuration files or other database
related files. A very common tool is mysqldump, it will pull data out of the server, and write it down in logical format. One great
advantage of using mysqldump is that is machine independent.
PHYSICAL BACKUP
The backup consists of exact copies of database directories and files. Typically this is a copy of all or part of the MySQL data
directory, so it normally will contain log and/or configuration files or other database related files. One very common way to
do physical backups is through filesystem snapshot, but there is a database level lock that has to happen for these to be
consistent. The steps for this type of backup are below:
a) FLUSH TABLES WITH READ LOCK;
b) Take the filesystem snapshot
c) UNLOCK TABLES;
You can also use MySQL Enterprise Backup, which is the “official” solution for hot backups of InnoDB tables, it will perform
warm backup for other storage engines.
[mysql@localhost Desktop]$ mysqldump -u pythian – p $pw --extended-insert=true --add-drop-database --
databases rene | gzip > /export/full_dump.gz
[root@localhost mysqlbackup ]# ./mysqlbackup --user=root -p --backup-dir=/home/admin/backups backup-and-
apply-log
...
140904 12:34:54 mysqlbackup: INFO: MEB logfile created at /home/admin/backups/meta/MEB_2014-09-04.12-34-
54_backup_apply_log.log
--------------------------------------------------------------------
Server Repository Options:
--------------------------------------------------------------------
datadir = /var/lib/mysql/
innodb_data_home_dir =
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /var/lib/mysql/
innodb_log_files_in_group = 2
innodb_log_file_size = 50331648
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /var/lib/mysql/
innodb_undo_tablespaces = 0
innodb_undo_logs = 128
…
innodb_undo_directory = /home/admin/backups/datadir
innodb_undo_tablespaces = 0
innodb_undo_logs = 128
mysqlbackup: INFO: Unique generated backup id for this is 14098484948398421
mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
140904 12:34:57 mysqlbackup: INFO: Full Backup operation starts with following threads
140904 12:34:57 mysqlbackup: INFO: Starting to copy all innodb files...
…
140904 12:35:14 mysqlbackup: INFO: Apply-log operation completed successfully.
140904 12:35:14 mysqlbackup: INFO: Full backup prepared for recovery successfully.
mysqlbackup completed OK!

Database
White Paper
MYSQL REPLICATION
MySQL supports many replication types. The way MySQL replication works is keeping track of all updates in log files (index
& binlog) on the Master (with log-bin enabled). The MySQL slave receives these binlog dumps and builds relay logs via I/O
thread. SQL thread applies all changes to the MySQL slave.
MASTER - MASTER (CIRCULAR)
Master - Master replication means MySQL servers (mysqld) on different hosts or same host with different port number (port)
setup with unique (server-id) and binlog (log-bin) enabled
MASTER - SLAVE (SINGLE - MULTI)
One of the most common replication styles that can be setup is with one master and multiple slaves. Master - Slave replication
means MySQL servers (mysqld) on different hosts or the same host with different port number (port) setup with unique
(server-id) and binlog (log-bin) enabled on Master.
MASTER - SLAVE - SLAVE (A-> B-> C->)
This type of MySQL replication is similar setup as master-slave replication and called chain replication. The only difference is
host B in the chain should have log_slave_updates (log-slave-updates) enabled on top of binlog (log-bin) variables. A (Master)
- > B (Master-Slave) -> C (Slave)

Database
White Paper
REFERENCES
a) MySQL 5.6 Manual
http://dev.mysql.com/doc/refman/5.6/en/
b) MySQL Architecture
https://thinkingmonster.wordpress.com/database/mysql/mysql-architecture/
c) A fast and furious guide to MySQL database engines
http://www.techrepublic.com/article/a-fast-and-furious-guide-to-mysql-database-engines/
d) MySQL Storage Engines
http://www.w3resource.com/mysql/mysql-storage-engines.php
e) Using EXPLAIN to Write Better MySQL Queries
http://www.sitepoint.com/using-explain-to-write-better-mysql-queries/
f) MySQL Enterprise Backup User's Guide
http://dev.mysql.com/doc/mysql-enterprise-backup/3.11/en/intro.html
g) Getting Started with MySQL - Ligaya Turmelle (Presentation)
h) Pythian Internal Documents

My First 100 days with a MySQL DBMS (WP)

More Related Content

What's hot (20)

Similar to My First 100 days with a MySQL DBMS (WP) (20)

Recently uploaded (20)

My First 100 days with a MySQL DBMS (WP)