OUGLS 2016: Guided Tour On The MySQL Source Code
Download as PPTX, PDF0 likes327 views
The document provides an overview of the MySQL source code, highlighting critical components such as the directory layout, session establishment, termination processes, and query execution mechanics. It also discusses the functionality of plugins and user-defined functions (UDFs), detailing how they can be implemented and their respective advantages and disadvantages. Additionally, it emphasizes the importance of using plugin services to maintain compatibility across versions.
OUGLS 2016: Guided Tour On The MySQL Source Code
- 1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Guided tour on the MySQL Source Code Georgi Kodinov MySQL team lead Confidential – Oracle Internal/Restricted/Highly Restricted
- 2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Confidential – Oracle Internal/Restricted/Highly Restricted 2
- 3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Agenda Confidential – Oracle Internal/Restricted/Highly Restricted 3 The directory layout of the codebase Session establishment and termination Query execution Add functionality to MySQL
- 4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Important directories in the codebase Directory Purpose sql/ Most of the server code storage/ Storage engines plugin/ Non-storage-engine plugins sql-common/ Code common between client and server libmysql/ Client library code libmysqld/ Embedded server client library glue code mysql-test/ Integral tests libservices/ The plugin services library, plugin part mysys*/ Portability library, utilities vio/ Network connection abstraction library client/ Client tools Confidential – Oracle Internal/Restricted/Highly Restricted 4
- 5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Session establishment and termination • Each transport has its own listening thread: *_conn_event_handler() – Runs Connection_acceptor::connection_event_loop() • Communicates via a Protocol class sub-class. Protocol_classic for the native MySQL protocol. Work in progress ! • Each session has a THD created. The THD is the session state. The THD is in a OS thread attribute (current_thd()). There can be nested THDs. • Connection_handler_manager class drives the threading model Confidential – Oracle Internal/Restricted/Highly Restricted 5
- 6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Session establishment and termination contd. • handle_connection() is the main worker thread execution function • check_connection() is where authentication happens • close_connection() is the function to close the connection. Called by both the worker thread or KILL Confidential – Oracle Internal/Restricted/Highly Restricted 6
- 7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Query execution at a glance • do_command() fetches and executes a command (RPC) • dispatch_command() has the command (RPC) id main switch() • The THD has the current command being executed • mysql_parse() parses and executes a single query. – Called in a loop for the COM_QUERY command • mysql_execute_command() is the execution part of mysql_parse() – Used to be a switch() over the SQL command id and call different C functions – Now in the process of being transformed to calls to Sql_cmd::execute(). – Lots of sub-classes for Sql_cmd. Confidential – Oracle Internal/Restricted/Highly Restricted 7
- 8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | The parser: work in progress • In older releases the parser produced the execution plan directly – Item class subclasses • Started adding a separate parse stage data structure – Parse_tree_node subclasses • Extra pass – Sql_cmd *PT_Statement::make_cmd() – Still called by the parser Confidential – Oracle Internal/Restricted/Highly Restricted 8
- 9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Query execution passes: handle_query() et al • “Open” all tables: open_tables_for_query () • Preparation: SELECT_LEX::prepare() and friends – Name resolution: setup_fields() et al – Some transformations, like flattening subqueries etc • Optimization: SELECT_LEX::optimize() and friends – Produces a JOIN object for each subquery level – Calls JOIN::optimize() to produce query execution plan • Execution: JOIN::exec() or st_select_lex_unit::execute() – Pumps result into a Query_result sub-class. Confidential – Oracle Internal/Restricted/Highly Restricted 9
- 10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Adding functionality to MySQL: Plugins Confidential – Oracle Internal/Restricted/Highly Restricted 10
- 11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Plugins overview • A plugin is: – An API that the server calls – init() and deinit() – A bunch of system variables – A bunch of status variables • A plugin does: – Initialize and de-initialize itself – Implement the API that the server will call – Call back into the server via the plugin services Confidential – Oracle Internal/Restricted/Highly Restricted 11
- 12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Plugins: Pros and Cons Pros Cons Clean versioned API to implement Can only implement a fixed set of APIs defined by the server Can be loaded and unloaded dynamically Can only interact with the server. Not other plugins Can use services to call back into the server Acquiring and releasing plugin handles by the server is expensive Can be built with the binary distro, no need for the server source Very poor set of plugin services Are versioned and can be reused across versions Linked with the server binary: can call into the server code without services The tree cmake files detect “foreign” plugin directories and compile them APIs cannot be related so APIs tend to be large (see e.g. the SE API : 200+ methods) Confidential – Oracle Internal/Restricted/Highly Restricted 12
- 13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Most notable plugin types • Storage engines • INFORMATION_SCHEMA tables • Daemon • Authentication • Audit • Keyring Confidential – Oracle Internal/Restricted/Highly Restricted 13
- 14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | How to write a plugin ? • Take an existing one and alter it • Plenty of examples – Live plugins – Test plugins • Put it in plugin/ or storage/ so it compiles Confidential – Oracle Internal/Restricted/Highly Restricted 14
- 15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Adding functionality to MySQL Plugin Services Confidential – Oracle Internal/Restricted/Highly Restricted 15
- 16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What is a plugin service ? • The reverse of a plugin API • Allows plugins to call into the server • Version controlled • In essence a set function pointers in each plugin that the server fills in at INSTALL PLUGIN time • Header in include/mysql/service_*.h Confidential – Oracle Internal/Restricted/Highly Restricted 16
- 17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Please use the plugin services ! • Calling server code directly => a “dirty” plugin – No portability across versions – No guarantee to work with future versions • Plugin services are the reverse of Plugin APIs – Inlcude/mysql/service_*.h • We have some good ones in 5.7: – command_service_st : execute a RPC command – mysql_locking_service_st : interface to MDL locks – security_context_service_st : interface to the security context – srv_session_service_st : handle “sessions” inside the server Confidential – Oracle Internal/Restricted/Highly Restricted 17
- 18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Adding functionality to MySQL: UDFs Confidential – Oracle Internal/Restricted/Highly Restricted 18
- 19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What are the UDFs • Can reside in external binaries • Can be loaded and unloaded • Can share binaries with plugins or other UDFs • Executed as single result or aggregate SQL functions • Easy to write, lots of examples Confidential – Oracle Internal/Restricted/Highly Restricted 19
- 20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | UDFs: Pros and Cons Pros Cons Easy to write Terrible for the DBA to administer Can share a binary with the plugins No global initialization and deinitialization Can be version safe No access to plugin services Somewhat limited data type range Only SQL functions, not commands to execute No THD context Confidential – Oracle Internal/Restricted/Highly Restricted 20