What to expect from MariaDB Platform X5, part 1

What to expect from MariaDB Platform X5, part 1

• 1.
  What to expectfrom MariaDB Platform X5 Part 1 Max Mether VP, Product Management MariaDB Corporation 1
• 2.
  Company Vision Autonomous ● Self-learning ●QoS guarantee Adaptive Scale ● Performance ● Reliable ● Scalable to petabytes Cloud Native • Easy consumption • Agile and portable • Hybrid
• 3.
  MariaDB Platform X5 MariaDBPlatform is our Flagship product and consists of ● MariaDB Enterprise Server, also including ○ ColumnStore ○ Xpand ● MariaDB MaxScale ● MariaDB Connectors 3
• 4.
  MariaDB Platform X5 MariaDBPlatform is our Flagship product and consists of ● MariaDB Enterprise Server, also including ○ ColumnStore ○ Xpand ● MariaDB MaxScale ● MariaDB Connectors 4
• 5.
  MariaDB Connectors 5
• 6.
  MariaDB Connectors Production 6 ● MariaDBConnector/C ○ API for C/C++ based applications ● MariaDB Connector/ODBC ○ Accessing Data via standard ODBC ● MariaDB Connector/J ○ Platform independent Java driver ● MariaDB Connector/Node.js ○ Javascript driver ● MariaDB Connector/Python ● MariaDB Connector/C++ ● MariaDB Connector/R2DBC Development
• 7.
  MariaDB Connector/Python ● PythonWrapper to MariaDB Connector/C 3.1 ● LGPL licensed ● Available on PyPi.org ● Supports Python 3.6 Connecting MariaDB and Python 7
• 8.
  MariaDB Enterprise Server 8
• 9.
  MariaDB Enterprise Server 9 ●Yearly cadence for new versions ● Features: ○ Maintenance releases on predictable schedule ○ Focus on robustness, stability and predictability ○ Enterprise specific features ○ Joint efforts with key customers
• 10.
  Enterprise Server 10.5 10
• 11.
  Adaptability ● ColumnStore 1.5as a Storage Engine ○ Columnar distributed storage ○ Improved compatibility features ○ Improved management ● Xpand, a new storage Engine ○ Built in HA ○ Read & write scaling ○ Automatic sharding & elasticity ● Binaries now using MariaDB as the name
• 12.
  Xpand Architecture 12 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node A minimum of three Xpand nodes is needed for HA
• 13.
  Xpand Architecture 13 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node Each Xpand table will be divided into slices that are split across the Xpand nodes
• 14.
  Xpand Architecture 14 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node Each Xpand table will be divided into slices that are split across the Xpand nodes
• 15.
  Xpand Architecture 15 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node Each Xpand table will be divided into slices that are split across the Xpand nodes
• 16.
  Xpand Architecture 16 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node Each Xpand table will be divided into slices that are split across the Xpand nodes
• 17.
  Xpand Architecture 17 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node Each Xpand table will be divided into slices that are split across the Xpand nodes The slices are then “replicated” across the different nodes using internal methods
• 18.
  Xpand Architecture 18 MariaDB instancewith Xpand engine MariaDB instance with Xpand engine MariaDB instance with Xpand engine MariaDB Connector / Application Xpand node Xpand node Xpand node Server nodeServer node Server node
• 19.
  Enhanced SQL ● INSERTand REPLACE support the extension RETURNING ○ Resultset of any expression based on the added value ○ Resultset Specified SELECT expression. ● EXCEPT ALL and INTERSECT ALL ○ Duplicates delivered with the result set. ● COMMENT for CREATE and ALTER DATABASE ● Aggregate functions JSON_ARRAYAGG and JSON_OBJECTAGG ● RENAME INDEX and ALTER TABLE … RENAME COLUMN ● CYCLE has been added to recursive Common Table Expressions (CTE) ● Full support of REFERENCES constraint in column definition ● ALTER … TABLE IF EXISTS / IF NOT EXISTS
• 20.
  The Data TypePlugin API ● New Plugin API to create custom data types via plugins ● New Data type as example implementation ○ INET/4, INET6 ● More data types can be added later based on customer feedback
• 21.
  Enhancements to TemporalTables ● WITHOUT OVERLAP for application-time period tables ● Configurable start date/time for interval partitioned history of system versioned tables ○ PARTITION BY SYSTEM_TIME INTERVAL 1 YEAR STARTS '2018-10-01 00:00:00' ● Automatic creation of history partitions when partitions are defined ○ CREATE … WITH SYSTEM VERSIONING PARTITION BY SYSTEM_TIME PARTITIONS 2;
• 22.
  Security ● New privilegesas subset of the SUPER privilege ○ Currently the SUPER privilege is used in many cases were only a subset is needed ● The new privileges are: ○ BINLOG ADMIN (administration of the Binary Log) ○ CONNECTION ADMIN (administering connection resource limit options) ○ FEDERATED ADMIN (CREATE SERVER, ALTER SERVER, and DROP SERVER) ○ READ_ONLY ADMIN (set read_only system variable; allow user to perform writes) ○ REPLICATION MASTER ADMIN (statements like SHOW SLAVE HOSTS) ○ REPLICATION SLAVE ADMIN (administering Replica Servers) ○ SET USER (setting DEFINER when creating Triggers, Views, Functions) ● SUPER still acts as an alias for all of the above where needed
• 23.
  Security ● New systemvariable “require_secure_transport” ○ Server forces TLS encrypted connections from clients ● Via the performance schema threads table it can be determined which connections are using SSL/TLS ○ Threads table includes column CONNECTION_TYPE ● Encryption Plugin for HashiCorp Vault ○ HashiCorp Vault used as KMS ○ Remote key store for MariaDB Enterprise Server Data-at-Rest
• 24.
  InnoDB Refactoring The StorageEngine InnoDB has undergone a huge refactoring and cleanup. ● Optimized tablespace handling ● New InnoDB thread pool for background tasks ○ Online change of the number of innodb purge threads ● Redo log changed to a more efficient format ○ Online resizing of the redo log ● InnoDB internal foreign key parser removed ○ Preparation for foreign key handling by the server ● Deprecated InnoDB system and status variables removed
• 25.
  InnoDB Refactoring ● Cleanupof InnoDB Data Scrubbing code ○ Related status variables have been removed ● Several variables from SHOW ENGINE INNODB STATUS added to SHOW GLOBAL STATUS ○ This simplifies monitoring of these parameters
• 26.
  Replication and BinaryLog ● SQL statements for controlling the replication now allow the use of the term REPLICA instead of SLAVE. ● Default mode for parallel replication changed from “conservative” to “optimistic” ○ The optimistic mode of in-order parallel replication is replaying transactions in parallel, ○ Preserving exact transaction semantics ○ Replica applier handles conflicts ■ The latter transaction is rolled back and applied again, keeping the transaction order ● Replica aware server shutdown ○ Primary only shuts down when replication transactions have been processed ○ New system variable to control default behavior
• 27.
  System Information andTracing ● Information schema ○ Includes internals from the generic thread pool ○ New column GLOBAL_VALUE_PATH in table SYSTEM_VARIABLES ■ Path to the file that the value of the variable was read from ● ANALYZE for statements shows the time spent for checking the WHERE clause ● Performance Schema has been refactored ○ New tables for memory, stored routines, prepared statements, transactions, connections, and more
• 28.
  MariaDB Cluster (Galera) ●Enforcement to allow DDL for engines supported by Galera ○ wsrep_strict_ddl=ON; ● Cluster now has full MariaDB GTID support ● Inconsistency voting protocol ○ An inconsistency occurrence can be solved by removing a node instead of shutting down the whole cluster ● New Non-blocking DDL replication mode ○ Avoids full cluster lock and aborted transactions ● “Galera Black Box” for troubleshooting ● gcache size default configuration increased to 1 GB
• 29.
  S3 & Aria ●Increase Aria and S3 index length limit from 1000 to 2000 bytes ● Replication Support for S3 tables ○ Shared S3 storage ○ S3 storage per Node ● Improved partitioning support for S3 tables 29
• 30.
  Others ● Spider ○ ODBCdata source support for Spider (gamma maturity) ● Information about pre-10.0 temporal data types ○ always displayed with a /* mariadb-5.3 */ comment in: ■ SHOW CREATE TABLE ■ DESCRIBE ■ INFORMATION_SCHEMA.COLUMNS.COLUMN_TYPE
• 31.
  Enterprise Server Backports 31
• 32.
  MariaDB Enterprise ServerBackports MariaDB ES 10.4 32 ● Encryption Plugin for HashiCorp Vault ● Replication aware server shutdown ● Increase Aria and S3 index length limit from 1000 to 2000 bytes ● S3 replication and partitioning enhancements ● Encryption Plugin for HashiCorp Vault ● Replication aware server shutdown ● Increase Aria and S3 index length limit from 1000 to 2000 bytes MariaDB ES 10.3 ● Encryption Plugin for HashiCorp Vault MariaDB ES 10.2
• 33.
  Thank You 33