Mysql NDB Cluster's Asynchronous Parallel Design for High Performance

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster
Bernd Ocklin
MySQL Cluster Development
Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
Asynchronous Parallel Database Design for High Performance

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract. It
is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Copyright 2015, oracle and/or its affiliates. All rights reserved 2

Bernd Ocklin
Snr Director
MySQL Cluster Engineering

Today
MySQL Cluster Overview
MySQL Cluster 8.0
Data Distribution
Asynchronous Model
Cluster Virtual Machine

MySQL Cluster
Proven in serving billions of people every day when making phone calls,
playing online games or handling financial transactions.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 6
Massive linear scale
Always-On 99.9999% Availability
Distributed In-Memory Datasets
Always Consistent
Parallel Real-Time Performance.
Auto-partitioning, data distribution
and replication built-in.
Read- and Write Scale-Out
to many TB on commodity hardware.
Designed for mission critical
systems. Masterless, shared-nothing
with no single point of failure.
Transactional consistency across
distributed and partitioned dataset.
Out of the box straightforward
application programming.
Ease of use
Open Source
Written in C++. Can be used standalone
or with MySQL as a SQL front-end.

MySQL Cluster 8.0
- MySQL Server 5.7
- 5x faster restarts
- JSON Support
- 50% faster reads
- 40% faster read/write
MySQL 7.4 / 7.5
MySQL 7.6
- MySQL Server 5.7
- Redesigned for Terabyte clusters
- Redesigned for modern hardware
- 50% faster joins
- 100% faster scans
- 1000% faster restarts
MySQL 8.0
- Adopting data dictionary
- Hundreds of nodes
- 3 - 4 replicas
- Larger rows
Download and test 8.0.13 now
https://dev.mysql.com/downloads/cluster/
DMR

MySQL Cluster 7.6.7 - 2 x faster scans
• example: Sysbench RO
0
2250
4500
6750
9000
1 2 4 8 16 32 64 128 256 384 512
RO 7.6.8 7.6.7 7.5.11 7.4.21

MySQL Cluster 7.6.7 - 10 x faster restarts
• example: restart with 600 DBT2 warehouses - 280M rows
7.5.11 7.6.6 7.6.7

MySQL Cluster Architecture
• Multiple data nodes form a cluster
• Shared nothing
• Data distributed to data nodes

• Auto-partitioning and
distribution
• No name-node or central
master
• Each dataset is split into
fragments and distributed
across data nodes
• Within a cluster data is always
transactionally consistent
Data distribution User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
1773467253 photos xxx
8235602099 reminders xxx
8437829249 location xxx
MySQL Cluster Data Nodes
Partition Key

• Dataset distribution to virtual
partitions
• Hash on partition key as
default
• No re-hashing for data re-
organisation
Data distribution User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
Partition Key

Data distribution
• Cluster uses thousands of virtual partitions
• Distributed to data nodes and within data nodes

Data distribution
• These virtual partitions are distributed to data nodes

Data distribution

Global consistent view of distributed data
User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
Partition Key

Across one or many parallel MySQL Servers and API nodes
Global consistent view of distributed data

Data distribution awareness
• Key-value with hash on primary key
• Complemented by ordered in-memory-
optimised T-Tree indexes for fast searches
• Cluster always knows where its data is - without a name node

Parallel cross-partition queries
•Parallel execution on
the data nodes and
within data nodes
• Utilizes up to 64 cores
efficiently
• Parallelizes all work
even on single queries
• Batches automatically
• Event driven and
asynchronous
• Always consistent access to the entire distributed and partitioned dataset

Just mentioning - not topic of these slides
Cluster and High Availability
Copy of 1
Copy of 1
• Multiple copies of data are
maintained for availability
• A group of data nodes shares the
same data
• 1 - 4 replicas/copies of data can
be configured

Replication locally and between physical locations
• With awareness of data locality and availability domains for cloud
Cluster
Data Nodes
Replication

Reading and writing data

Asynchronous design for reads and writes
• Tasks organized in working
blocks
• Signals for communication
between blocks
• Job buffers for each
working block
• Executed in cluster VM
• Cluster virtual machine kernel
Working  
„block“
Job  
queue
Signals 
(Events)
• Transaction  
Coordination
• Data Manager
• …

Single asynchronous simple read
• A single transaction is coordinated by one Transaction Coordinator
• One Data Manager per partition
Transaction
Coordinator (TC)
Data
Managers (DM)
Async Signals

Single asynchronous simple read - distributed
• Signals are send between blocks inside and between data nodes
Transaction
Coordinator (TC)
Data
Manager (DM)
Data Node Data Node

Multiple asynchronous simple reads
Transaction
Coordinator (TC)
Data
Managers (DM)

Single asynchronous table scan
Transaction
Coordinator (TC)
Data
Managers (DM)
• Scanning multiple partitions in parallel

In a bit more detail
LQH ACC TUP TUX API TC
Scan request
Stored procedure request
Next scan request
prepareTupleKeyRequest
Next scan confirmation
Tuple key request
TransAI

Cluster virtual machine

Cost of CPU context switches - around 3 - 5us
Context
switch

• Multiple Threads per data node
• Each thread handles some blocks
• Parallel execution of
• … multiple queries from …
• … multiple users on …
• … multiple MySQL Servers
• Communication with signals
• Goal: minimize context
switching
Virtual machine model
NIC
Main thread
Data manager
Receive
Send
Disk/SSD IO

• Data is partitioned inside the
data nodes
• Communication asynchronously,
event driven on cluster’s VM
• No group communication -
instead using
• Distributed row locks
• Non-blocking 2-phase commit
Lock free and multi core VM
NIC
Main thread
Data manager
Receive
Send
Disk/SSD IO

Concurrent requests in multithreaded Virtual Machine
• Out of order execution possible
Transaction
Coordinator (TC)
Data
Managers (DM)

Multithreaded Virtual Machine
• Each partition can be managed in its own thread via Data Manager
• There can be separate threads for disks, sending and receiving over
network as well
• Threads can be distributed and locked to CPU cores for real time
Data manager threads
Data nodes with  
data partitions

Cross partition joins
•Parallel execution
across data nodes and
within data nodes
•Cluster queries
distributed data as if
it was a single
consolidated database
• Joins are pushed
down to data nodes
• Result consolidation
in MySQL Server
• Parallel joins with MySQL Server query engine

Massive parallel system executing parallel queries
Receive Send
Transaction Data Manager
Data Node
Receive Send
Transaction Data Manager
Data Node

Some conclusions
• Asynchronous event driven model
• Very hard to establish (distributed, out of order execution, …)
• Once established it is easy to extend (create new blocks, signals)
• Very simple to split, scale and multithread as blocks already communicate
asynchronously
• Allows to parallelize even single queries over multiple machines and cores
• Network is the bottleneck
• Virtual machine model helps to avoid CPU overhead
• Context switches
• Memory

Mysql NDB Cluster's Asynchronous Parallel Design for High Performance

More Related Content

What's hot (20)

Similar to Mysql NDB Cluster's Asynchronous Parallel Design for High Performance (20)

Recently uploaded (20)

Mysql NDB Cluster's Asynchronous Parallel Design for High Performance