Contents
Executive Summary ...................................................................................................................................... 3
Why Micron for this Solution ................................................................................................................... 3
Ceph Distributed Architecture Overview ....................................................................................................... 4
Reference Architecture Overview ................................................................................................................. 6
Software .................................................................................................................................................. 6
Ceph Luminous 12.2.8....................................................................................................................... 6
Red Hat Enterprise Linux 7.5 ............................................................................................................ 6
Software by Node Type .......................................................................................................................... 7
Hardware ................................................................................................................................................ 8
Ceph Storage Node ........................................................................................................................... 8
Ceph Monitor Node ............................................................................................................................ 9
Micron 9200 MAX NVMe SSDs ......................................................................................................... 9
Network Switches .............................................................................................................................. 9
Mellanox ConnectX
®
-5 EN Dual Port NICs ..................................................................................... 10
Planning Considerations ............................................................................................................................. 11
Number of Ceph Storage Nodes .......................................................................................................... 11
Number of Ceph Monitor Nodes ........................................................................................................... 11
Replication Factor ................................................................................................................................. 11
CPU Sizing ........................................................................................................................................... 11
Ceph Configuration Tuning ................................................................................................................... 11
Networking ............................................................................................................................................ 11
Number of OSDs per Drive ................................................................................................................... 12
OS Tuning/NUMA ................................................................................................................................. 13
Measuring Performance .............................................................................................................................. 14
4KB Random Workloads: FIO + RBD .................................................................................................. 14
4MB Object Workloads: RADOS Bench ............................................................................................... 14
Baseline Test Methodology ......................................................................................................................... 15
Storage Baseline Results ..................................................................................................................... 15
Network Baseline Results ..................................................................................................................... 15
Test Results and Analysis ........................................................................................................................... 15
4KB Random Workload Testing ........................................................................................................... 16
4KB Random Write Workload Analysis ........................................................................................... 16
4KB Random Read Workload Analysis ........................................................................................... 18
Random Read Results Summary .................................................................................................... 20
4KB Random 70% Read / 30% Write Workload Analysis ............................................................... 20
Random 70/30 R/W Results Summary ............................................................................................ 22
4MB Object Workloads ......................................................................................................................... 22
Summary ..................................................................................................................................................... 24
Appendix A: Configuration Details .............................................................................................................. 25
About Micron ............................................................................................................................................... 31
Why Community Edition? ............................................................................................................................ 31
About Ceph Storage .................................................................................................................................... 31
