Ceph BlueStore - новый тип хранилища в Ceph / Максим Воронцов, (Redsys)
1 like2,382 views
The document discusses Ceph's new storage technology, Bluestore, contrasting it with the older Filestore system. It details the features and advantages of Bluestore, including improved performance and scalability in software-defined storage solutions. Additionally, it provides insights into various storage systems and technologies while referencing the author's experience and professional background.
![HW test
$ iperf3 -c osd00
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 7.40 GBytes 6.35 Gbits/sec 3278 sender
[ 4] 0.00-10.00 sec 7.39 GBytes 6.35 Gbits/sec receiver
$ iperf3 -c osd00-ci
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec 64 sender
[ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec receiver](https://image.slidesharecdn.com/cephbluestore-cephredsys-160607200514/85/Ceph-BlueStore-Ceph-Redsys-34-320.jpg)
Ceph BlueStore - новый тип хранилища в Ceph / Максим Воронцов, (Redsys)
- 1. Ceph new store: BlueStore Максим Воронцов
- 2. About me ● Главный инженер по вычислительным комплексам ● Работаю с Linux 8 лет ● WAS/DB2/MQ вот это все ● Много разных проектов
- 3. About RedSys ● Бизнес интегратор ● Существует более 20 лет ● Офисы в MOW, LED, OVB, GOJ, ROV, KHV ● RED = Responsibility + Efficiency + Development ● Отрасли - ТЭК, ВПК, Госы, Телеком, etc.
- 4. Customers
- 5. TOC ● Before Ceph ● Ceph first advent ● Ceph temptations ● BlueStore prophecy ● Ceph FileStore vs BlueStore ● Let's fight ● Results ● Awaiting Ceph second advent
- 6. Software Defined Storage ● Unlimited scalability ● Storage virtualization ● Policy-driven administration ● API services ● Support for block, file and object data types
- 7. IBM definition «SDS in today's business context refers to IT storage that goes beyond typical array interfaces (for example, command line and graphic user) to operate within a higher architectural construct.»
- 8. Examples ● AWS S3 ● EMC ScaleIO ● Ceph ● GlusterFS ● Huawei FusionStorage ● IBM ElasticStorage ● NexentaStor
- 9. Issue ● DB2 on z/OS
- 10. Issue ● DB2 on z/OS ● XML in DB2
- 11. Issue ● DB2 on z/OS ● XML in DB2 ● Signed XML in DB2 (no way)
- 12. Issue ● DB2 on z/OS ● XML in DB2 ● Signed XML in DB2 (no way) ● You really shouldn't store blobs in relational store
- 13. To find a way ● More money to IBM?
- 14. To find a way ● More money to IBM? ● More money to someone else?
- 15. To find a way ● More money to IBM? ● More money to someone else? ● Something else?
- 16. Which one? ● AWS S3 ● Ceph ● IBM ElasticStorage ● Huawei OceanStor ● Swift
- 17. Why this one?
- 18. Standing on the shoulders of giants ● CERN ● Cisco ● Deutsche Telecom ● Yahoo ● Cloudmouse.ru ● ...
- 19. Preborn 7 guests in VMWare: ● 1 MON ● 3 OSD ● 1 ActiveMQ ● 1 Tomcat ● 1 ElasticStorage
- 20. Long story short Long long story about...
- 21. Long story short Long long story about… What is English for «импортозамещение»?
- 22. Long story short Long long story about… What is English for «импортозамещение»? Catch up and overtake z/OS
- 23. Long story short Long long story about… What is English for «импортозамещение»? Catch up and overtake z/OS What is Russian for LTFS?
- 24. Long story short Long long story about… What is English for «импортозамещение»? Catch up and overtake z/OS What is Russian for LTFS? What is Russian for WORM?
- 25. BlueStore prophecy Ceph Jewel Preview: a new store is coming, BlueStore
- 26. Ceph scheme
- 27. OSD scheme
- 28. FileStore scheme
- 29. BlueStore scheme
- 31. Mount directory structure $ ls -R /var/lib/ceph/osd/ceph-0 | wc -l
- 32. Mount directory structure $ ls -R /var/lib/ceph/osd/ceph-0 | wc -l FileStore BlueStore 18656 16
- 33. HW test $ sudo dd bs=1G count=1 oflag=direct if=/dev/zero of=zerofile 1+0 records in 1+0 records out 1073741824 bytes (1,1 GB) copied, 10,275 s, 105 MB/s
- 34. HW test $ iperf3 -c osd00 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 7.40 GBytes 6.35 Gbits/sec 3278 sender [ 4] 0.00-10.00 sec 7.39 GBytes 6.35 Gbits/sec receiver $ iperf3 -c osd00-ci - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec 64 sender [ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec receiver
- 35. Ceph tests $ ceph osd pool create radosbench 64 $ rados bench -p radosbench 300 write --no-cleanup $ rados bench -p radosbench 300 seq $ rados bench -p radosbench 300 rand $ rbd create fio_test --size 10G $ fio rbd.fio
- 36. Results
- 37. Results
- 38. Not so fast $ ceph-disk prepare --bluestore /dev/sdd /dev/sdb $ ls /dev/disk/by-partlabel/ -l osd-device-2-block -> ../../sdb2 osd-device-2-data -> ../../sdd1
- 39. Not so fast $ ceph-disk prepare --bluestore /dev/sdd /dev/sdb $ ls /dev/disk/by-partlabel/ -l ceph%20data -> ../../sdb1 ceph%20block -> ../../sdb2
- 40. Not so fast Here be dragons Tech preview CPU regression on too fast disks ;-) Did you do backup today?
- 41. Hot to reach me Mail + hangouts: [email protected] mail: [email protected] http://redsys.ru
- 42. ● https://www.redbooks.ibm.com/abstracts/redp5121.html ● http://www.sersc.org/journals/IJMUE/vol10_no11_2015/27.p df ● http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf ● https://ceph.com ● https://www.sebastien-han.fr/blog/ ● https://cds.cern.ch/record/2015206/files/CephScaleTestMa rch2015.pdf ● http://rocksdb.org