OSDC 2013 | Distributed Storage with GlusterFS by Dr. Udo Seidel
0 likes•25 views
The document presents an overview of distributed storage using GlusterFS, highlighting its history, architectural components, and operational features. Key points include the transition from block to object storage, the modular architecture of GlusterFS, and its applications as NAS and SAN replacements. The summary emphasizes the active community development and practical use cases for various storage needs.
OSDC 2013 | Distributed Storage with GlusterFS by Dr. Udo Seidel
- 1. OSDC 2013 1 Distributed Storage with GlusterFS Dr. Udo Seidel Linux-Strategy @ Amadeus
- 2. OSDC 2013 2 Agenda â—Ź Introduction â—Ź High level overview â—Ź Storage inside â—Ź Use cases â—Ź Summary
- 4. LinuxCon Europe 05-07/11 2012 4 Me ;-) â—Ź Teacher of mathematics & physics â—Ź PhD in experimental physics â—Ź Started with Linux in 1996 â—Ź Linux/UNIX trainer â—Ź Solution engineer in HPC and CAx environment â—Ź Head of the Linux Strategy team @Amadeus
- 5. OSDC 2013 5 Storage: History â—Ź Reviewing storage task responsibilities â—Ź Block allocation â—Ź Space management â—Ź Extension of SCSI standard â—Ź Object based storage â—Ź Meta-Data handling separated from data management
- 6. OSDC 2013 6 Object based storage â—Ź Storage objects quite general â—Ź Partition, file, ... â—Ź Unique identifier â—Ź OSD (Object based Storage Device) â—Ź Hardware -> original trigger â—Ź Software -> common implementation â—Ź Main component of distributed file systems
- 7. OSDC 2013 7 Distributed storage: Paradigm changes â—Ź Block -> Object â—Ź Central -> Distributed â—Ź Few -> Many â—Ź Big -> Small â—Ź Server <-> Storage
- 8. OSDC 2013 8 Distributed File Systems â—Ź 'Recent' attention on distributed storage â—Ź Cloud hype â—Ź Big Data â—Ź See also CEPH talk the other day
- 9. OSDC 2013 9 Distributed storage – Now what?!? ● Several implementations ● Different functions ● Support models ● Storage vendors initiatives ● Relation to Linux distributions Here and now ==> GlusterFS
- 10. OSDC 2013 10 High level overview
- 11. OSDC 2013 11 History â—Ź Gluster founded in 2005 â—Ź Gluster = GNU + cluster â—Ź Acquisition by Red Hat in 2011 â—Ź Community project â—Ź 3.2 in 2011 â—Ź 3.3 in 2012 â—Ź Commercial product: RedHat Storage Server
- 12. OSDC 2013 12 The Client â—Ź Native â—Ź 'speaks' GLUSTERFS â—Ź Not part of the Linux Kernel â—Ź FUSE-based â—Ź NFS â—Ź Normal NFS client stack â—Ź S3/Swift compatible â—Ź Proxy needed
- 13. OSDC 2013 13 The Server â—Ź Data â—Ź Bricks â—Ź Translators â—Ź Volumes -> exported/served to the client â—Ź Meta-Data â—Ź No dedicated instance â—Ź Distributed hashing approach
- 14. OSDC 2013 14 The picture
- 15. OSDC 2013 15 Storage inside
- 16. OSDC 2013 16 The Brick â—Ź Trust each other â—Ź Interconnect via TCP/IP or RDMA/Infiniband â—Ź Dedicated file systems on GlusterFS server â—Ź XFS recommended, EXT4 works too â—Ź Extended attributes a must â—Ź Two main processes/daemons â—Ź glusterd â—Ź glusterfsd
- 17. OSDC 2013 17 The Translator â—Ź One per purpose â—Ź Replication â—Ź POSIX â—Ź Quota â—Ź I/O behaviour â—Ź Chained -> brick graph â—Ź Technically: configuration
- 18. OSDC 2013 18 The Volume â—Ź Service unit â—Ź Layer of configuration â—Ź 'RAID level' -> distributed, replicated, striped, combinations â—Ź NFS â—Ź Cache â—Ź Permissions â—Ź ....
- 19. OSDC 2013 19 Meta Data â—Ź 2 kinds â—Ź More of local file system style â—Ź Related to distributed nature â—Ź Some stored in backend file system â—Ź Permissions â—Ź Time stamps â—Ź Distribution/replication â—Ź Some calculated on the fly â—Ź Brick location
- 20. OSDC 2013 20 Elastic Hash Algorithm â—Ź Based on file names â—Ź Name space divided â—Ź Full brick handled via relinking â—Ź Stored in extended attributes â—Ź Client needs to know topology
- 21. OSDC 2013 21 Self-Healing â—Ź On demand vs. Scheduled â—Ź File based â—Ź Based on extended attributes â—Ź Split-brain â—Ź Quorum function â—Ź Sometimes: manual intervention
- 22. OSDC 2013 22 Geo replication â—Ź Asynchronous â—Ź Based on rsync/ssh â—Ź Master-Slave â—Ź If needed: cascading â—Ź One way street â—Ź Clocks in sync!
- 23. OSDC 2013 23 From files to objects â—Ź Introduced with version 3.3 â—Ź Hard links with some hierarchy â—Ź Re-uses GFID (inode number) â—Ź UFO â—Ź Unified File and Object â—Ź Combination with RESTful API â—Ź S3 and swift compatible
- 24. OSDC 2013 24 Operations: Growth, shrinkage .. failures â—Ź A Must! â—Ź Easy â—Ź Rebalance! â—Ź Order of servers important
- 25. OSDC 2013 25 What else ...? â—Ź Encryption :-| â—Ź Compression :-( â—Ź Snapshots :-( â—Ź Hadoop connector :-) â—Ź Locking granularity :-| â—Ź File system statistics :-)
- 26. OSDC 2013 26 Use cases
- 27. OSDC 2013 27 NAS replacement â—Ź NFS as 1:1 â—Ź Server: GlusterFS â—Ź Client: NFS â—Ź NFS as such â—Ź Server: GlusterFS â—Ź Client: GlusterFS
- 28. OSDC 2013 28 SAN replacement â—Ź Not quite advanced (yet) â—Ź New translator needed â—Ź Development started â—Ź Presenting GlusterFS as block device â—Ź Additional items needed â—Ź Locking â—Ź ...
- 29. OSDC 2013 29 Storage back-end for KVM and Co â—Ź Stacked (indirect) â—Ź Not smart â—Ź Workable for main hypervisors â—Ź Direct â—Ź QEMU â—Ź libvirt â—Ź oVirt/RHEV
- 31. OSDC 2013 31 Take aways â—Ź Thin distributed file system layer â—Ź Modular architecture â—Ź Operationally ready â—Ź Still some surprises â—Ź Active development and community
- 32. OSDC 2013 32 References â—Ź http://www.gluster.org â—Ź http://www.sxc.hu (pictures)
- 33. OSDC 2013 33 Thank you!
- 34. OSDC 2013 34 Distributed Storage with GlusterFS Dr. Udo Seidel Linux-Strategy @ Amadeus