Red Hat Storage Day LA - Persistent Storage for Linux Containers
2 likes•528 views
This document discusses persistent storage options for Linux containers and how Red Hat Storage addresses the storage needs of containerized applications. It begins by explaining how containers package applications and dependencies for portability and ease of management. Typical workloads for containers often require persistent storage. The document then evaluates options like NFS, GlusterFS, Ceph RBD, iSCSI/FC and public cloud storage, noting that Red Hat Storage provides scalable, resilient, flexible software-defined storage. It presents Red Hat Storage and OpenShift Enterprise as a solution that allows applications and storage to run together on servers for improved utilization and costs. The document closes with two customer case studies demonstrating how Red Hat Storage supports containerized workloads at CapitalOne and Ver
Red Hat Storage Day LA - Persistent Storage for Linux Containers
- 1. Persistent Storage for Linux Containers Louis Gordon Senior Storage Solutions Architect, Red Hat
- 2. Linux Containers in brief A Software packaging concept that typically includes an application and all of its runtime dependencies • Higher Quality Software Releases • Shorter Test Cycles • Easier Application Management HOST OS SERVER CONTAINER LIBS APP CONTAINER LIBS APP Benefits
- 3. Greater Portability, Automation, and Integration Code Registry Push image Code & Build Test Deploy Pull image
- 4. The Red Hat Stack – From PaaS to Storage DevOps Tools and User Experience Language Runtimes and Middleware Databases and Other Services Container Orchestration and Management Container API Storage Container Host
- 5. Typical workloads for Containers Base: 194 IT operations and development decision-makers at enterprises in APAC, EMEA, and North America Source: A commissioned study conducted by Forrester Consulting on behalf of Red Hat, January 2015 “For which workloads or application use cases have you used/do you anticipate to use containers?” Scalable, Cost Effective, Distributed Storage for Containers
- 6. Why Containers require persistent Storage? • Web & presentation layers should be stateless ... but ... • That’s probably < 5% of all application instances in a DC • Most other infrastructure applications / services require persistence for storing application and configuration data • Building a different infrastructure for stateful vs. stateless apps ? • Modern IT requires Standardization (e.g. Cloud Computing) • Software-defined DC is a hybrid cloud enabler
- 7. The requirements for Container Storage? • Scalable – Scale out capacity & performance • Resilient – Likely there will be important data • Flexible – Allow different application access profiles • Software-defined – To allow flexible deployment models • Open – For customer choice and lowest TCO RED HAT STORAGE➔
- 8. Current Options for Persistent Storage Shared Filesystems: • NFS • GlusterFS Block Storage: • Ceph RBD • iSCSI or FC • GCE Persistent Disk • AWS Elastic Block Store
- 9. Storage Options Comparison NFS-Filer GlusterFS Ceph RBD iSCSI / FC GCE PD AWS EBS Scalability - + ++ - ++ ++ Availability + ++ ++ + + + Cost - ++ + ++ - - Deploymen t Flexibility - ++ ++ - - - Data Privacy ++ ++ ++ ++ -- -- Sharing Data ++ ++ +/- - - - Use Cases Generic Data Sharing Generic Data Sharing Very Large Deployments, Object Store High Performance Block Cheap Block Storage Cheap Block Storage
- 10. STORAGE APPLIANCE COMPUTE NODE + + + + + STORAGE NODE COMPUTE NODE + + + + + + + + + + + + + + + + Scale-Out Storage • Increased Performance & Capacity • Avoid Fork-Lift Upgrades • Built-In Redundancy
- 11. Why Bother? PROPRIETARY HARDWARE HARDWARE-BASED INTELLIGENCE SCALE-UP ARCHITECTURE CLOSED DEVELOPMENT PROCESS Common, off-the-shelf hardware Lower cost, standardized supply chain Scale-out architecture Increased operational flexibility Software-based intelligence More programmability, agility, and control Open development process More flexible, well-integrated technology
- 12. Persistent Storage requires Redundancy • Red Hat Storage includes important data redundancy features • 2-way and 3-way replication • Erasure Coding • Geo-Replication • Snapshots
- 13. Red Hat Storage and Containers
- 14. Node 1 NGINX Container Node 2 NGINX Container Persistent Storage for OSE/RHEL Atomic Node 1 NGINX Container Node 3 MySQL Container Fileshare Fileshare BlockDevice/ ObjectStore Atomic Host Atomic HostOPENSHIFTOPENSHIFT Gluster Ceph Gluster
- 15. Host OS Container Container Host OS Container Container /share (Gluster/NFS) Container Volume mapping options Gluster Volume Brick 1 Brick 2 Brick 3 Brick n /share (Gluster/NFS) /share/sub1 ➔ /data /share/sub2 ➔ /data
- 16. Host OS Container Container Host OS Container Container /shared-volume (Gluster-Fuse or NFS) Sharing amongst hosts /shared-volume /shared-volume /shared-volume /shared-volume Gluster Volume Brick 1 Brick 2 Brick 3 Brick n
- 17. Host OS Container Container Host OS Container Container Mapping Ceph RBDs to containers /shared ➔ /data Ceph OSD OSD OSD OSD CephRBD+ FS CephRBD+FS /shared Ceph RBD + FS /host-share ➔ /data /local-mnt /shared ➔ /data /host-share
- 18. App Container App Container App Container App Container Containerized Gluster Storage /shared-volume /shared-volume Gluster Container (privileged) /bricks/brick2 Gluster Container (privileged) /bricks/brick1 /shared-volume (Gluster-Fuse or NFS) Host OS Host OS Gluster Volume /shared-volume /shared-volume
- 19. Hyperconverged Containers and Storage • Lower TCO • Unified Orchestration • Ease of Use • Greater control ContainerJBoss NGINX NGINX Container MASTER Kubernetes Node 2 ContainerElastic Spark MongoDB Container Kubernetes Node 3 RedisPostgreSQL Container Kubernetes Node 1 Kubernetes Node 4
- 20. Container Container Container Container Containerized Ceph Storage /local-mnt1 Container RHCS Container RHCS Ceph RBDs (and Object Storage) Host OS Host OS Ceph Pool /local-mnt2 /local-mnt3 /local-mnt4
- 21. Flexible Deployment Options x86 x86 x86 Physical VM VM VM Virtualized VM VM VM Public / Private Cloud OpenStack, AWS, Azure, GCE VMware, RHEV, KVM Container JBoss NGINX NGINX Container MASTER Kubernetes Node 2 Container Elastic Spark MongoDB Container Kubernetes Node 3 Redis PostgreSQL Container Kubernetes Node 1 Kubernetes Node 4 Container JBoss NGINX NGINX Container MASTER Kubernetes Node 2 Container Elastic Spark MongoDB Container Kubernetes Node 3 Redis PostgreSQL Container Kubernetes Node 1 Kubernetes Node 4 Container JBoss NGINX NGINX Container MASTER Kubernetes Node 2 Container Elastic Spark MongoDB Container Kubernetes Node 3 Redis PostgreSQL Container Kubernetes Node 1 Kubernetes Node 4
- 23. Customer Case Study: CapitalOne Business Challenge: • A leading diversified bank with 65 million customers • Fast growing business and customer base • Need to be disruptive and different • Analytics plays a big role in growth strategy Solution Description: • Predefined docker images with a wide variety of analytics tools • Self-service Portal for developers to pick and instantiate • Integrated monitoring and metrics • Automated lifecycle management of containers • High availability through MESOS • Shared and consolidated Storage Platform with Gluster Solution Benefits: • More agile application development • Larger choice of technologies • Optimal resource usage and performance
- 24. Customer Case Study: CapitalOne
- 25. Customer Case Study: Verizon D o c k e r c o n t a i n e r s r u n n i n Business Challenge: • Verizon is the largest provider of cellular services in the United States with more than 100 million subscribers. Pretty much every subscriber takes ‘selfies’ and snaps of the kittens and these need to be backed up. Whilst many users use Apple’s iCloud, or Yahoo’s Flickr, Verizon also has their own branded Cloud offering – Verizon Cloud. • Verizon wished to use a file-based format rather than converting all these images to Objects and to have an extremely efficient architecture in terms of the use of their server infrastructure. Solution Description: • Simplified Deployment via containers • Seamless upgrade and rollback • Dedicated and Containerized Gluster for FSaaS • Dynamic scaling (up) of capacity as needed • Performance and health metrics collection via container monitoring agents and sending alerts. Solution Benefits: • Quicker time to market for new services • Increased subscriber ‘stickiness’ and improving customer satisfaction. • Reduced infrastructure Costs • Infrastructure can grow as demand grows
- 26. ETH/IB app app app app app app Converged Computing Architecture Mixed App and Storage Workloads App-only servers Converged servers app app + Storage-only servers app app app app app app app app app app app app app app app Storage stack imposes only 3% - 10% load on compute processing • Applications and storage stacks can co-exist on same compute substrate. • This achieves higher server utilization and lower operational costs across the cloud Customer Case Study: Verizon
- 27. Summary • Most containerized applications will require Persistent Storage • Software-defined Storage allows hyper-convergence for applications and storage • Red Hat Storage and OpenShift Enterprise provide a complete PaaS solution with full deployment flexibility from on-premise to hybrid clouds
- 28. Thank You