OpenStack Day Italy: openATTC as an open storage platform for OpenStack
1 like•2,114 views
The document discusses building a flexible software-defined storage (SDS) platform for OpenStack, focusing on open-source solutions like OpenAttic that optimize storage management and reduce vendor lock-in. It highlights the importance of SDS in enabling the software-defined datacenter (SDDC) by providing cost-effective and scalable storage solutions capable of managing massive data growth. The overall message promotes the transition to open-source storage infrastructure, emphasizing its advantages over traditional proprietary systems.
OpenStack Day Italy: openATTC as an open storage platform for OpenStack
- 1. www.it-novum.com © it-novum GmbH Seite 1 Building a flexible and large-scale software- defined storage platform for OpenStack OPENSTACK DAY Italy May 30th 2014, Milan Michael Kienle, CEO it-novum GmbH [email protected]
- 2. www.it-novum.com © it-novum GmbH Seite 2 Source: RED HAT Open Clouds are picking up right now!
- 3. www.it-novum.com © it-novum GmbH Seite 3 Open Stack: Open cloud operating system supported by the OpenStack Foundation backed up by ### well-known companies Want to learn more: listen to the other tracks Major Design Guidelines are OPEN: Open source (apache), Open design (blueprints), Open development (branches), Open community (tech leads) Although OpenStack is purposely open it partly relies on underlying proprietary storage products. To build your cloud upon a truly OPEN software-defined storage platform (SDS) you might achieve greater flexibility and $$$$- savings. openATTIC is a SDS cloud storage platform optimized for OpenStack's Cinder component. What is this all about?
- 4. www.it-novum.com © it-novum GmbH Seite 4 Introduction & Agenda OpenStack Focus on Storage within OpenStack, esp. Cinder Requirements & Optimization of current approaches SDS – software-defined storage How it works SDS as part of the software-defined datacenter (SDDC) Why now? Advantages openATTIC How to bring SDS and OpenStack together Video/Examples Agenda
- 5. www.it-novum.com © it-novum GmbH Seite 5 Leading provider of open source business solutions 75 employees currently; approx. 20% growth p.a. Headquarters in Fulda/DE; Branch office in Vienna/AT Hundreds of mid-sized and large corporate clients Operating public listed parent company‘s data center for 20+ years and migrated most of this to open source Mission: Help customers to build and run a 100% open-source software- defined datacenter Proven Expertise in Business Open Source Infrastructure and Applications SAP Extended Business Partner (for integrating SAP into open source solutions) Open Business Solutions. Delivered!
- 6. www.it-novum.com © it-novum GmbH Seite 6 Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM,VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an „old virtualized landscape“ Enable the software-defined datacenter (SDDC) OpenStack: Value Proposition
- 7. www.it-novum.com © it-novum GmbH Seite 7 Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, common APIs Manage Virtualization across different Hypervisors (KVM,VMware, Xen, Hyper-V etc.) Support the whole IaaS stack (Storage, Security, Networking etc.) Scale far beyond the limitations of an „old virtualized landscape“ Enable the software-defined datacenter (SDDC) OpenStack: Value Proposition
- 8. www.it-novum.com © it-novum GmbH Seite 8 Storage costs optimization ($$$$) Data growth of >>40% p.a. needs to be managed automatically and way cheaper Flexibility and reduced vendor lock-in (open) Incumbent players are perfect in securing their huge margins Using innovative technology without renewing existing assets Cloud Computing has characterized the next generation IT delivery model, easy orchestration/automation of (storage) services is essential need scale-up/scale-out of underlying storage at lesser costs Software-Defined DataCenter (SDDC) becomes a major disruptive IT trend (SDS beeing part of this) Standardisation enables automation & abstraction of software from hardware -> Commodity hardware and SDS gained strength Customers‘ Perspective on Storage
- 9. www.it-novum.com © it-novum GmbH Seite 9 Cinder / Block Storage Swift / Object Storage Objectives • Storage for running VM disk volumes on a host • Ideal for performance sensititve apps • Ideal for cost effective, scale-out storage • Backup & Archive • Dropbox-like services Use Cases • Production Applications • Traditional IT Systems • Database-driven Apps • Dev-/Test Landscape • ISO Image- and VM- Templates • Snapshots • Backup & Archive • Repository Workloads • High chance content • Smaller, Random R/W • Higher / Bursty IO • More static content • Larger, sequential r/w • Lower IOPS Storage in OpenStack
- 10. www.it-novum.com © it-novum GmbH Seite 10 Architecture to provide block-level resources to OpenStack („block storage as a service“) Persistent block-level storage volumes for use with OpenStack Nova compute instances (e.g. iSCSI target) Manages creation, attach/detach, clone of these storage volumens Intention of Cinder: keep you from beeing locked-in: Ability to change the backend (the storage system) easily Migrate data & expect the same base level of functionality Many vendor/storage plug-ins for Cinder available; the choice between functionality and $$$$ Cinder - Details Legend Presistent volume control Presistent volume data VM instance Note that iSCSI is just an example – several additional protocols are supported (e.g., FC, NFS) Nova /dev/vda iSCSI initiator Cinder KVM iSCSI target
- 11. www.it-novum.com © it-novum GmbH Seite 11 Limitations of Commodity Storage (like a simple Linux server or comparable) because of lacking features like: node redundancy & clustering & HA, scaling QoS, Deduplication/Compression, Replication Example: Image Cloning (e.g. image from Glance) could be faster than block-based copy&paste… Limitations of Enterprise Storage (the big names): Vendor lock-in $$$$$ Consider Open Source & SDS ! Choose your underlying storage system!
- 12. www.it-novum.com © it-novum GmbH Seite 12 SDS is the abstraction of the software from the hardware. Benefits: Independence from big-name commercial developer of proprietary systems (bundle of hardware and storage os) for better pricing and freedom of choice Hardware is treated as a pool, scalable across multiple storage systems Innovative functionality (e.g. top-down policy driven), easier to administrate, faster to implement (software can be extended and maintained more easily than hardware) Part of the Software-defined DataCenter (SDDC) By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner,“Tech Go-to-Market: Storage Providers Should Focus on Three Growth Opportunities Through 2016”, Nov 2013), by 2018 OpenSource Storage will gain 20% of the market , approx. 10$B (Gartner,“2014 Strategic Roadmap for Storage”, March 2014). Software-defined Storage (SDS)
- 13. www.it-novum.com © it-novum GmbH Seite 13 Configuration Management System (CMS) Service Transition Service OperationService Design Monitoring Event Correlation User Other Monitors Capacity Management Risk Management Cloud Self Service System Application End-2-End Process Change Management Problem Management SAP ApplicationsService Desk Network Operation Center Release Management Cloud Management Incident Management Infrastructure Software-defined Storage Virtualization CMDB Discovery ReportingServiceLevelManagement The Big Picture: SDDC with 100% open sourceThe Big Picture: SDDC with 100% open source
- 14. www.it-novum.com © it-novum GmbH Seite 14 ITSM i-doit & OTRS Umbrella Monitoring openITCOCKPIT Cloud Management OpenStack Reporting Pentaho Billing, Backup,... SAP, SEP sesam Configuration Deployment Services & Tools API Web CLI Apps Status Checking Application Management Access Samba, NFS, AFP, iSCSI, FC Availability DRBD, Pacemaker, Corosync, ... Efficiency ZFS, LVM, BTFRS, XFS Backup Snapshotting, Tape- Backup, ... Analysis openITCOCKPIT, Nagios, ... Back end Core Front end openATTIC Framework - Architecture Interfaces
- 15. www.it-novum.com © it-novum GmbH Seite 15 Mirroring Block- & File-based Synchronous & asynchronous Object-based Clustering Automated Failover Multi Node Support Support & Services Training & Optimization Integration & Development Guaranteed Response Time Guaranteed Restore Time (Appliance only) Protection Integration Tape-Systems Snapshotting Consistent Snapshotting Efficiency Deduplication Compression Performance Config-Set Config-Set for Virtualization Unified and versatile SAN / Block Storage (FC, iSCSI, FCoE) NAS / File-based (CIFS, NFS) Object Storage Central Configuration Web Frontend Command Line Interface Dynamic Monitoring Availability Performance Capacity Cloud Support OpenStack/Cinder openQRM Umbrella Management Master/Slave Architecture Current FeaturesOpen API, Scalable, Cinder-ready
- 16. www.it-novum.com © it-novum GmbH Seite 16 Current Features Benchmark Example: Read/write performance using ZFS Pool
- 17. www.it-novum.com © it-novum GmbH Seite 17 All access protocols, block (FC,iSCSI, FCoE) and files storage systems (CIFS, NFS) in every single openATTIC system In its latest definition Unified Storage contains object storage as well (Ceph, Swift). Object storage architecture manages data as objects and allows inexpensive, scalable and self-healing retention of massive amounts of unstructured Data openATTIC can use Ceph for Object Storage and enables the following Extending storage capacity and performance seamlessly on demand (multi-node scalibility) No upfront investment costs for eventual higher requirements in the future Scale out capacity and performance without migrate the existing infrastructure Unified Storage Expample: Scale out with Object Storage
- 18. www.it-novum.com © it-novum GmbH Seite 18 More Cloud Support/Integration: openstack Swift, CloudStack, Eucalyptus… QoS for top-down orchestration of services and policy enforcement Business-aware/application-aware SnapApps SnapLock: Permanent and compliant long-term storage for meeting regulatory requirements Flash/SSD Support for Appliances Better Security Hardening and Performance Optimization for Appliances Better support for easy scale-up/scale-out Deeper Integration of Big Data solutions: Hadoop, Spark Eye-Candy GUI front-end Better documentation and more video tutorials …. Product Roadmap
- 19. www.it-novum.com © it-novum GmbH Seite 19 Volume create/delete Snapshot create/delete Volume attach/detach (called by Nova) Create Volume from Snapshot Copy Image to Volume / Volume to Image Get Volume Stats, Clone Volume Extend Volume (mandatory since Icehouse) openATTIC Cinder plug-in:Video We show you some examples how this works… How it works: Cinder API (Icehouse)
- 20. www.it-novum.com © it-novum GmbH Seite 20 Unit tests Cinder driver certification file Release hopefully next one… More Info: www.openattic.org/en Integration in other cloud platforms has been done already OpenStack Cinder / openAttic: Status
- 21. www.it-novum.com © it-novum GmbH Seite 21 Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openATTIC functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openATTIC runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs Are there limitations of software-defined storage / software-defined datacenter? Summary: openATTIC within OpenStack
- 22. www.it-novum.com © it-novum GmbH Seite 22 Open and comprehensive API achivies great flexibility for integration into different cloud platforms Optimization of huge data through intelligent openATTIC functionalities Consistent snapshots relieve backup, accelerate desaster recovery and reduce downtimes Deduplication, Compression, efficient mirroring and redundancy Huge cost savings: openATTIC runs on commodity hardware reducing vendor lock-in Open Source approach saves license costs However, the bad news is: even software-defined datacenters still need some underlying hardware… Summary: openATTIC within OpenStack
- 23. www.it-novum.com © it-novum GmbH Seite 23 Thank you! Questions? - More than welcome! Cloud Storage Intelligent SDS Unified Storage Power your cloudPower your cloud Rock-solid storageRock-solid storage Scale out data centerScale out data center Power your cloud & scale out your data center with rock-solid Software- Defined Storage (SDS) truly on open source, join the community & more about the openATTIC project at http://openattic.org/en Power your cloud & scale out your data center with rock-solid Software- Defined Storage (SDS) truly on open source, join the community & more about the openATTIC project at http://openattic.org/en