Challenges in hybrid and federated cloud computing
4 likes2,817 views
The document discusses challenges and architectures in hybrid and federated cloud computing, highlighting the importance of interoperability and resource sharing between different cloud environments. It addresses various models of cloud deployment (private, public, hybrid) and the functionalities of tools like OpenNebula for managing cloud infrastructures. Future trends include enhanced scalability, reliability, and performance benefits through cloud federation and innovative architectures.
Challenges in hybrid and federated cloud computing
- 1. Challenges in Hybrid and Federated Cloud Computing Ignacio M. Llorente DSA-Research.org Distributed Systems Architecture Research Group Universidad Complutense de Madrid © OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License 1/33
- 2. Contents Challenges in Hybrid and Federated Cloud Computing! ● What is Cloud Computing? ● What is OpenNebula? ● The Future of Cloud Computing ● Levels of Coupling ● Common Architectures for Federation ● Challenges for Interoperability ● Cloud Federation in Grid Infrastructures Challenges in Hybrid and Federated Cloud Computing 2/33
- 3. What is Cloud Computing? Provision of IT Capabilities as a Service! What Who On-demand End-user Software as a Service access to any (does not care about hw or sw) application Platform for Developer Platform as a Service building and (no managing of the underlying hw delivering web & swlayers) applications Infrastructure as a Raw computer System Administrator ﾺ Service infrastructure (complete management of the computer infrastructure) Physical Infrastructure Challenges in Hybrid and Federated Cloud Computing 3/33
- 4. What is Cloud Computing? Provision of Virtualized Resources as a Service! • Raw infrastructure • Management Console resources • Simple REST API’s Manage Instances • Pay-as-you-go & elastic capacity Challenges in Hybrid and Federated Cloud Computing 4/33
- 5. What is Cloud Computing? Different Models of Deployment! Model Definition Cloud Cases Infrastructure is owned by • Optimize and simplify internal operation Private a single organization and • SaaS/PaaS support made available only to the • IT consolidation within large organizations organization (Goverment Clouds, University Clouds…) Infrastructure is owned by • Low cost solutions with limited control/ a single organization and configuration and security/reliability good made available to other enough Public organizations over the • Commercial cloud providers, mostly Internet hosting providers • Science public clouds by ICT service centers to enable scientific and educational projects to experiment with cloud computing Infrastructure is owned by • Premium solutions with additional control/ Private Virtual a single organization and configuration and security/reliability made available to other • Telecom cloud providers organization over a dedicated private network Challenges in Hybrid and Federated Cloud Computing 5/33
- 6. What is OpenNebula? Cloud as an Evolution of the Data Center! Constraints from Requirements from Existing Infrastructure Usage and Deployment and Processes Scenarios “One solution does not fit all requirements and constraints” OpenNebula makes cloud an evolution by leveraging existing IT assets, protecting your existing investments, and avoiding vendor lock-in Challenges in Hybrid and Federated Cloud Computing 6/33
- 7. What is OpenNebula? IaaS Cloud Computing Tool for Managing a Data Center's Virtual Infrastructure! Most Advanced and Flexible, Enterprise-grade IaaS Cloud Manager Adaptable ● Customizable and Extensible Proven ● Many Massive Scale Production Deployments Powerful ● Most Advanced Enterprise-class Functionality No Lock-in ● Platform Independent and Interoperable Interoperable ● Most popular cloud APIs and standard based Openness ● Fully open-source ● Apache license Challenges in Hybrid and Federated Cloud Computing 7/33
- 8. What is OpenNebula? The Internals of the Cloud! Interfaces & API • CLI (local/remote) • API (java, ruby bindings) • Sunstone • Cloud (EC2, OCCI) VM Networking • VLANs Image Repository of VM disks • Firewall hooks • ACLs (public + private) • User defined • OS and Data types (persistent) • Multiple storage backends User & Roles Hosts: Cluster workernodes to run VMs. Storage • Multiple hypervisors defined per host • NAS • Grouped in logical clusters • SAN • Custom monitoring probes and technologies • Custom (bittorrent, ssh…) Challenges in Hybrid and Federated Cloud Computing 8/33
- 9. What is OpenNebula? EU Success Story in Cloud Computing Research and Innovation! Flagship EU Project on Deployments with Cloud Computing Research 4,000 tens of thousands downloads/month of VMs Mature Wordwide engaged technology community started in 2005 Innovative Technology Spin-off Management Tool Innovation Platform Challenges in Hybrid and Federated Cloud Computing 9/33
- 10. The Future of Cloud Computing Next Step in the Evolution of an Utility! Utility Utility Generation Distribution Utility Grid Challenges in Hybrid and Federated Cloud Computing 10/33
- 11. The Future of Cloud Computing Benefits of Federation! Scalability ● Cloudbursting to address peak demands ● Sharing of infrastructure between partners ● Infrastructure aggregation across sites Reliability ● Fault tolerance architectures across sites Performance ● Deployment of services closer to end users Cost ● Dynamic placement to reduce the overall infrastructure cost Energy Consumption ● Minimize energy consumption Challenges in Hybrid and Federated Cloud Computing 11/33
- 12. The Future of Cloud Computing The Cluster Computing Use Case! Virtual Computing Cluster Flexibility • Dynamic elasticity rules Cluster Users • Dynamic provision of cluster configurations Workernodes Completely Transparent • LRMS interface Front-end Simplify and Optimize Internal Operations • Dynamic performance partitioning • Consolidation • Faster upgrades Challenges in Hybrid and Federated Cloud Computing • Maintenance 12/33
- 13. The Future of Cloud Computing The Cluster Computing Use Case! Scalability Performance Cost Fault tolerance Challenges in Hybrid and Federated Cloud Computing 13/33
- 14. Levels of Coupling Different Levels of Control, Monitoring, Cross-site Functionality and Security! Private Cloud Partner Cloud Company Cloud Public Cloud Challenges in Hybrid and Federated Cloud Computing 14/33
- 15. Levels of Coupling Loosely Coupled Federation! Federation with a Cloud without Interoperation Support • Basic operations over VMs (start, shutdown, restart…) Control • Different instance types Monitoring & • Basic virtual resource monitoring (resource consumption…) Accounting Cross-site • None Security • Single account representing the organization Challenges in Hybrid and Federated Cloud Computing 15/33
- 16. Levels of Coupling Partially Coupled Federation! Federation with a Cloud with Partial Interoperation Support • Advanced operations over VMs (live migration…) Control • VM location and affinity constraints Monitoring & • Advanced virtual resource monitoring (energy consumption, Accounting VM placement…) • Virtual networks Cross-site • Virtual storage Security • Framework agreement Challenges in Hybrid and Federated Cloud Computing 16/33
- 17. Levels of Coupling Tightly Coupled Federation! Federation with a Cloud with Advanced Interoperation Support • Placement on specific physical resources Control • Same instance types Monitoring & • Physical resource consumption Accounting • Live migration Cross-site • High availability Security • User space sharing Challenges in Hybrid and Federated Cloud Computing 17/33
- 18. Common Architectures for Cloud Federation Organization of Multi-site Cloud Environments! Challenges in Hybrid and Federated Cloud Computing 18/33
- 19. Common Architectures for Cloud Federation Cloudbursting Architecture! Corporate cloud Cloud OS Public Public Cloud 1 Cloud 2 Local data center Aim • Meet peak demands • Public cloud by commercial provider Cloud Type • VPC by telecom provider Coupling • Loosely and partially coupled Challenges in Hybrid and Federated Cloud Computing 19/33
- 20. Common Architectures for Cloud Federation Cloudbursting Architecture! Enhancing Grid Infrastructures with Cloud Computing EU grant agreement RI-261552 Simplify and optimize its use and operation, providing a more (2010-2012) e-Infrastructure flexible, dynamic environment for scientists; and enhance existing computing infrastructures with “IaaS” paradigms www.StratusLab.eu www.stratuslab.org Challenges in Hybrid and Federated Cloud Computing 20/33
- 21. Common Architectures for Cloud Federation Dynamic Combination of Local with Remote Cloud Resources! Service Level Agreement • SLO 1: Average_CPU_Utilization > 75% • Action: Execute remote WN Grid Services StratusLab StratusLab Challenges in Hybrid and Federated Cloud Computing 21/33
- 22. Common Architectures for Cloud Federation Cloud Broker Architecture! Cloud OS (Broker) Public Public Public Cloud 1 Cloud 2 Cloud 3 Aim • Cost, performance and reliability optimization Cloud Type • Public clouds by commercial provider Coupling • Loosely coupled Challenges in Hybrid and Federated Cloud Computing 22/33
- 23. Common Architectures for Cloud Federation Cloud Broker Architecture! Building Service Testbeds on FIRE Agreement 257386 (2010-2013) Design, build and operate a multi-site cloud-based facility to New Infrastructure Paradigms support research across applications, services and systems and Experimental Facilities targeting services research community on Future Internet www.BonFIRE-Project.eu Source: BonFIRE Project Challenges in Hybrid and Federated Cloud Computing 23/33
- 24. Common Architectures for Cloud Federation Aggregated Cloud Architecture! Corporate Corporate cloud 1 cloud 2 Cloud OS Cloud OS Local data center Local data center Aim • Sharing of resources to meet peak demands Cloud Type • Partner or company clouds Coupling • Partially or tightly coupled Challenges in Hybrid and Federated Cloud Computing 24/33
- 25. Common Architectures for Cloud Federation Aggregated Cloud Architecture! Resources and Services Virtualization without Barriers Agreement 215605 (2008-2011) Open source technology to enable deployment and Service and Sw Architectures and management of complex IT services across different Infrastructures administrative domains www.reservoir-fp7.eu Challenges in Hybrid and Federated Cloud Computing 25/33
- 26. Common Architectures for Cloud Federation Multi-tier Cloud Architecture! Corporate cloud Cloud OS Cloud Cloud site 1 site 2 Cloud OS Cloud OS Local data center Local data center • Very large scale or geographically distributed Aim data centers Cloud Type • Company clouds Coupling • Tightly coupled Challenges in Hybrid and Federated Cloud Computing 26/33
- 27. Common Architectures for Cloud Federation Multi-tier Cloud Architecture! CloudFront • Portal • Cloud API (EC2, OCCI) User • Global AuthN OpenNebula Zone OpenNebula Zone • Multi-tenancy • Multi-tier arch. • Runs Instances • Scalable • Stores Images • Isolation • Networking • Multiple-site • Authorization support Challenges in Hybrid and Federated Cloud Computing 27/33
- 28. Challenges for Interoperability Leveraging Existing Standards and Implementing Interoperation! Standardization ● Implement standards ● Integrate with standards Which Standard? ● Different de jure standards ● Several de facto standards Interoperation ● Implement adaptors ● Use transformers Challenges in Hybrid and Federated Cloud Computing 28/33
- 29. Challenges for Interoperability I Want to Combine My Private Cloud with a Remote Cloud without Changes! Transparent Combination of Local Resources with Cloud Resources 1. Management Interfaces 2. Management Interfaces for Virtual Workloads for Data Elements CDMI VM VM VM VM VM 4. Contextualization OVF 3. VM Image Description 5. Cross-site Networking and Instance Type and Latency OVF Challenges in Hybrid and Federated Cloud Computing 29/33
- 30. Cloud Federation in Grid Infrastructures Grid and Cloud as Complementary Computing Models! Usage Grids Usage Clouds § Job Processing § Raw infrastructure § Big Batch System § Elasticity & Pay-per-use § File Sharing Services § Simple Web Interface Achievements Achievements § Federation of Resources § Agile Infrastructures § VO Concept § IT is another Utility But… But… § User experience § Interoperability § Complexity § Federation Resource Sharing Resource Management Scientific Applications Customize Environments Uniform Security Flexibility & Simplicity Challenges in Hybrid and Federated Cloud Computing 30/33
- 31. Cloud Federation in Grid Infrastructures The StratusLab Vision! • Batch Job Processing • Custom Execution Environments 2 • Industry Applications • Other WMS (pilots) Access • Grid Service Integration • Complete Services (cluster) Grid Middleware IaaS Interface LRMS (LSF, PBS…) StratusLab Toolkit Service 1 Virtual CE, WN… Other (web, mail...) Raw machines Provision 3 Grid Sites Challenges in Hybrid and Federated Cloud Computing External Providers 31/33
- 32. Cloud Federation in Grid Infrastructures The StratusLab Architecture! Grid Services MarketPlace • Federation facilities • Security • Sharing existing VM images • Grid specific services • Registry of metadata • Image are kept elsewhere • Supports trust Appliance Repo • Storage VM images • Distributed • Multi-protocol Grid Services Cloud API Grid Services Cloud API Cloud/Grid Site Cloud/Grid Site Challenges in Hybrid and Federated Cloud Computing 32/33
- 33. Questions? We Will Be Happy to Answer Any Question ! CloudPlan.org @imllorente Challenges in Hybrid and Federated Cloud Computing 33/33