2012 CloudStack Design Camp in Taiwan--- CloudStack Overview-2
- 1. CloudStack Overview (Cont’d) Roxanne_Chang @ Tcloud compu4ng
- 2. Outline • Management Server Internals • Workflow of Management Server • Extending Plugins to CloudStack • High Availability • Scalability • CloudStack v.s. OpenStack v.s. Eucalyptus • CloudStack Par4cipa4on
- 3. Management Server Internals • Architecture • Workflow • High Availability • Scalability
- 4. Old Architecture EC2 CloudStack API Layer Access Control Virtual Machine Manager Console Proxy Manager Async Job Manager Template Manager Snapshot Manager Network Manager Storage Manager … Agent Manager XenServer KVM Resource SRX Resource NetScaler Other Resources F5 Resource Resource Resource 4
- 6. New Architecture – API Server Other UI Cloud Portal CLI Clients REST API Server Pluggable API Engine OAM&P API End User API EC2 API Other APIs Management Services & Authen4ca4on ACL Integra4on -‐ Resource management -‐ Accounts, Domains, and -‐ Configura4on Projects -‐ Addi4onal opera4ons added by -‐ ACL, limits checking third party Framework -‐ Job Queue -‐ Database Access Layer -‐ OSGi
- 7. New Architecture – Execu4on Server Execution Server Services API Kernel Plugins • Drives long running VM opera4ons • Storage Handling • Syncs between resources managed and • Network Handling DB • Deployment • Generates events planning • Hypervisor Handling Framework • Cluster Management • Job Management • Alert & Event Management • Database Access Layer • Messaging Layer
- 8. New Architecture – Resources Agent • Resources are carried in Hypervisor Resources service VMs to be in close network proximity to the Network Resources physical resources it Storage Resources manages Image & Template Resources • Easily scales to u4lize the most abundant resource in Snapshot Resources data center (CPU & RAM) • Communicates with Execu4on Server over message bus (JSON) • Can be replicated for fault tolerance
- 9. Comparison of two Approaches • Stats Collector – collects capacity sta4s4cs – Fires every five minutes to collect stats about host CPU and memory capacity – Smart server and dumb client model: • Resource only collects info and management server processes • VM Sync – Fires every minute – Peer to peer model: • Resource does a full sync on connec4on and delta syncs thereacer. Management server trusts on resource for correct informa4on.
- 10. Cloud Other UI CLI Clients Portal Management Server REST API OAM&P API End User API EC2 API Other APIs Pluggable Service API Engine Console Proxy ACL & Authen4ca4on Security Adapters Management -‐ Accounts, Domains, and Projects -‐ ACL, limits checking Account Management Connectors Template Access Services API Deployment Planning Plugin API Kernel HA -‐ Drives long running VM opera4ons Network Configura4ons Services API -‐ Syncs between resources managed Usage and DB Calcula4ons -‐ Generates events Network Elements Addi4onal Services Hypervisor Gurus Cluster Resource Job Alert & Event Database Management Management Management Management Access Event Bus Message Bus Hypervisor Network Storage Image Snapshot Resources Resources Resources Resources Resources
- 11. Workflow of Management Server
- 12. Workflow of Management Server Plugins cmd.execute() Plugins Cmds Plugins Async CS API API Job Services Servlet Queue API Mgr Kernel Responses Agent API (Commands) Agent Resources Manager Local Or Remote Hypervisor Network Na4ve Device APIs API MySQL
- 13. Balancing Incoming Requests • Each management server has two worker thread pools – Executor threads provided by tomcat – Job threads wai4ng on job queue • DB opera4ons – short in dura4on – executed by executor threads • Requests needing resources – ocen have long running dura4ons, – checked against ACL by the executor threads – queued and executed by job threads.
- 14. Interac4ons OVM Cluster Primary Storage vcenter Monitoring Primary CS API vSphere Cluster Storage End User UI Primary XS Cluster Storage Admin UI Clustered CloudStack XAPI Domain CS Admin & CloudStack CloudStack End-‐user API Primary Admin UI Management JSON KVM Cluster Storage Server NetConf Juniper SRX Cloud user Nitro API {API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console {ec2 API client } Proxy VM Proxy VM NFS MySQL Server {Proxied} SSH Sec. Storage NFS NFS Sec. Storage VM Ajax HTTPS VM Console Router VM HTTP (Template Download) Router VM HTTP (Template Copy) Router VM Cloud user HTTP (Swic)
- 15. Extends Plugins to CS
- 16. Plugins • Various ways to add more capability to CloudStack • Implements clearly defined interfaces • All opera4ons must be idempotent • Compiles only against the Plugin API module
- 17. Anatomy of a Plugin Rest API -‐ Op4onal. Required only if needs to expose configura4on API to admin. ServerResource -‐ Op4onal. Required if Plugin needs to be co-‐located with the resource -‐ Implements Plugin API transla4on layer to Implementa4on talk to resource -‐ Communicates with server component via JSON Data Access Layer
- 18. Anatomy of a Plugin • Can be two jars: – server component to be deployed on management server – an op4onal ServerResource component to be deployed co-‐located with the resource • Server component can implement mul4ple Plugin APIs to affect its feature – As an example, OVS plugin actually implements both NetworkGuru and NetworkElement • Can expose its own API through Pluggable Service so administrators can configure the plugin
- 19. Plugin Interfaces Available • NetworkGuru – Implements various network isola4on technologies and ip address technologies • NetworkElement – Facilitate network services on network elements to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc) • DeploymentPlanner – Different algorithms to place a VM and volumes. • Inves4gator – Ways to find out if a host is down or VM is down. • Fencer – Ways to fence off a VM if the state is unknown • UserAuthen4cator – Methods of authen4ca4ng a user • SecurityChecker – ACL access • HostAllocator – Provides different ways to allocate host • StoragePoolAllocator – Provides different ways to allocate volumes
- 21. High Availability • Service Offering contains a flag for whether HA should be supported for the VM • Does not use the na4ve HA capability of hypervisors • Uses adapters to fine tune HA process
- 22. Triggering High Availability VM HA are triggered via the following methods: • VM Sync detects out of band VM changes • Resource Management detects that a resource is unreachable and its state can not be determined. • VM start/stop has been sent to the resource but resource does not return • Details of how high availability is done is at hip://docs.cloudstack.org/CloudStack_Documenta4on/Design_Documents/CloudStack_High_Availability_-‐ _Developer's_Guide
- 23. Scalability
- 24. Current Status • 10k resources managed per management server node • Real produc4on deployment of tens of thousands of resources • Internal tes4ng with socware simulators up to 30k physical resources with 300k VMs managed by 4 management server nodes
- 25. CloudStack vs. OpenStack vs. Eucalyptus
- 26. CloudStack • Mainly wriien in Java • ASL2.0 license • Has more than 100 produc4on clouds (Around May, 2012) • Support private/hybrid/public cloud • Scale to 30K physical host in commercial environment • Support XenServer/Vsphere/KVM/OVM/Baremetal as hypervisor • Mul4ple geographically distributed datacenters management • Flexible and rich network func4onality • Easy installa4on and management • Amazon EC2 API compa4ble • Well documented • Ac4ve community
- 27. OpenStack • Mainly wriien in Python • ASL2.0 license • Support private/hybrid/public cloud • Immature for commercial usage • Support XenServer/Vsphere/KVM/Xen/Hyper-‐V as hypervisor • Network is single point of failure • Weak VPN support for enterprise hybrid cloud • All inter-‐module communica4on are based on Message Queue • Not well documented • A bit hard to install • Amazon EC2 API par4ally compa4ble
- 28. Eucalyptus (Open Source edi4on) • Mainly wriien in Java • GPLv3 license • Focus on private cloud • Support KVM/Xen as hypervisor • Fully compa4ble with Amazon EC2 • Fully compa4ble with Amazon S3 via Walrus • Both web UI and command line tools for cloud administra4on • Well documented • Difficult to gepng started
- 30. Par4cipate Incubator Projects • As a user • Who uses the socware • Ac4ve user communi4es • Get/Provide help • As a developer • Help to develop the socware • Known as contributor, Be a coder, documenter or answerer • The first step to be a commiier • As a commi8er • Write access to source repo • Vote/nominate developers (PMC) • As a Mentor • Informal: Familiar with open source/apache knowledge • Formal: Self-‐introduc4on, volunteer their services to [email protected] during development
- 31. Apache CloudStack – Gepng Started • Homepage hip://incubator.apache.org/cloudstack/ Apache Incubator Site hip://incubator.apache.org/projects/cloudstack.html • Tracker hips://issues.apache.org/jira/browse/CLOUDSTACK • Jenkins hip://jenkins.cloudstack.org/ • Git hips://git-‐wip-‐us.apache.org/repos/asf/incubator-‐cloudstack.git • wiki: hips://cwiki.apache.org/CLOUDSTACK/ • Freenode IRC: #cloudstack; #cloudstack-‐dev; #cloudstack-‐mee4ng
- 32. CloudStack Mailing Lists • cloudstack-‐[email protected] • For Users & Administrators • Subscrip4on: cloudstack-‐users-‐[email protected] • cloudstack-‐users-‐[email protected] • For Users & Administrators • Subscrip4on: cloudstack-‐users-‐cn-‐[email protected] • cloudstack-‐[email protected] • CloudStack developments discussion center • Subscrip4on: cloudstack-‐dev-‐[email protected] • cloudstack-‐[email protected] • Automa4c commit message from CloudStack’s SCM • Subscrip4on: cloudstack-‐commits-‐[email protected]
- 33. Thank you !