Virtualization in the Cloud @ Build a Cloud Day SFO May 2012
Download as PPTX, PDF•2 likes•753 views
Virtualization in the Cloud was designed for cloud computing from the outset. Xen was initially a university research project that provided isolation between virtual machines (VMs) and has since become widely used in cloud computing. The Xen Cloud Platform (XCP) provides a complete virtualization stack and management API called XenAPI that allows integration with cloud orchestration platforms like OpenStack. XCP packages Xen, XenAPI, and associated components into Linux distributions for flexibility. XCP provides enterprise-ready virtualization with high performance, security, and scalability for cloud computing.
Virtualization in the Cloud @ Build a Cloud Day SFO May 2012
- 1. Virtualization in the Cloud Lars Kurth Xen Community Manager [email protected] @lars_kurth @xen_com_mgr
- 2. A Brief History of Xen in the Cloud Late 90s XenoServer Project (Cambridge Univ.) Global Public Computing The XenoServer project is building public infrastructure for wide-area “This dissertation proposes a new distributed computing distributed computing. paradigm, termed global public computing, which allows any user to run any code anywhere. Such platforms price We envisage a world in which XenoServer computing resources, and ultimately charge users for execution platforms will be scattered across resources consumed.“ the globe and available for any member of the public to submit code for execution. Evangelos Kotsovinos, PhD dissertation, 2004
- 3. A Brief History of Xen in the Cloud Late 90s Oct ‘03 ‘06 ‘08 ‘09 ‘11 ‘12 XenoServer Project Amazon EC2 XCP 1.x (Cambridge Univ.) and Slicehost Cloud Mgmt launched Xen Presented Rackspace at SOSP Cloud XCP packages in Linux XCP Announced
- 4. The Xen Hypervisor was designed for the Cloud straight from the outset!
- 5. Xen.org • Guardian of Xen Hypervisor and related OSS Projects • Xen project Governance similar to Linux Kernel • Projects – Xen Hypervisor (led by Citrix) – Xen Cloud Platform aka XCP (led by Citrix) – Xen ARM : Xen for mobile devices (led by Samsung) – PVOPS : Xen components and support in Linux Kernel (led by Oracle) • 10+ vendors contributing more than 1% to the project (AMD, Citrix, Fujitsu, Huawei, Intel, Novell, Oracle, Samsung, Suse, …)
- 6. Xen Overview
- 7. Basic Xen Concepts Control Domain aka Dom0 Console • Dom0 kernel with drivers • Xen Management Toolstack VMn • Trusted Computing Base Control domain VM1 (dom0) Guest Domains One or more VM0 Toolstack driver, stub or • Your apps Dom0 Kernel service domains Guest OS and Apps • E.g. your cloud management stack Driver/Stub/Service Domain(s) Scheduler, MMU Xen Hypervisor • A “driver, device model or control Host HW service in a box” I/O Memory CPUs • De-privileged and isolated • Lifetime: start, stop, kill 7
- 8. Xen Variants for Server & Cloud Xen Hypervisor XCP Toolstack / Console Default / XL (XM) Libvirt / VIRSH XAPI / XE Increased level of functionality and integration with other components Get Binaries from … Linux Distros Linux Distros Debian & Ubuntu XCP from Xen.org Products Oracle VM Huawei UVP Citrix XenServer Many Used by … Others 8
- 9. Xen : Types of Virtualization
- 10. PV Domains & Driver Domains Control domain Guest VMn Driver Domain Linux PV guests have limitations: (dom0) e.g. • limited set of virtual hardware Apps • Disk • Network Advantages PV Back Ends PV Front Ends PV Back End • Fast • Works on any system HW Drivers HW Driver (even without virt extensions) Guest OS Dom0 Kernel* Driver Domains Xen Hypervisor • Security • Isolation I/O Memory CPUs Host HW • Reliability and Robustness *) Can be MiniOS 10
- 11. HVM & Stub Domains Dom0 Guest VMn Stubdomn Guest VMn Disadvantages • Slower than PV due to Emulation (mainly I/O devices) IO Emulation IO Emulation Device Model Device Model Advantages • Install the same way as native Linux IO Event Stub Domains IO Event VMEXIT Mini OS VMEXIT • Security Xen Hypervisor • Isolation • Reliability and Robustness Host HW 11
- 12. PV on HVM • A mixture of PV and HVM • Linux enables as many PV interfaces HVM PV on PV as possible HVM Boot Sequence Emulated Emulated PV • This has advantages Memory HW HW PV – install the same way as native Interrupts, Emulated PV* PV – PC-like hardware Timers & – access to fast PV devices Spinlocks – exploit nested paging Disk & Network Emulated PV PV – Good performance trade-offs Privileged HW HW PV Operations • Drivers in Linux 3.x *) Emulated for Windows
- 13. Xen and Linux
- 14. Xen and the Linux Kernel Xen was initially a University research project Invasive changes to the kernel to run Linux as a PV guest and Dom0
- 15. Current State PVOPS Project Xen support in Linux 3.0+ (it is functional but not yet fully optimized) On-going optimization work in Linux 3.4 +
- 16. What does this mean? • Xen Hypervisor is not in the Linux kernel • BUT: everything Xen needs to run is! • Xen packages are (or will be) in Linux distros – Install Dom0 Linux distro – Install Xen package(s) or meta package – Reboot – Config stuff: set up disks, peripherals, etc.
- 17. “Security and QoS/Reliability are amongst the top 3 blockers for cloud adoption” www.colt.net/cio-research
- 18. Security and the Next Wave of Virtualization • Security is a key requirement for Cloud • Security is the primary goal of virtualization on the Client – Desktop, Laptops, Tablets & Smart Phones • Maintaining isolation between VMs is critical
- 19. Xen Security & Robustness Advantages • Even without Advanced Security Features – Well-defined trusted computing base (much smaller than on type-2 HV) – No extra services in hypervisor layer • More Robustness: Mature, Tried & Tested, Architecture • Xen Security Modules (or XSM) – Developed and contributed to Xen by NSA – Generalized Security Framework for Xen • Can also run with SELinux 19
- 20. Advanced Security: Disaggregation • Split Control Domain into Driver, Stub and Service Domains – Each contains a specific set of control logic – See: ”Breaking up is hard to do” @ Xen Papers • Unique benefit of the Xen architecture – Security: Minimum privilege; Narrow interfaces – Robustness: ability to safely restart parts of the system – Performance: lightweight, e.g. Mini OS directly on hypervisor – Scalability: more distributed system (less reliable on Dom0) • Used by Qubes OS and Citrix XenClient XT
- 21. Advanced XenClient Architecture Per host/device Per guest Service VMs Service VMs VPN Isolation Control Domain Management VPN Isolation User VM User VM Emulate Emulation Device Network Isolation Domain (dom0) Device Policy Granularity Policy Granularity Xen Hypervisor Xen Security Modules I/O Memory CPUs Host HW
- 22. Example: Network Driver Domain for HA • Detect failure e.g. 350 – Illegal access 300 – Timeout 250 • Kill domain, restart 200 150 – E.g. Just 275ms outage from 100 failed Ethernet driver 50 • Auto-restarts to 0 0 5 10 15 20 25 30 35 40 enhance security time (s)
- 23. BUT… • Today, XCP and commercial Xen based Server products – Do not yet make use of XSM – Do not yet make use of Advanced Security Features (Disaggregation) • In XCP, work has started to add these features – Various articles of how this may be done on the xen wiki – More information soon (likely at XenSummit) – Commitment on improving docs for Security, Reliability & Tuning
- 24. XCP Project
- 25. XCP Complete vertical stack for server virtualization Distributed as a closed appliance (ISO) with CentOS 5.5 Dom0, misc DomU’s, network & storage support and Xen API Open source distribution of Citrix XenServer
- 26. XCP Overview • Open source version of Citrix XenServer wiki.xen.org/wiki/XCP/XenServer_Feature_Matrix • Enterprise-ready server virtualization and cloud platform Extends Xen beyond one physical machine and other functionality Lots of other additional functionality compared to Xen • Datacenter and cloud-ready management API XenAPI (XAPI) is fully open source CloudStack and OpenStack integration
- 27. Major XCP Features • VM lifecycle: live snapshots, checkpoint, migration • Resource pools: flexible storage and networking • Event tracking: progress, notification • Upgrade and patching capabilities • Real-time performance monitoring and alerting • Built-in support and templates for Windows and Linux guests • Open vSwitch support built-in
- 28. XCP 1.5 (in beta) • Architectural Improvements: Xen 4.1, GPT, smaller Dom0 • GPU pass through: for VMs serving high end graphics • Performance and Scalability: – 1 TB mem/host – 16 VCPUs/VM, 128 GB/VM • Networking: Open vSwitch (default), Active-Backup NIC Bonding • Virtual Appliance: multi-VM and boot sequenced, OVF support • More guest OS templates
- 29. XAPI: What is it? • XAPI is the backbone of XCP – Provides the glue between all components – Is the backend for all management applications – Also called XenAPI • It's a XML-RPC style API, served via HTTPS – Provided by a service on every XCP dom0 host – Designed to by highly programmable – API bindings for many languages: .NET, Java, C, Powershell, Python • XAPI is Extensible via plugins – E.g. used by OpenStack
- 30. XCP-XAPI Packages in Linux • Make the XAPI toolstack independent of CentOS 5.5 • Extend the delivery model – Deliver Xen, XAPI and everything in between (storage manager, network support, OCaml libs, etc.) via your favorite Linux distro “apt-get install xcp-xapi” or “yum install xcp-xapi” • Debian 7.0 “Wheezy" • Ubuntu 12.04 LTS • Next: Fedora & CentOS
- 31. XAPI Management Options • XAPI frontend command line tool: XE (tab-completable) • Desktop GUIs o Citrix XenCenter (Windows-only) o OpenXenManager (open source cross-platform XenCenter clone) • Web interfaces o Xen VNC Proxy (XVP) o XenWebManager (web-basedclone of OpenXenManager) • XCP Ecosystem: o xen.org/community/vendors/XCPProjectsPage.html o xen.org/community/vendors/XCPProductsPage.html
- 32. XCP and Cloud Orchestration Stacks
- 34. • Designed for the Cloud : many advantages for cloud use! – Resilience, Robustness & Scalability – Security: Small surface of attack, Isolation & Advanced Security Features • Widely used by Cloud Providers and Vendors • XCP & XAPI – Ready for use with cloud orchestration stacks – XCP-XAPI packages in Linux distros: flexibility and choice – Lots of additional improvements for cloud coming in 2012 • Open Source with a large community and eco-system
- 35. • IRC: ##xen @ FREENODE • Mailing List: xen-users & xen-api • Wiki: wiki.xen.org – Beginners & User Categories – XCP Category • Excellent XCP Tutorials – A day worth of material @ xen.org/community/xenday11 • Ecosystem pages Questions … Slides available under CC-BY-SA 3.0
Editor's Notes
- #3: XenoServer : enablers as well the concept
- #4: Note: 10th birthday of the project is coming up
- #5: Hold this thought! We will come back to this later….!
- #7: PVOPS is the Kernel Infrastructure to run a PV Hypervisor on top of Linux
- #8: Dom 0:In a typical Xen set-up Dom0 contains a smorgasboard of functionality:System bootDevice emulation & multiplexingAdministrative toolstackDrivers (e.g. Storage & Network)Etc.LARGE TCB – BUT, Smaller as in a Type 2 hypervisorDriver/Stub/Service Domains: also known as Disaggregation
- #10: PVOPS is the Kernel Infrastructure to run a PV Hypervisor on top of Linux
- #12: Device Model emulated in QEMUModels for newer devices are much faster, but for now PV is even faster
- #13: Automatic PerformancePV on HVM guests are very close to PV guests in benchmarks that favour PV MMUsPV on HVM guests are far ahead of PV guests in benchmarks that favour nested paging
- #14: PVOPS is the Kernel Infrastructure to run a PV Hypervisor on top of Linux
- #16: Where are we?1) Linux 3 contains everything needed to run Xen on a Vanilla Kernel, both as Dom0 and DomU2) That’s of course a little bit of an old hat now3) But it is worth mentioning that it only took 5 years to upstream that PVOPS into the kernel
- #18: Just one example of a survey, many morehttp://www.colt.net/cio-research/z2-cloud-2.htmlAccording to many surveys, security is actually the main reason which makes or breaks cloud adoptionBetter security means more adoptionConcerns about security means slowed adoption
- #19: So for a hypervisor, as Xen which is powering 80% of the public cloud – rackspace, AWS and many other VPS providers use Xen and with cloud computing becoming mainstream, furthering security is really importantOne of the key things there is isolation between VMs, but also simplicity as I pointed out earlierBut there are also a number of advanced features in Xen, which are not that widely know. So I wanted to give you a short overview of two of them
- #20: Ask some questions
- #21: Example: XOARSelf-destructing VMs (destroyed after initialization): PCIBack = virtualize access to PCI Bus configRestartable VMs (periodic restarts): NetBack (Physical network driver exposed to guest) = restarted on timerBuilder (instantiate other VMs) = Restarted on each request
- #24: What about domain 0 itself?Once we've disaggregated domain 0, what will be left? The answer is: very little! We'll still have the logic for booting the host, for starting and stopping VMs, and for deciding which VM should control which piece of hardware... but that's about it. At this point domain 0 could be considered as a small "embedded" system, like a home NAT box or router.
- #28: VM lifecycle (start, stop, resume) ... automation is the key pointLive snapshots: Takes a snapshot of a live VM (e.g. for disaster recovery or migration)Resource pools (multiple physical machines): XS & XCP onlylive migration: VM is backed up while running, onto shared storage (e.g. NFS) in a pool and when completed restarted elsewhere in that pool. disaster recovery: you can find lots of information on how this works at http://support.citrix.com/servlet/KbServlet/download/17141-102-19301/XenServer_Pool_Replication_-_Disaster_Recovery.pdf (the key point is that I can back up the metadata for the entire VM)Flexible storage: XAPI does hide details for storage and networkingI.e. I apply generic commands (NFS, NETAPP, iSCSI ... once its created they all appear the same) from XAPI. I only need to know the storage type when I create storage and network objects (OOL)Upgrading a host to a later version of XCP (all my configs and VMs stay the same) …and patching (broken now - bug, can apply security patches to XCP/XS or Dom0 but not DomU)
- #29: * Host Architectural Improvements. XCP 1.5 now runs on the Xen 4.1 hypervisor, provides GPT (new partition table type) support and a smaller, more scalable Dom0. * GPU Pass-Through. Enables a physical GPU to be assigned to a VM providing high-end graphics. * Increased Performance and Scale. Supported limits have been increased to 1 TB memory for XCP hosts, and up to16 virtual processors and 128 GB virtual memory for VMs. Improved XCP Tools with smaller footprint. * Networking Improvements. Open vSwitch is now the default networking stack in XCP 1.5 and now provides formal support for Active-Backup NIC bonding. * Enhanced Guest OS Support. Support for Ubuntu 10.04 (32/64-bit).Updated support for Debian Squeeze 6.0 64-bit, Oracle Enterprise Linux6.0 (32/64-bit) and SLES 10 SP4 (32/64-bit). Experimental VM templates for CentOS 6.0 (32/64-bit), Ubuntu 10.10 (32/64-bit) and Solaris 10. * Virtual Appliance Support (vApp). Ability to create multi-VM and boot sequenced virtual appliances (vApps) that integrate with Integrated Site Recovery and High Availability. vApps can be easily imported and exported using the Open Virtualization Format (OVF) standard.
- #30: Note: not exactly 1:1 with XEComparisons to other APIs in the virtualization space (source: Steven Maresca)Generally speaking XAPI is well-designed and well-executedXAPI makes it pleasantly easy to achieve quick productivityXAPI is set up to work with frameworkssuch as CloudStack and OpenStack. Some SOAPy lovers of big XML envelopes and WSDLs scoff at XML-RPC, but it certainly gets the job done with few complaintsExample codehttp://bazaar.launchpad.net/~nova-core/nova/github/files/head:/plugins/xenserver/xenapi/etc/xapi.d/plugins/ https://github.com/xen-org/xen-api/blob/master/scripts/examples/python/XenAPIPlugin.py
- #34: Hold this thought! We will come back to this later….!
- #35: Performance : similar to other hypervisorsMaturity: Tried & Tested, Most Problems that are Problems are well knownOpen source: Good body of Knowledge, Tools