Hardware support for efficient virtualization
Download as PPTX, PDF1 like2,341 views
The document discusses hardware support for efficient virtualization. It begins by classifying virtualization techniques as full virtualization, paravirtualization, or hardware-assisted virtualization. It then covers the challenges of software-only virtualization on Intel x86 processors and describes hardware virtualization extensions like Intel VT-x and VT-d, as well as AMD-V. These extensions address issues like ring compression and address space compression. The document also discusses I/O virtualization techniques like Intel VT-c and AMD IOMMU, as well as the performance of different virtualization platforms like KVM, Xen, and VirtualBox on Linux.
Hardware support for efficient virtualization
- 2. Outline • Classifications • Processor virtualization Two main Software-based solutions • Challenges to virtualize Intel x86(software-only) • Hardware-based Virtualization • Intel VT-x : x86 • Intel VT-I :Itanium (X) • Intel EPT/AMD NPT • AMD-V • Sun SPARC (X) • ARM Virtualization Extensions(X) • IBM Power(X) • • I/O virtualization • Intel VT-d • AMD IOMMU(AMD-V) • Intel VT-c • Dose these techniques work? 2
- 3. Classifications • VMM(virtual machine monitor) = hypervisor • By techniques Full Virtualization • Paravirtualization • Hardware Assisted Virtualization • • Robert P. Goldberg(Harvard University,1973) • Type 1/native/bare metal hypervisors Hypervisors run directly on the host's hardware to control the hardware and to manage guest operating systems. • Oracle VM Server for SPARC, the Citrix XenServer, KVM, VMware ESX/ESXi, and Microsoft Hyper-V hypervisor. • • Type 2/hosted hypervisors Hypervisors run within a conventional operating system environment. • VMware Workstation/player, Microsoft Virtual PC and VirtualBox • 3
- 6. Two main Software-based solutions(1) • Full virtualization using binary translation • Transforming guest OS binaries on-the-fly • • Guest applications don’t use privilege instructions Pros Support unmodified OSs (The only way of pure-software solutions) • Offer best isolation and security • Simplify migration and portability of guest OS • Cons: low performance • Examples: VMware, MS Virtual PC, Virtual box • • Disable HW virtualization 6
- 7. Full virtualization using binary translation 7
- 8. • Two main Software-based OS assisted virtualization or paravirtualization solutions(2) guest OSs help the VMM OS assisted virtualization • • • paravirtualization refers to communication between the guest OS and the VMM to improve performance and efficiency Modify the guest OS to cooperate with the VMM • Modify the OS kernel to replace non-virtualizable instructions with hypercalls(the functions provided by the VMM) Pros: Offer higher performance • Cons: Need the source code of an OS • Example: Xen, KVM(*), VMware(*) • (*) Vmware tool uses some paravirtualization techniques • optimize virtual device drivers • time synchronization • logging and guest shutdown. • Vmxnet is a paravirtualized I/O device driver • 8
- 9. OS assisted virtualization or paravirtualization 9
- 10. KVM KVM 10
- 11. Challenges to virtualize Intel x86(software-only)(1/3) • Ring Aliasing • Guest-OSes run at the Ring3 • • Original: OS:Ring 0, APP:Ring 3 (Ring0>ring3) A guest OS can know its run level • Address-Space Compression VMM must use some of the guest’s virtual-address space to manage transition between guest OS and VMM • VMM’s address spaces must be protected • • Guest could detect that it is running in a VM 11
- 12. Challenges to virtualize Intel x86(software-only)(2/3) • Non-Faulting Access to Privileged State • Some instructions should be intercepted by VMM do not involve faults • Adverse Impact on Guest System Calls • VMM must emulate every system calls • Interrupt Virtualization A VMM may manage external interrupts and deny guest to control interrupt masking • Some OS frequently mask and unmask • • VMM must process these requests. 12
- 13. Challenges to virtualize Intel x86(software-only) (3/3) • Ring Compression • Guest OS runs at the same privilege level as applications • The guest OS can’t protect guest applications • Frequent Access to Privileged Resources • VMM should deny the accesses • Address translation • Guest OS doesn’t know the physical address, so the VMM must intercepted guest page table updates 13
- 14. Intel VT-x overview(1/4) • VT=virtualization technology • Two new form of CPU operation VMX root operation : for VMM • VMX non-root operation: for guest-software • Both forms of operation support all four privilege levels(Ring0~Ring3) • • Guest OS can run at its intended privilege level 14
- 15. Two new form of CPU operation 15
- 16. Intel VT-x overview(2/4) • Two new transitions • VM entry • VMX root operation (VMM) non-root operation(VM) • VM exit • VMX non-root operation (VM) root operation (VMM) • Under VMX non-root operation, many instructions/events cause VM exits • configurable 16
- 17. Intel VT-x overview(3/4) • VMCS (Virtual Machine Control Structure) A new data structure includes guest-state area and host-state area • VM entry: load the guest-state area and save the host-state area • VM exit : load the host-state area and save the guest-state area • The exiting conditions controlled by the VM-execution fields • Switch the structure will switch the address space • 17
- 18. Intel VT-x overview(4/4) • VMCS supports interrupt virtualization • Determine the conditions of VM to cause VM exit • • • • • All interrupt Whenever guest OS is ready to receive interrupts Which exception? Which port access attempts? Which Model Specific Register access attempts? 18
- 19. Intel EPT / AMD NPT(1) • EPT (Extended Page Tables) • • “EPT provides performance gains of up to 48% for MMU-intensive benchmarks and up to 600% for MMU-intensive microbenchmarks.” – VMware AMD’s nested page table (NPT) is similar to EPT • A.k.a Rapid Virtualization Indexing (RVI) • “RVI provides performance gains of up to 42% for MMU-intensive benchmarks and up to 500% for MMU-intensive microbenchmarks.” -VMware 19
- 20. Intel EPT / AMD NPT(2) • Software MMU (software-only) Hardware uses the shadow page table • VMM must maintain the shadow page table • 20
- 21. Intelmaintains guest page tables EPT / AMD NPT(3) Guest-OS • Hardware MMU • VMM maintains PPN->MPN mappings in an additional level of page tables • The hardware will find the LPN->MPN with the two pages • 21
- 22. AMD-V(1/2) • Tagged TLB Add the ASID • Hardware features that facilitate efficient switching between virtual machines for better application responsiveness • • Two new form of CPU operation Host mode : for VMM (similar to Intel’s VMX root operation) • Guest mode : for guest software (similar to Intel’s VMX non-root operation) • new instructions • • • • • vmrun : host mode guest mode exit : guest mode host mode vmcall: it lets the operating system and VMM communicate directly A new structure • Virtual Machine Control Block (VMCB) • Similar to Intel’s VMCS 22
- 23. AMD-V(2/2) • Nested page table (NPT)/ Rapid Virtualization Indexing (RVI) • VMM migration • Use the CPUID to identify the ability of the processor where the VMM runs, and the VMM use the supported functions. 23
- 24. Hardware-base solution with VTx(1/2) • Address-Space Compression • VM Exits / VM Entries change the linear address space • Ring Aliasing • & Ring Compression VT-x allows guest OS to run at its intended privilege level • Nonfaulting Access to Privileged State Either causes transition to VMM • Or becomes unimportant to VMM • 24
- 25. Hardware-base solution with VTx(2/2) • Guest System Calls • a guest OS can run at privilege level 0 • Frequent Access to Privileged Resources • VT-x provides TPR shadow. VMM is only involved when the value drops below the threshold VMM only processes the situation it cares. 25
- 26. Hardware Assisted Virtualization of x86 26
- 28. Current I/O virtualization techniques • Emulation The VMM supports virtual devices that guest OS can recognize • The virtual device models are responsible to translate commands and data. • Pros. No requirement to modify guest-OSs • Cons. Low performance • • Paravirtualization Modify the guest software (driver)to enhance the performance • Pros. better performance • Cons. Limited applicability. (modify need the source code) • • Direct assignment Bind a specify device to a VM • VMM allow the owning VM to connect directly • Issue command (go) low overhead • DMA? (back) • 28
- 29. DMA on a virtualizing system • DMA Driver issue a packet consists of command, physical address, etc. • DMA controller read/write data from/to the physical address • Challenge? • A physical address that a Guest-OS knows is not really physical ! • The really physical address space is managed by the VMM • The DMA controller will incorrectly write data to an address. • 29
- 30. Intel VT-d(1/2) • Need the support of the North bridge • Two functions • Bind devices to a specify VM • • DMA remapping Interrupt virtualization • Interrupt remapping • DMA remapping DVA (DMA Virtual Address), GPA(Guest Physical Address), HPA(Host Physical Address) • A guest-OS issue a DMA request with DVA(=GPA) • The VT-d hardware will translate the DVA to HPA • • The concept: lookup tables 30
- 31. DMA remapping 31
- 32. Intel VT-d (2/2) • Interrupt Remapping • Assign an interrupt attribute • • Destination processor, vector, etc. A VMM enables the interrupt requests from the I/O device to target the physical CPUs running the appropriate virtual CPUs of the legacy VM • AMD IOMMU is similar to Intel VT-d 32
- 33. Intel VT-c • Virtualization Technology for Connectivity • Virtualization on devices • A collection of technologies that improve the performance of network I/O on a virtualized system • VT-c is comprised of two components • VMDq (Virtual Machine Device Queues) A hardware-base enhancement • Target: throughput • • VMDc (Virtual Machine Direct Connect) Virtualizing physical I/O ports of a network controller into multiple virtual I/O ports, and then to map the virtual ports to individual VMs • Target :VT-x + VT-d + VT-c nearly native performance • 33
- 34. Why VMDq? 34
- 35. 35
- 36. VMDc 36
- 38. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(1) 38
- 39. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(2) 39
- 40. Ubuntu 11.10: Xen vs. KVM vs. VirtualBox(3) 40
- 41. Ubuntu 12.10: KVM vs. Xen (1) 41
- 42. Ubuntu 12.10: KVM vs. Xen (2) 42
- 43. Ubuntu 12.10: KVM vs. Xen (3) 43
- 44. Ubuntu 12.10: KVM vs. Xen (4) 44
- 45. Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD(1) • Ubuntu 12.04 LTS, an Intel Core i7 3960X "Sandy Bridge" Extreme Edition and AMD FX-8150 "Bulldozer" systems were used. 45
- 46. Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD(2) 46
- 47. Intel Ivy Bridge Linux Virtualization Performance(1) 47
- 48. Intel Ivy Bridge Linux Virtualization Performance(2) 48
- 49. Summarization of Hardware Assisted Virtualization • Hardware provides some mechanisms to reduce overheads of virtualization to improve performance • Pros. The highest performance in theory (a counter example, 2006 VMware) • Support unmodified Oss • Simplify the development of VMM • • Cons. • Need newer processors • Example • KVM(basic requirements) 49
- 50. References • Performance Evaluation of Intel EPT Hardware Assist, VMware • I/O Virtualization and AMD's IOMMU • • Processor-Based Virtualization, AMD64 Style, Part I • • http://developer.amd.com/documentation/articles/pages/630200614.aspx Processor-Based Virtualization, AMD64 Style, Part II • • http://developer.amd.com/documentation/articles/pages/892006101.aspx http://developer.amd.com/documentation/articles/pages/630200615.aspx Intel technology Journal, vol 10, issue 3, 2006 Intel virtualization technology: Hardware Support for Efficient processor virtualization • Intel virtualization technology for Directed I/O • • ARM virtualization Extension Architecture Specification • A Comparison of software and hardware techniques for x86 virtualization,Vmware • http://www.intel.com/network/connectivity/solutions/vmdc.htm • http://www.intel.com/network/connectivity/solutions/vmdq.htm • http://software.intel.com/en-us/blogs/2009/09/30/understanding-vt-c-virtualizationtechnology-for-connectivity/ 50
- 51. References • Ubuntu 11.10: Xen vs. KVM vs. VirtualBox http://www.phoronix.com/scan.php?page=article&item=ubuntu_11 10_xenkvm&num=1 • Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD http://www.phoronix.com/scan.php?page=article&item=ubuntu_12 04_virt&num=1 • Intel Ivy Bridge Linux Virtualization Performance http://www.phoronix.com/scan.php?page=article&item=intel_iv y_virtualization&num=5 • http://en.wikipedia.org/wiki/Hypervisor 51
- 52. Q&A 52
- 53. THANK YOU 53
Editor's Notes
- #8: 按照INTEL的說法GOS該放RING3比較好, Ring Compression
- #9: Hypercalls不是硬體指令，是VMM開放出來的FUNCTION，供GOS和VMM溝通(*)採用一些優點，但CODE不用改
- #11: Performance of HW-based is based on techniques in 2006. Now(2012) the performance is improved! Look later slides.
- #12: Ring AliasingThe problem that arise when software is run at a privilege level other than the privilege level for which it was written
- #13: Adverse 不利的
- #14: Ring Compression:Ring 0-2 在記憶體方面是沒區分的，為了保護VMM，GOS也要放RING3
- #25: Address-Space Compression -VM Exits / VM Entries -> switch VMCS->switch address space
- #32: Device 1 is binded to domain 1
- #46: Switching over to the computationally-intensive tests, beginning with Google's libvpx VP8 encoding test