Control hypervisor via libvirt

sean666666@gmail.com P1
Control Hypervisor via Libvirt
Sean Chang
2011/4

大綱
• 為什麼要用 libvirt
• Lifecycle of VM
• 順帶一提 virsh
• 順帶一提 xm
• 進入重點 libvirt
• Libvirt 使用範例
• Libvirt 使用經驗
• Libvirt 議題
• DEMO

管理 hypervisor 的方法
• 下指令
o xm
o virsh
• 自己寫程式
o Libvirt
o A virtualization API
o Open source
o 可程式化

飲水思源
了解VM(Guest domain)的生命週期

Guest domain status
State Description
Undefine
d
This is the baseline state. An undefined guest domain has not been
defined or created in any way.
Defined A defined guest domain has been defined but is not running. This
state could also be described as Stopped.
Running A running guest domain is defined and being executed on a hypervisor.
Paused A paused guest domain is in a suspended state from the Running
state. Its memory image has been temporarily stored, and it can be
resumed to the Running state without the guest domain operating
system being aware it was ever suspended.
Saved A saved domain has had its memory image, as captured in the
Paused state, saved to persistent storage. It can be restored to the
Running state without the guest domain operating system being
aware it was ever suspended.

Guest domain lifecycle

VM(Guest domain) status

VM Lifecycle
What does
Libvirt actully
do？

工欲善其事
必先利其器

xm
• Xen management user
interface
• The xm program is the main
interface for managing Xen
guest domains. The program
can be used to create, pause,
and shutdown domains. It can
also be used to list current
domains, enable or pin
VCPUs, and attach or detach
virtual block devices.
• Example
o xm create Fedora12
o xm start Fedora12
o xm help
o xm list
o xm mem-set {domain-id,
mem}
o xm migrate {domain-id,
host-ip}
o etc…沒事多玩指令，多
玩指令沒事

virsh(1/2)
• virsh 是個用來管理客座端和
hypervisor（管理程序）的指令
列介面工具。
• 不過說穿了，這個指令列也是
用 libvirt 作為底層函式庫寫出
來的。
指令 Description
help 印出基本協助資訊。
list 列出所有客座端。
dumpxml 輸出客座端的 XML 配置檔案。
create 由一個 XML 配置檔案來建立客座端並啟用新
的客座端。
start 啟用一個未啟用中的客座端。
destroy 強制客座端停下。
define 輸出客座端的 XML 配置檔案。
domid 顯示客座端的 ID。
domuuid 顯示客座端的 UUID。
dominfo 顯示客座端的相關資訊。
domname 顯示客座端的名稱。
domstate 顯示客座端的狀態。
quit 退出互動式終端機。
reboot 重新啟動客座端。
restore 恢復之前已儲存於一個檔案中的客座端。
resume 恢復一個暫停中的客座端。
save 將客座端目前的狀態儲存至一個檔案中。
shutdown 正常地停用客座端。
suspend 將客座端暫停。
undefine 刪除與某個客座端有關聯的所有檔案。
migrate 將客座端遷移至另一部主機上。

virsh(2/2)
• Example
o # virsh connect {hostname
OR URL}
o # virsh create
configuration_file.xml
o # virsh suspend {domain-
id, domain-name or
domain-uuid}
o etc…沒事多玩指令，多
玩指令沒事
指令 Description
setmem 為客座端設置分配記憶體。
setmaxmem 設定 hypervisor 所能使用的記憶體最大限制。
setvcpus 更改分配給客座端的虛擬 CPU 數量。
vcpuinfo 顯示有關於客座端的虛擬 CPU 資訊。
vcpupin 控制客座端的虛擬 CPU 相似性。
domblkstat 顯示正在執行中的客座端的區塊裝置數據。
domifstat 顯示執行中的客座端的網路介面卡數據。
attach-device 透過使用某個 XML 檔案中的裝置定義來將裝置附加至客座端上。
attach-disk 附加新磁碟裝置至客座端。
attach-interface 附加新網路介面卡至客座端。
detach-device 將裝置從客座端上分離，接受與 attach-device 指令相同類型的
XML 描述。
detach-disk 將磁碟裝置由客座端上移除。
detach-interface 將網路介面卡由客座端上移除。

不是不寫程式
不過時後也已經到了

Libvirt(http://libvirt.org/index.html)
• A toolkit to interact with the virtualization
capabilities of recent versions of Linux (and
other OSes), see our project goals for details.
• Free software available under the GNU Lesser
General Public License.(LGPL)
• A long term stable C API
• A set of bindings for common languages.(Java、
Python…etc)
• Latest release: 0.8.8: Feb 17 2011

Terminology and goals
Xen
…
一般稱呼 Libvirt的術語解釋
VM Domain An instance of an operating system (or
subsystem in the case of container
virtualization) running on a virtualized machine
provided by the hypervisor
Xen Hypervisor A layer of software allowing to virtualize a
node in a set of virtual machines with possibly
different configurations than the node itself
Physical
machine
Node A single physical machine

Libvirt drivers
• Hypervisor drivers
o LXC - Linux Containers
o OpenVZ
o QEMU
o Test - Used for testing
o UML - User Mode Linux
o VirtualBox
o VMware ESX
o VMware
Workstation/Player
o Xen
o 支援Multi-Vendor(表面上…)
• Storage drivers
o Directory backend
o Local filesystem backend
o Network filesystem backend
o Logical Volume Manager
(LVM) backend
o Disk backend
o iSCSI backend
o SCSI backend
o Multipath backend

API concepts(1/2)
Objects exposed

API concepts(2/2)
Libvirt drivers
就是OS
或是應用程式
API呼叫
deamon
remotelocal

Xen config V.S. XML
• VM的設定檔
o For Xen
o For libvirt

細述 XML
unique
全虛擬
最大的實體資源分配
網路採bridge方式對外連通
這張虛擬網卡
掛在node的xenbr0
VNC port

Python example – 基本VM操作

Python example – 掛載/卸載磁碟
已經準備好
掛載的image空間

我不入地獄誰…
往下挖

認證
• 呼叫遠端的libvirtd需要先經過認證，認證方
式如下
• ssh
o ssh username root: xen+ssh://root@node.example.com/
o 然後輸入帳號密碼，缺點，如果程式要全自動化會被卡在這裡。
• authorized key
o 製作Public / private key
o #ssh-keygen -t rsa
o 放在相對應的目錄: /root/.ssh/

Debug / loggin
• Log message
o Generated at runtime by the libvirt code
o Timestamp, a category, a priority level, function name and line number indicating
where the message originated from, and a formatted message.
• Priority levels
o 1 (or debug) - log all messages
o 2 (or info) - log all non-debugging information
o 3 (or warn) - log only warnings and errors - this is the default
o 4 (or error) - log only errors
• Filter
o x:name
o 1:qemu
o 1:qemu 4:remote
• Log output
o 3:syslog:libvirtd 1:file:/tmp/libvirt.log

虛擬裝置也支援熱插拔嗎?
• 半虛擬技術支援在不開機的情況下，CPU、MEM、DISK
都可以做調整。
• 全虛擬CPU、MEM要調整一定得重開機。
• 全虛擬DISK熱插拔則視VM的kernel版本而定。
• The Xen device model is more or less unchanged in the pv-ops
kernel. Converting a driver from the xen-unstable or 2.6.18-xen tree
should mostly be a matter of getting it to compile. There have been
changes in the Linux device model between 2.6.18 and 2.6.26, so
converting a driver will mostly be a matter of forward-porting to the
new kernel, rather than any Xen specific issues.
• Kernel version > 2.6.26 可能比較穩定有機會實現全虛擬的硬碟熱插
拔。

Xen disk kinds and supporting
• Disk type
o IDE
o Virtual disk
o iSCSI
o USB Disk
• 實測熱插拔
xen 4.0 + 2.6.32.11
/ Dell
Storage
IDE Virtual SCSI USB Disk
centos5.4final
Kernel 2.6.18
× ○ × ×
RH6beta
Kernel 2.6.18
× × × ×

API Issues
• create 跟 define 有什麼不同?
o 都是用來創建VM的API
o 差別在生命週期不完全相同
o create: 創建VM之外，順便把VM開起來，shutdown後該VM自動
從hypervisor卸除。
o define: 僅創建VM，須靠start API把VM開起來，但永遠定義在
hypervisor裡。甚至連migrate後也在。
• shutdown 跟 destroy 有什麼不同?
o 都是要把VM的狀態從running改成stop
o Shutdown：就是一般作業系統的關機。
o Destroy：以實體主機來看，就是直接把插頭拔掉。

Development trick(1/3) - Problem
• Libvirt 在偵測 xen 上的VM實際狀態不穩定
• 連續下五十次偵測VM status的結果
[2, 665600L, 444416L, 1, 178818312669L][2, 665600L, 444416L, 1, 178818786889L][2, 665600L, 444416L, 1, 178819270686L][2
665600L, 444416L, 1, 178819782827L][2, 665600L, 444416L, 1, 178820286716L][0, 665600L, 444416L, 1, 178825799216L][2,
665600L, 444416L, 1, 178826484955L][2, 665600L, 444416L, 1, 178827054507L][2, 665600L, 444416L, 1, 178827646602L][2,
665600L, 444416L, 1, 178828162295L][2, 665600L, 444416L, 1, 178828718993L][2, 665600L, 444416L, 1, 178830313690L][2,
665600L, 444416L, 1, 178831831052L][2, 665600L, 444416L, 1, 178832305724L][2, 665600L, 444416L, 1, 178832859023L][2,
665600L, 444416L, 1, 178833339605L][2, 665600L, 444416L, 1, 178833848167L][2, 665600L, 444416L, 1, 178834372305L][2,
665600L, 444416L, 1, 178834895557L][2, 665600L, 444416L, 1, 178835386865L][2, 665600L, 444416L, 1, 178835922103L][2,
665600L, 444416L, 1, 178836494779L][2, 665600L, 444416L, 1, 178836973848L][2, 665600L, 444416L, 1, 178837452908L][2,
665600L, 444416L, 1, 178838008290L][2, 665600L, 444416L, 1, 178838553784L][2, 665600L, 444416L, 1, 178839024984L][2,
665600L, 444416L, 1, 178839462699L][2, 665600L, 444416L, 1, 178839987321L][2, 665600L, 444416L, 1, 178840510368L][2,
665600L, 444416L, 1, 178841185524L][2, 665600L, 444416L, 1, 178841873368L][2, 665600L, 444416L, 1, 178842393659L][2,
665600L, 444416L, 1, 178842910878L][2, 665600L, 444416L, 1, 178843454749L][2, 665600L, 444416L, 1, 178843947320L][2,
665600L, 444416L, 1, 178844427452L][2, 665600L, 444416L, 1, 178844942837L][2, 665600L, 444416L, 1, 178845472009L][2,
665600L, 444416L, 1, 178846036556L][2, 665600L, 444416L, 1, 178846502921L][2, 665600L, 444416L, 1, 178847082404L][2,
665600L, 444416L, 1, 178847612475L][2, 665600L, 444416L, 1, 178848158639L][2, 665600L, 444416L, 1, 178848527989L][2,
665600L, 444416L, 1, 178849177317L][2, 665600L, 444416L, 1, 178849704749L][2, 665600L, 444416L, 1, 178850274864L][2,
665600L, 444416L, 1, 178850717630L][2, 665600L, 444416L, 1, 178851307315L][2, 665600L, 444416L, 1, 178852079044L][2,
665600L, 444416L, 1, 178852861964L][2, 665600L, 444416L, 1, 178854056260L][2, 665600L, 444416L, 1, 178854907434L][2,
665600L, 444416L, 1, 178856457345L][2, 665600L, 444416L, 1, 178856918293L][2, 665600L, 444416L, 1, 178857394787L][1,
665600L, 444416L, 1, 178857716163L][2, 665600L, 444416L, 1, 178858177594L][2, 665600L, 444416L, 1, 178858773954L][2,
665600L, 444416L, 1, 178859309076L][2, 665600L, 444416L, 1, 178859897892L][2, 665600L, 444416L, 1, 178860400666L][2,
665600L, 444416L, 1, 178860937146L][2, 665600L, 444416L, 1, 178861383848L][1, 665600L, 444416L, 1, 178861889219L][2,
665600L, 444416L, 1, 178862333614L][2, 665600L, 444416L, 1, 178862848250L][2, 665600L, 444416L, 1, 178863325489L][2,
665600L, 444416L, 1, 178863674106L][2, 665600L, 444416L, 1, 178864153743L][2, 665600L, 444416L, 1, 178864647934L][2,
665600L, 444416L, 1, 178864970657L][2, 665600L, 444416L, 1, 178865558487L][2, 665600L, 444416L, 1, 178865890459L][2,
665600L, 444416L, 1, 178866309383L][2, 665600L, 444416L, 1, 178866793058L][2, 665600L, 444416L, 1, 178867306348L][2,

Development trick(2/3) - Solution
• 如果狀態不穩，我們無法取得VM確實的狀態。
• 參考 VirtManager 的解法，利用以下API的特性
o 找出可以幫忙的 libvirt API。
o listDomainsID(): 列出所有xen上在跑的domain。
o 因為只有running的VM狀態不穩，所以利用listDomianID()這個
API把這些可能不穩的VM先抓出來，在把0,1,2等狀態都歸納為
runnung。

Development trick(3/3) - Code
31
1
2
3
4
5

libvirt 處理 Xen 與 KVM 的行為
• 不同的 hypervisor，libvirt的行為也會有所差異
• 以下以Xen跟KVM為例：

開發心得
• 多用python的特有的API查詢指令help()、dir()來
幫助自己掌握libvirt的版況
• 不要太期待libvirt可以幫你做太多很強大的事情，
官網很多只定義好介面，尚未實作。舉例:
• 要熟悉XML config
• 要熟悉hypervisor的行為

參考資料
• http://docs.fedoraproject.org/zh-
TW/Fedora/12/html/Virtualization_Guide/chap-Virtualization_Guide-
Managing_guests_with_virsh.html
• http://linux.die.net/man/1/xm
• http://libvirt.org/index.html
• http://libvirt.org/sources/virshcmdref/html/
• http://wiki.xensource.com/xenwiki/XenParavirtOps

DEMO
• 準備 VM
o define
o create
• 操作 VM
o start
o suspend
o resume
o reboot
o shutdown
• 添加/卸載設備
o attach
o detach
• 取得 VM
o lookupByXXX
• 跨主機(下回分曉…)
o virtManager 介紹
o migration
o live migration

Thank you
• 末記
o 這份資料的技術實作於2010~2011，如非最新資料，敬
請見諒
• End
o The technology information was implemented in 2010 ~
2011, sorry for no update on time.

Control hypervisor via libvirt

More Related Content

What's hot (20)

Similar to Control hypervisor via libvirt (20)

Recently uploaded (20)

Control hypervisor via libvirt