OpenShift Virtualization - VM and OS Image Lifecycle
1 like732 views
1. Select "Create Virtual Machine" from the Workloads menu. 2. On the General tab, choose the source of the virtual machine such as a Container image, URL, or existing disk. Then select the Operating System. 3. Configure resources for the virtual machine including CPU, memory, and storage on the Hardware tab. 4. Review and create the virtual machine. The new virtual machine will be added to the list and can be managed like other workloads.
![Creating additional networks
Edit the networks operator
oc edit networks.operator.openshift.io cluster
Configure additional networks
additionalNetworks:
- name: test-network-1
namespace: default
rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-1",
"type": "bridge", "master":"bond0",
"ipam" :{"type":"static", "addresses":[{"address":"161.156.177.60/28"}]}}'
type: Raw
- name: test-network-2
namespace: default
rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-2",
"type": "bridge", "master":"bond0",
"ipam" :{"type":"static", "addresses":[{"address":"161.156.177.61/28"}]}}'
type: Raw
31](https://image.slidesharecdn.com/mihai-criveti-04-211023130507/85/OpenShift-Virtualization-VM-and-OS-Image-Lifecycle-36-320.jpg)
![Packer: Variables
Variables to parametrized builds and hide secrets
{
"variables": {
"my_secret": "{{env `MY_SECRET`}}",
"not_a_secret": "plaintext",
"foo": "bar"
},
"sensitive-variables": ["my_secret", "foo"],
}
51](https://image.slidesharecdn.com/mihai-criveti-04-211023130507/85/OpenShift-Virtualization-VM-and-OS-Image-Lifecycle-60-320.jpg)
![Packer: Builders
KVM Builder (qemu)
"builders": [{
"type": "qemu",
"accelerator": "kvm",
"format": "qcow2",
"disk_interface": "virtio-scsi",
"boot_command": [
"<up><wait><tab>",
" text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/
{user `vm_name`}}.cfg net.ifnames=0",
"<enter><wait>"
],
"vm_name": "{{split build_type "-" 0}}-{{user `vm_name`}}",
"iso_urls": [
"{{user `iso_local_url`}}",
"{{user `iso_download_url`}}"
],
"iso_checksum": "{{user `iso_sha256`}}",
"output_directory": "{{user `builds_dir`}}/{{split build_type "-" 0}}
-{{user `vm_name`}}", }] 52](https://image.slidesharecdn.com/mihai-criveti-04-211023130507/85/OpenShift-Virtualization-VM-and-OS-Image-Lifecycle-61-320.jpg)
![Provisioners: run post-install tasks
Chaining multiple provisioners
"provisioners": [
{
"type": "shell",
"script": "setup.sh"
},
{
"type": "ansible",
"playbook_file": "{{user `playbook_file`}}"
}],
53](https://image.slidesharecdn.com/mihai-criveti-04-211023130507/85/OpenShift-Virtualization-VM-and-OS-Image-Lifecycle-62-320.jpg)
![Post-processors: compress or upload your image
Compress, post-process and upload the results
{
"post-processors": [
{
"type": "compress",
"format": "tar.gz"
},
{
"type": "upload",
"endpoint": "http://example.com"
}
]
}
54](https://image.slidesharecdn.com/mihai-criveti-04-211023130507/85/OpenShift-Virtualization-VM-and-OS-Image-Lifecycle-63-320.jpg)
OpenShift Virtualization - VM and OS Image Lifecycle
- 1. OpenShift Virtualization - VM and Image Lifecycle Building and packaging OS Images with KVM, qemu-img and podman Mihai Criveti, CTO Cloud Native and Red Hat Solutions IBM, STSM, RHCE, OpenShift Certified November 23, 2020 1
- 2. OpenShift Virtualization Overview Container Build Tools: Podman, Skopeo and Buildah Container Registries Prerequisities and Build Tools Create Virtual Machine Wizard Building OS Images Automating image builds with Kickstart and Ansible IBM Cloud: VPC Gen 2 Custom Images Continuous Image Build Layered Image Compliance with OpenSCAP 2
- 4. Overview • Prerequisites • KVM and qemu tools for building OS images • qemu, cloud-init and virtio drivers • podman and skopeo for building container images • Virtual Machine Options • OS Support (RHEl 6-8, Fedora, Windows 10, 2012R2, 2016, 2019) • Type: Ephemeral or Persistent • Source: PXE, URL, Container or Disk • Virtual Machine Disks • Volume Mode: Filesystem or Block • Storage volume types: persistentVolumeClaim, dataVolume, ephemeral, containerDisk, emptyDisk, cloudInitNoCloud • CDI - Containerized Data Importer • CDI supported operations matrix • CDI operations that require scratch space • Building container images with KVM virt-install • Building a Fedora image • Installing VirtIO, qemu and cloud-init • Using qemu-img to compress the image • Packaging the image as a container using podman • Enable the OpenShift Registry for external access • Upload images with podman • Copy images with skopeo • Testing OS Images • QCOW2 images on HTTP Servers or S3 Storage • Container images from the container registry • Using the IBM Cloud OS Images! 3
- 5. Provision OpenShift Virtualization on IBM Cloud Deploy ROKS • Create a ROKS cluster with at least 1 Bare Metal worker node. Make sure you select version 4.5.x or higher. Install the OpenShift Virtualization Operator and create a cluster • Install the OpenShift Virtualization Operator (2.4 or higher) • Create a OpenShift Virtualization Operator Deployment: kubevirt-hyperconverged cluster on the Bare Metal node(s). Create a new project namespace oc new-project virtual-machines You can now create Virtual Machines by clicking on Workloads > Virtualization. 4
- 6. Red Hat Portfolio integration RHV 4.4 can manage OpenShift Virtualization VMs OpenShift virtualization is a feature of OpenShift Container Platform and is delivered, integrated and managed via the OpenShift Operator framework. OpenShift Virtualization integrates with Red Hat Virtualization. Figure 1: RHV Integration RHV Console With RHV 4.4, the RHV Manager allows you to easily add an existing OpenShift virtualization cluster as an external provider and then to perform basic management of underlying VMs side-by-side with RHV VMs. Figure 2: RHV Integration 5
- 7. Migration VMware Migration You can import Virtual Machines from VMware. RHV Migration You can import Virtual Machines from RHV. 6
- 8. Virtual Machine Types Ephemeral • When using a ephemeral storage volume type, or containerDisk. • The ephemeral image is created when the virtual machine starts and stores all writes locally. The ephemeral image is discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not mutated in any way. Persistent (persistentVolumeClaim) • When using a persistentVolumeClaim • Attaches an available PV to a virtual machine. Attaching a PV allows for the virtual machine data to persist between sessions. • Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual machine instance is the recommended method for importing existing virtual machines into OpenShift Container Platform. Note CDI: Containerized Data Importer. 7
- 9. Storing VMI Disks in the Container Registry Use Cases Immutable VMIs booting from ephemeral disk • launch VMI workloads backed by local ephemeral storage. • VMI workload does not need to remain persistent across VMI restarts and the workload does not require live migration support. Questions: • I this emptyDir? • What happens when it fills up? • Can we setup a quota? 8
- 10. Storage Features Features Live Migration Host-assisted VM disk cloning OpenShift Container Storage: RBD block-mode volumes Yes Yes OpenShift Virtualization hostpath provisioner No Yes Other multi-node writable storage Yes (1) Yes (1) Other single-node writable storage No Yes (2) 1. PVCs must request a ReadWriteMany access mode. 2. PVCs must request a ReadWriteOnce access mode. You cannot live migrate virtual machines that use: - A storage class with ReadWriteOnce (RWO) access mode - Passthrough features such as SRI-OV and GPU 9
- 11. CDI: Containerized Data Importer 10
- 13. CDI: Containerized Data Importer Overview CDI Function • persistent storage management add-on for Kubernetes. • provides a declarative way to build Virtual Machine Disks on PVCs for Kubevirt VMs • provides a way to populate PVCs with VM images or other data upon creation. • data can come from different sources: a URL, a container registry, another PVC (clone), or an upload from a client. Import from URL This method is selected when you create a DataVolume with an http source. Supports basic authentication (secret) and custom TLS certificates (ConfigMap). Import from container registry When a DataVolume has a registry source CDI will populate the volume with a Container Disk downloaded from the given image URL. 12
- 14. CDI supported operations matrix DataVolumes DataVolume objects are custom resources that are provided by the Containerized Data Importer (CDI) project. DataVolumes orchestrate import, clone, and upload operations that are associated with an underlying PersistentVolumeClaim (PVC). DataVolumes are integrated with KubeVirt, and they prevent a virtual machine from being started before the PVC has been prepared. Understanding scratch space The Containerized Data Importer (CDI) requires scratch space (temporary storage) to complete some operations, such as importing and uploading virtual machine images. During this process, the CDI provisions a scratch space PVC equal to the size of the PVC backing the destination DataVolume (DV). The scratch space PVC is deleted after the operation completes or aborts. Note: The CDI requires requesting scratch space with a file volume mode, regardless of the PVC backing the origin DataVolume. If the origin PVC is backed by block volume mode, you must define a StorageClass capable of provisioning file volume mode PVCs. 13
- 15. CDI operations that require scratch space Registry imports The CDI must download the image to a scratch space and extract the layers to find the image file. The image file is then passed to QEMU-IMG for conversion to a raw disk. Upload image QEMU-IMG does not accept input from STDIN. Instead, the image to upload is saved in scratch space before it can be passed to QEMU-IMG for conversion. HTTP imports of archived images QEMU-IMG does not know how to handle the archive formats CDI supports. Instead, the image is unarchived and saved into scratch space before it is passed to QEMU-IMG. HTTP imports of authenticated images QEMU-IMG inadequately handles authentication. Instead, the image is saved to scratch space and authenticated before it is passed to QEMU-IMG. HTTP imports of custom certificates QEMU-IMG inadequately handles custom certificates of HTTPS endpoints. Instead, the CDI downloads the image to scratch space before passing the file to QEMU-IMG. 14
- 16. Container Build Tools: Podman, Skopeo and Buildah
- 17. Podman Overview What is Podman? Figure 3: podman - manage pods, containers and OCI compliant container images How is Podman different? • Can be run as a regular user without requiring root. • Can manage pods (groups of one or more containers that operate together). • Lets you import Kubernetes definitions using podman play. • Fork-exec model instead of client-server model (containers are child processes of podman). • Compatible with Docker, Docker Hub or any OCI compliant container implementation. 15
- 18. Buildah What is Buildah? Figure 4: buildah - build container images from CLI or Dockerfiles How is Buildah different? • Containers can be build using simple CLI commands or shell scripts instead of Dockerfiles. • Images can then be pushed to any container registry and can be used by any container engine, including Podman, CRI-O, and Docker. • Buildah is also often used to securely build containers while running inside of a locked down container by a tool like Podman, OpenShift/Kubernetes or Docker. 16
- 19. Skopeo What is Skopeo? Figure 5: skopeo - inspect and copy containers and images between different storage How does Skopeo help? • It can copy images to and from a host, as well as to other container environments and registries. • Skopeo can inspect images from container image registries, get images and image layers, and use signatures to create and verify images. 17
- 20. Install podman, buildah and skopeo Fedora 32 / RHEL 8 # Install podman, buildah and skopeo on Fedora 32 sudo dnf -y install podman buildah skopeo slirp4netns fuse-overlayfs Ubuntu / Debian sudo apt update && sudo apt -y install podman buildah skopeo Getting help podman version podman --help # list available commands man podman-ps # or commands like run, rm, rmi, image, build podman info # display podman system information https://podman.io/getting-started/installation 18
- 22. Container Image Registries available to OpenShift quay.io • Public / private container repository from Red Hat • Quay is a container image registry that enables you to build, organize, distribute, and deploy containers. • Quay gives you security over your repositories with image vulnerability scanning and robust access controls. IBM Cloud Container Registry • IBM Cloud multi-tenant private image registry • store and distribute Docker images in a managed, private registry. Image Registry Operator in OpenShift Container Platform • The Image Registry Operator installs a single instance of the OpenShift Container Platform registry, and manages all registry configuration, including setting up registry storage. • Integrated OpenShift Container Platform registry: built-in container image registry that runs as a standard workload on the cluster. • Can be exposed externally (not exposed by default). Other options • JFrog Artifactory. • GitHub / GitLab provide a container registry as well. • Docker Hub (not recommended, will limit free pulls, security concerns). 19
- 23. Private Registry Security: Create a Secret Get the secret base64 oc whoami | base64 oc whoami -t | base64 Create a Secret in the same namespace as the DataVolume secret.yaml apiVersion: v1 kind: Secret metadata: name: endpoint-secret labels: app: containerized-data-importer type: Opaque data: accessKeyId: "" # <optional: your key or user name, base64 encoded> secretKey: "" # <optional: your secret or password, base64 encoded> podman create secret.yaml podman login -u $(oc whoami) -p $(oc whoami -t) $REGISTRY 20
- 24. Private Registry Security: Create a Volume Create your volume apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume ... spec: source: registry: url: "docker://my-private-registry:5000/my-username/my-image" secretRef: my-docker-creds certConfigMap: my-registry-certs ... Get the pvc oc apply -f datavolume.yaml oc get pvc watch oc get pvc,dvs,pod 21
- 25. Registry TLS Certificate configuration Create a ConfigMap with certificates in the same namespace as the DataVolume apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume ... spec: source: registry: url: "docker://my-private-registry-host:5000/my-username/my-image" certConfigMap: my-registry-certs ... 22
- 26. OpenShift Internal Registry Exposing the OpenShift Container Registry # Set the DefaultRoute to True: oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge Log in with podman HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') podman login -u $(oc whoami) -p $(oc whoami -t) --tls-verify=false $HOST Accessing the registry from inside the cluster image-registry.openshift-image-registry.svc:5000 Accessing the registry from OUTSIDE the cluster $HOST/namespace/myimage 23
- 27. Prerequisities and Build Tools
- 28. Image Build and Management Server You need a Bare Metal server running RHEL 8 to build images using KVM (virt-install) and setup the virtctl client. Install KVM yum install @virt virt-top libguestfs-tools virt-manager virt-install virt-viewer qemu-img sudo systemctl enable --now libvirtd Install virtctl client on RHEL 7 subscription-manager repos --enable rhel-7-server-cnv-2.4-rpms yum -y install kubevirt-virtctl Install the virtctl client on RHEL 8 subscription-manager repos --enable cnv-2.4-for-rhel-8-x86_64-rpms dnf -y install kubevirt-virtctl 24
- 29. HTTP Server for Images with Basic Access Install Apache dnf install -y httpd httpd-tools systemctl start httpd && systemctl enable httpd Enable Basic Auth for /etc/apache2/apache2.conf <Directory "/var/www/html"> AuthType Basic AuthName "Restricted Content" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Directory> Create a password file htpasswd -c /etc/apache2/.htpasswd cmihai Store OS images cp image.qcow2 /var/www/html 25
- 30. Configure authentication in OpenShift for your HTTP Server Secret apiVersion: v1 kind: Secret metadata: name: endpoint-secret labels: app: containerized-data-importer type: Opaque data: accessKeyId: "" secretKey: "" Storage apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: my-data-volume spec: source: http: url: "https://server/image.img" secretRef: endpoint-secret pvc: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi 26
- 31. Create Virtual Machine Wizard
- 32. Creating Virtual Machines Creating Virtual Machines can be done with: • Create Virtual Machine wizard • YAML file with OpenShift Console • Using the CLI • Importing a VMware virtual machine or template with the virtual machine wizard Notes • If either URL or Container are selected as the Source in the General step, a rootdisk disk is created and attached to the virtual machine as the Bootable Disk. You can modify the rootdisk but you cannot remove it. • When you create your virtual machine using the wizard, your virtual machine’s storage medium must support Read-Write-Many (RWX) PVCs. • Network Interface Cards (NICs) and storage disks can be created and attached to virtual machines after they have been created. Create ephemeral Fedora VM Figure 6: New VM from Container Image in internal registry 27
- 33. Create VM - General: Source PXE • Provision virtual machine from PXE menu. • Requires a PXE-capable NIC in the cluster. URL Provision virtual machine from an image available from an HTTP or S3 endpoint. Ex: http://server/image.qcow2 It is possible to configure basic authentication using a secret and specify custom TLS certificates in a ConfigMap. Container Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Ex: image-registry.openshift-image- registry.svc:5000/virtual-machines/windows2016 Disk Provision virtual machine from a disk. Ex: Select an existing PVC in the Storage tab. Source Figure 7: Internal Registry Container Note: for an external container registry - you need to set up a secret with the credentials. Ex: oc create secret generic <secret_name> --from-file=.dockercfg=<.dockercfg> --type=kubernetes.io/dockercfg Source URL Figure 8: HTTP Source URL 28
- 34. Create VM - General: Operating System The following operating systems are supported (64-bit x86 only) in OpenShift Virtualization 2.4. Linux • Fedora 31 or higher (desktop). • Red Hat Enterprise Linux 6.x, 7.x, 8.x. Linux systems need to be built with VirtIO drivers, cloud-init and SSH enabled. Windows • Windows 10 (desktop). • Windows Server 2012 R2, 2016, 2019. Windows images need to be built with VirtIO drivers, sysprep, cloud-init and RDP enabled. When installing Windows from a DVD, you need to use a SATA disk. Workload Profile • High Performance • Server • Desktop Create Virtual Machine: Operating System Figure 9: Supported Operating Systems Notes: • OpenShift Virtualization is certified in Microsoft’s Windows Server Virtualization Validation Program (SVVP) to run Windows Server workloads on Red Hat Enterprise Linux CoreOS 8 workers. • Other operating system templates shipped with OpenShift Virtualization are not supported. 29
- 35. Create VM - Networking Attach one or more network interface Network interface options: • Model: VirtIO (requires drivers, fastest), e1000, e1000e, ne2kPCI, pcnet, rtl8139 (various simulated drivers) • Network: Select a Network Definition in OpenShift. • Type: bridge, masquerade, sriov 30
- 36. Creating additional networks Edit the networks operator oc edit networks.operator.openshift.io cluster Configure additional networks additionalNetworks: - name: test-network-1 namespace: default rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-1", "type": "bridge", "master":"bond0", "ipam" :{"type":"static", "addresses":[{"address":"161.156.177.60/28"}]}}' type: Raw - name: test-network-2 namespace: default rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-2", "type": "bridge", "master":"bond0", "ipam" :{"type":"static", "addresses":[{"address":"161.156.177.61/28"}]}}' type: Raw 31
- 37. Storage: Add Disk Disks Source One of: Blank, URL, Container (registry image), Attached Cloned Disk (PVC), Attach Disk (PVC). Size GiB when using Blank, URL or Attached Cloned Disks. Container will be dynamic, and Attach Disk - the size of the PVC. Interface VirtIO, sata or scsi. Pick VirtIO when guest image has VirtIO drivers installed. Adding a blank disk Storage Class The StorageClass that is used to create the disk. Ex: ibmc-block-bronze, ibmc-file-silver (as configured in ROKS). See: oc get storageclass. 32
- 38. Storage: Advanced Disks: Avanced Options Figure 10: Add Disk: Advanced Options Volume Mode • Filesystem: Stores the virtual disk on a filesystem-based volume. • Block: Stores the virtual disk directly on the block volume. Only use Block if the underlying storage supports it. Access Mode • Single User (RWO): The disk can be mounted as read/write by a single node. • Shared Access (RWX): The disk can be mounted as read/write by many nodes. This is required for some features, such as live migration of virtual machines between nodes. • Read Only (ROX): The disk can be mounted as read-only by many nodes. 33
- 39. Storage: Virtual machine storage volume types (persistent) persistentVolumeClaim • Attaches an available PV to a virtual machine. • Attaching a PV allows for the virtual machine data to persist between sessions. • Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual machine instance is the recommended method for importing existing virtual machines. dataVolume • DataVolumes build on the persistentVolumeClaim disk type by managing the process of preparing the virtual machine disk via an import, clone, or upload operation. • VMs that use this volume type are guaranteed not to start until the volume is ready. • Specify type: dataVolume or type: ””. If you specify any other value for type, such as persistentVolumeClaim, a warning is displayed, and the virtual machine does not start. 34
- 40. Storage: Virtual machine storage volume types (ephemeral) ephemeral • A local copy-on-write (COW) image that uses a network volume as a read-only backing store. The backing volume must be a PersistentVolumeClaim. • The ephemeral image is created when the virtual machine starts and stores all writes locally and is discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not mutated in any way. containerDisk • References an image, such as a virtual machine disk, that is stored in the container image registry. • The image is pulled from the registry and embedded in a volume when the virtual machine is created. • A containerDisk volume is ephemeral and is discarded when the VM is stopped, restarted, or deleted. • Container disks are not limited to a single virtual machine and are useful for creating large numbers of virtual machine clones that do not require persistent storage. • Only RAW and QCOW2 formats are supported. QCOW2 recommended for reduced image size. emptyDisk • Creates an additional sparse QCOW2 disk that is tied to the life-cycle of the virtual machine interface. Used to store data that otherwise exceeds the limited temporary file system of an ephemeral disk. • The data survives guest-initiated reboots in the virtual machine but is discarded when the virtual machine stops or is restarted from the web console. 35
- 41. Storage: Virtual machine storage volume types (supporting) cloudInitNoCloud Attaches a disk that contains the referenced cloud-init NoCloud data source, providing user data and metadata to the virtual machine. A cloud-init installation is required inside the virtual machine disk. Example: - cloudInitNoCloud: userData: | #!/bin/bash echo "fedora" | passwd fedora --stdin name: cloudinitdisk 36
- 42. Advanced: Cloud-init Post-provisioning script used to setup users, passwords, insert SSH keys and customize the image. Example cloud-init #cloud-config users: - default - name: cmihai gecos: Mihai Criveti selinux-user: staff_u groups: users,wheel ssh_pwauth: True ssh_authorized_keys: - ssh-rsa AA..vz [email protected] chpasswd: list: | root:password cmihai:password expire: False 37
- 43. Advanced: Virtual Hardware (CD-ROM) ISO CD-ROM image Figure 11: Add a CD-ROM ISO Image from URL Boot order You will need to change the boot order if you wish to boot from the CD by editing the YAML: oc edit vm example devices: disks: - bootOrder: 1 cdrom: bus: sata name: cd-drive-1 38
- 45. Building a Windows Image from ISO Download container-native-virtualization/virtio-win - Red Hat Container Catalog. podman login registry.redhat.io podman pull registry.redhat.io/container-native-virtualization/virtio-win Create a image disk at least 15GB in size qemu-img create -f qcow2 w2016.qcow2 15G Install Windows using virt-install virt-install --connect qemu:///system --name ws2016 --ram 4096 --vcpus 2 --network network=default,model=virtio --disk path=ws2016.qcow2,format=qcow2,device=disk,bus=sata --cdrom Windows_Server.ISO --disk path=virtio-win-0.1.189.iso,device=cdrom --vnc --os-type windows --os-variant win2k16 Installation • Install QEMU guest agent and VirtIO Drivers • Configure RDP • Install Cloud Init and Sysprep the image 39
- 46. Building a Red Hat OS Image with Kickstart Create a Kickstart file (response file) You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can use this as part of a CI/CD OS build. Kickstart install the OS # virt-install --name guest1-rhel7 --memory 2048 --vcpus 2 --disk size=8 --location http://example.com/path/to/os --os-variant rhel7 --initrd-inject /path/to/ks.cfg --extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8" Setup QEMU guest agent on virtual machines systemctl enable qemu-guest-agent 40
- 47. Processing and converting images Compress the image qemu-img convert -O qcow2 -c w2016.qcow2 windows2016.qcow2 qemu-img convert -O qcow2 -c r7.qcow2 rhel7.qcow2 Create a SHA256 for your images This is optional, but good practice when uploading your images to a webserver, etc. sha256sum *qcow2 > SHA256SUMS Optionally, sign your image with GPG. gpg --sign myfile 41
- 48. Creating and pushing images to the container registry Create a Dockerfile FROM scratch ADD windows2016.qcow2 /disk/ Create a docker container podman build -t cmihai/windows2016 . Login to the container registry REGISTRY="$(oc get route/default-route -n openshift-image-registry -o=jsonpath='{.spec.host}')" podman login ${REGISTRY} Tag and push the image to your desired namespace (ex: virtual-machines) podman tag localhost/virtual-machine/fedora31 ${REGISTRY}/virtual-machines/windows2016 podman push ${REGISTRY}/virtual-machines/windows2016 42
- 49. Creating container images with Buildah Create a Dockerfile in /tmp/vmdisk cat << END > Dockerfile FROM kubevirt/container-disk-v1alpha ADD fedora32.qcow2 /disk END Build and push to registry buildah bud -t vmidisk/fedora32:latest /tmp/vmdisk buildah push --tls-verify=false vmidisk/fedora32:latest docker://cdi-docker-registry-host.cdi/fedora28:latest 43
- 50. Import the registry image into a Data volume YAML apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: fedora31image spec: source: registry: url: "docker://image-registry.openshift-image-registry.svc:5000/ virtual-machines/fedora31" pvc: accessModes: - ReadWriteMany resources: requests: storage: 20Gi Get the image info oc apply -f datavolume.yaml oc get pvc, dvs, pods # look for importer-fedora31image 44
- 51. Uploading local disk images by using the virtctl tool Creating an upload DataVolume YAML apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: <upload-datavolume> spec: source: upload: {} pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi> Create the volume oc create -f <upload-datavolume>.yaml Upload the image virtctl image-upload dv <volume_name> --size=2G --image-path=</path/to/image> Verify that a DataVolume was created View all DataVolume objects oc get dvs 45
- 52. Automating image builds with Kickstart and Ansible
- 53. Building a Red Hat OS Image with Kickstart Create a Kickstart file (response file) You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can use this as part of a CI/CD OS build. Kickstart install the OS # virt-install --name guest1-rhel7 --memory 2048 --vcpus 2 --disk size=8 --location http://example.com/path/to/os --os-variant rhel7 --initrd-inject /path/to/ks.cfg --extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8" Setup QEMU guest agent on virtual machines systemctl enable qemu-guest-agent 46
- 54. Kickstart File RHEL 8 KickStart ignoredisk --only-use=sda clearpart --none --initlabel text repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream cdrom keyboard --vckeymap=us --xlayouts='us' lang en_US.UTF-8 ... part /boot --fstype="xfs" --ondisk=sda --size=512 part / --fstype="xfs" --ondisk=sda --size=15360 part /home --fstype="xfs" --ondisk=sda --size=10240 %post sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers /bin/echo 'UseDNS no' >> /etc/ssh/sshd_config 47
- 55. IBM Cloud: VPC Gen 2 Custom Images
- 56. VPC Gen2 Image Format Custom Image Support for VPC Gen 2 You can create your own custom image, and import it to IBM Cloud™ Virtual Private Cloud infrastructure from IBM Cloud Object Storage. Then, you can use your custom image to create new virtual server instances that run on the KVM hypervisor. This is the same type of image used for OpenShift Virtualization, OpenStack, KVM, RHV, etc. Requirements • Contains a single file or volume • Is in qcow2 format • Is cloud-init enabled • The operating system is supported as a stock image operating system (including RHEL 7/8, Windows 2012 R2 and 2016) • Size doesn’t exceed 100 GB 48
- 59. Packer: build multiple images from a single source 50
- 60. Packer: Variables Variables to parametrized builds and hide secrets { "variables": { "my_secret": "{{env `MY_SECRET`}}", "not_a_secret": "plaintext", "foo": "bar" }, "sensitive-variables": ["my_secret", "foo"], } 51
- 61. Packer: Builders KVM Builder (qemu) "builders": [{ "type": "qemu", "accelerator": "kvm", "format": "qcow2", "disk_interface": "virtio-scsi", "boot_command": [ "<up><wait><tab>", " text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ {user `vm_name`}}.cfg net.ifnames=0", "<enter><wait>" ], "vm_name": "{{split build_type "-" 0}}-{{user `vm_name`}}", "iso_urls": [ "{{user `iso_local_url`}}", "{{user `iso_download_url`}}" ], "iso_checksum": "{{user `iso_sha256`}}", "output_directory": "{{user `builds_dir`}}/{{split build_type "-" 0}} -{{user `vm_name`}}", }] 52
- 62. Provisioners: run post-install tasks Chaining multiple provisioners "provisioners": [ { "type": "shell", "script": "setup.sh" }, { "type": "ansible", "playbook_file": "{{user `playbook_file`}}" }], 53
- 63. Post-processors: compress or upload your image Compress, post-process and upload the results { "post-processors": [ { "type": "compress", "format": "tar.gz" }, { "type": "upload", "endpoint": "http://example.com" } ] } 54
- 64. Building a VirtualBox image for RHEL 8 using Kickstart 55
- 65. Layered Image Compliance with OpenSCAP
- 67. Automatic Remediation as shell, ansible or puppet 57
- 68. Continuous Inspection and Automated Compliance Install OpenSCAP dnf install openscap-scanner Generate a report sudo oscap xccdf eval --report report.html --profile xccdf_org.ssgproject.content_profile_pci-dss /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml 58
- 69. Cheet Sheet
- 70. Get info oc get vms 59
- 71. Get virt-launcher details Debug virt-launcher and look for the mounted storage ls run/kubevirt/container-disks mount | grep disk 60