Linux Integrity Mechanisms - Protecting Container Runtime as an example
0 likes703 views
This document discusses using Linux Integrity Mechanisms (IMA) to protect container runtime executables from tampering. It begins with an overview of container technology and a motivating example attack (CVE-2019-5736). It then provides background on IMA and how it can be used to measure and appraise the integrity of container runtime executables. Specifically, it details using IMA rules and digital signatures to enforce that the runc binary has not been tampered with. This helps prevent attacks that aim to overwrite the runc binary and escape the container.
![# cat policy2 > /sys/kernel/security/ima/policy
# cat /sys/kernel/security/ima/ascii_runtime_measurements
[root@goldenarmor ima-policy]# cat /sys/kernel/security/ima/ascii_runtime_measurements
10 1d8d532d463c9f8c205d0df7787669a85f93e260 ima-ng sha1:0000000000000000000000000000000000000000 boot_aggregate
10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.5cBnGN
10 d8ad76de60ab2242cf08bdc1b8bfd0ede36bc983 ima-ng sha1:a8c05f516f368a0654e9e3ae3b3217b6dd40e491 /tmp/sh-thd.5cBnGN
10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.ccEXk2
10 fb5a5e532147ef93b494ae391022620cf8e2c78b ima-ng sha1:a8c05f516f368a0654e9e3ae3b3217b6dd40e491 /tmp/sh-thd.ccEXk2
10 37a4d23f6ba7f2d5c13747f463f9ff1a2f155be8 ima-ng sha1:359819795e404a69d8a4a52dddccdf2bf1d3859e /usr/bin/gawk
10 a791d8db0076287a372053cbf0882feeea70b57e ima-ng sha1:cf13787cf9f3961d8c94eee05b75727c862cc986 /usr/lib64/ld-2.28.so
10 032d456298ce54f8f4bcd50bd3ddfbe73a16abb7 ima-ng sha1:e337d4fefc5c23e89c57da45bdc48cefed10d547 /etc/ld.so.cache
10 d4bc023942ec7eb0f9093502b7d2234252b2f143 ima-ng sha1:8cb57415f34031b50a7193c6a0176f1a74f2d03c /usr/lib64/libsigsegv.so.2.0.4
10 b7e0667a05d829ac3c3798d9fecc2b8c1921f059 ima-ng sha1:bb4c4b67723c732f12352e5449da1c1f0d2123d7 /usr/lib64/libreadline.so.7.0
10 d085818f8e9e0e7a8d2c24bc2e2b2e5af2d7c426 ima-ng sha1:e028f33771036ece3505a586b4f562992e188996 /usr/lib64/libtinfo.so.6.1
10 fc48f8598aae7723d0c9f7e3035f2eb900d772f0 ima-ng sha1:9403913e901469a4c97219ac9daf2fdb701bb483 /usr/share/locale/locale.alias
10 69f01172cf5cb9c2f6d5c17f3f065f2de924fee7 ima-ng sha1:692a1e79d6e453c99714ec37bb4999abb57f5b44 /usr/lib/locale/en_US.utf8/LC_CTYPE
10 77a0b37e3bc8d8fe55eb1ad3a8fa6f108be73bab ima-ng sha1:a82fb20c382fc9e5dec432ca5ee2f04684536e94 /usr/lib64/gconv/gconv-modules.cache
10 1393afae5534259ab7ad4ee7a759643ccccafc5c ima-ng sha1:aa5248b0ac085665477b2feb1266205b3bed557d /usr/lib/locale/en_US.utf8/LC_ADDRESS
10 eb7c76027b0b21e55693619d884995916d140058 ima-ng sha1:a1a2eabc4a2a2da47d7a90d36b4d0826aac03c73 /usr/lib/locale/en_US.utf8/LC_NAME
...
10 d02d14d189677572ce53d868447b1c9961b9faa6 ima-ng sha1:c5b36af5bfba8854a4209bb6e905c1a7eb47caad /usr/lib64/libidn2.so.0.3.6
10 3477e8cb1563071d5b09695709e7e2d31f567e90 ima-ng sha1:e6dc3965b93be37385e783727fbb0dfdabd4c7c4 /usr/lib64/libattr.so.1.1.2448
10 ce4a6d8b052af0c8d9cff507552d3f5839d9871f ima-ng sha1:47dab9dcb38148a217773302a9384bcfa21ba974 /usr/lib64/libdevmapper.so.1.02
10 f0289ee791a681cf00f512a66351fc295c1d0507 ima-ng sha1:f7259220100a2377ea7fb6ccda3c3037c63debfe /usr/lib64/libssl.so.1.1.1c
10 942fe25b8dd8b66c0f906af71b4c986fb96a67b6 ima-ng sha1:1b533c9e2826d8652e133c979dab4906ec37fdce /usr/lib64/libcrypto.so.1.1.1c
10 0ca2d280e7e3b714caf850203322d7eaa8fae9b8 ima-ng sha1:e6f458475bb286dfdcd98171550fbb0c38917cc6 /usr/lib64/libjson-c.so.4.0.0
10 0e964e27ce222afe78a3e4a030a06ff7eca96048 ima-ng sha1:d3fa4ee3c0ef05ac5b3bf4126facceb7ca59dced /usr/lib64/libpcap.so.1.9.0
10 323e40c5dd45055ece2032058ace3d604e6bc4f8 ima-ng sha1:340c7b865b132a0ef0ea51985435861e347fa2ab /usr/lib64/libunistring.so.2.1.0
10 a042f42bc53ec00abaf8de4efe536b4e50ac3ea9 ima-ng sha1:dec7b77de0dab91f28e9d9f265775c04642d70d5 /usr/lib64/libsepol.so.1
10 8820e6b8ea6d401c8e287b66a8e11bf4b50f435c ima-ng sha1:1802538af67a8306fbf85e12a1a9e18589de475b /usr/lib64/libudev.so.1.6.11
10 7e6a73ee70e9829795945ad3d02d94048cc64f5c ima-ng sha1:442d6bbeb51f089ccb9a0f31e1cf79d9eb8cb42c /usr/lib64/libz.so.1.2.11
10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.YUyyzn
10 cfb6053df14de2785ef1caf7a77d8e84c7a2406d ima-ng sha1:5f30a9ee5335c8ef0aa17876ddc5d622daec8b15 /tmp/sh-thd.YUyyzn
10 0f60e2884367dee70864927ba04f2e657209355e ima-ng sha1:6207eaf514d465da23059e66759456dfe162349d /usr/bin/cat](https://image.slidesharecdn.com/coscup2020-claychang-200802134510/85/Linux-Integrity-Mechanisms-Protecting-Container-Runtime-as-an-example-11-320.jpg)
![# docker run cve
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
HAX2: argv: /proc/self/fd/3
HAX2: fd: 4
HAX2: res: 13, 0
# strings /usr/bin/runc | tail -n 2
.debug_gdb_scripts
cve-2019-5736
# docker run cve
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
ERRO[0000] Error removing container
6ed3c7ab94169e53352b3df9e461d4406d7851389948c4de39cc2cd0df7b4909 from runtime after
creation failed
Error: container create failed (no logs from conmon): EOF](https://image.slidesharecdn.com/coscup2020-claychang-200802134510/85/Linux-Integrity-Mechanisms-Protecting-Container-Runtime-as-an-example-15-320.jpg)
Linux Integrity Mechanisms - Protecting Container Runtime as an example
- 1. Linux Integrity Mechanisms - Protecting Container Runtime as an Example Clay Chang [email protected] 1-Aug-2020, COSCUP’20
- 2. Who Am I? • Master Technologist (Linux) at Hewlett Packard Enterprise (HPE) • Researcher at NTU • COSCUP’17 - Memory-Style Storage in Linux • COSCUP’19 - Remote Memory This talk is a view of my own!
- 3. Contents • Introduction to Container Technology • Motivating Example - CVE-2019-5736 • Background of Linux Integrity Measurement Architecture (IMA) • Protecting Container Runtime Executables with IMA
- 4. Container Technology Docker Engine (dockerd) Podman containerd runc Docker CLI Podman CLI CRI-O kubelet
- 5. CVE-2019-5736 aka Container Escape • Description: … allows attackers to overwrite the host runc binary (and consequently obtain host root access) by leveraging the ability to execute a command as root within one of these types of containers: (1) a new container with an attacker-controlled image, or (2) an existing container, to which the attacker previously had write access, that can be attached with docker exec. This occurs because of file-descriptor mishandling, related to /proc/self/exe. https://blog.dragonsector.pl/2019/02/cve-2019-5736-escape-from-docker-and.html
- 6. # git clone https://github.com/q3k/cve-2019-5736-poc # docker build -t cve . # strings /usr/bin/runc | tail -n 2 .gnu_debuglink .gnu_debugdata # docker run cve Emulate Docker CLI using podman. Create /etc/containers/ nodocker to quiet msg. HAX2: argv: /proc/self/fd/3 HAX2: fd: 4 HAX2: res: 13, 0 # strings /usr/bin/runc | tail -n 2 .debug_gdb_scripts cve-2019-5736
- 7. Integrity Measurement Architecture (IMA) • Since v2.6.30 (9 Jun 2009) • Key Features of IMA • Detect if files have been altered accidentally or maliciously, both remotely or locally • Appraise a file’s measurement against a “good” value stored as an extended attributes • Enforce local file integrity
- 8. IMA Characteristics • Policy-based rules • Actions: measure / dont_measure / appraise / dont_appraise / … • Conditions: • File-based operations: BPRM_CHECK / MMAP_CHECK / FILE_CHECK / … • Masks: MAY_EXEC / MAY_READ / MAY_WRITE / … • Specific fsmagic / fsuuid / uid / … • Supports LSM (Linux Security Module, e.g. SELinux): obj_type / obj_user / obj_role / … • Options: appraise_type / keyring / …
- 9. # PROC_SUPER_MAGIC dont_measure fsmagic=0x9fa0 dont_appraise fsmagic=0x9fa0 # SYSFS_MAGIC dont_measure fsmagic=0x62656572 dont_appraise fsmagic=0x62656572 # DEBUGFS_MAGIC dont_measure fsmagic=0x64626720 dont_appraise fsmagic=0x64626720 # TMPFS_MAGIC dont_measure fsmagic=0x01021994 dont_appraise fsmagic=0x01021994 # RAMFS_MAGIC dont_appraise fsmagic=0x858458f6 # DEVPTS_SUPER_MAGIC dont_measure fsmagic=0x1cd1 dont_appraise fsmagic=0x1cd1 # BINFMTFS_MAGIC dont_measure fsmagic=0x42494e4d dont_appraise fsmagic=0x42494e4d # SECURITYFS_MAGIC dont_measure fsmagic=0x73636673 dont_appraise fsmagic=0x73636673 # SELINUX_MAGIC dont_measure fsmagic=0xf97cff8c dont_appraise fsmagic=0xf97cff8c # CGROUP_SUPER_MAGIC dont_measure fsmagic=0x27e0eb dont_appraise fsmagic=0x27e0eb # NSFS_MAGIC dont_measure fsmagic=0x6e736673 dont_appraise fsmagic=0x6e736673 measure func=BPRM_CHECK measure func=FILE_MMAP mask=MAY_EXEC measure func=FILE_CHECK mask=MAY_READ uid=0 appraise fowner=0 Measure all executables in bprm_check 3 1 2 4 1 Measure all files mapped executable in file_mmap 2 Measure all files open for read by root in do_filp_open 3 Appraise all files owned by root4 IMA Policy Example
- 10. # cat policy > /sys/kernel/security/ima/policy # cat /sys/kernel/security/ima/ascii_runtime_measurements -bash: /usr/bin/cat: Permission denied # dmesg -bash: /usr/bin/dmesg: Permission denied # ls -bash: /usr/bin/ls: Permission denied # reboot -bash: /usr/bin/ls: Permission denied
- 11. # cat policy2 > /sys/kernel/security/ima/policy # cat /sys/kernel/security/ima/ascii_runtime_measurements [root@goldenarmor ima-policy]# cat /sys/kernel/security/ima/ascii_runtime_measurements 10 1d8d532d463c9f8c205d0df7787669a85f93e260 ima-ng sha1:0000000000000000000000000000000000000000 boot_aggregate 10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.5cBnGN 10 d8ad76de60ab2242cf08bdc1b8bfd0ede36bc983 ima-ng sha1:a8c05f516f368a0654e9e3ae3b3217b6dd40e491 /tmp/sh-thd.5cBnGN 10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.ccEXk2 10 fb5a5e532147ef93b494ae391022620cf8e2c78b ima-ng sha1:a8c05f516f368a0654e9e3ae3b3217b6dd40e491 /tmp/sh-thd.ccEXk2 10 37a4d23f6ba7f2d5c13747f463f9ff1a2f155be8 ima-ng sha1:359819795e404a69d8a4a52dddccdf2bf1d3859e /usr/bin/gawk 10 a791d8db0076287a372053cbf0882feeea70b57e ima-ng sha1:cf13787cf9f3961d8c94eee05b75727c862cc986 /usr/lib64/ld-2.28.so 10 032d456298ce54f8f4bcd50bd3ddfbe73a16abb7 ima-ng sha1:e337d4fefc5c23e89c57da45bdc48cefed10d547 /etc/ld.so.cache 10 d4bc023942ec7eb0f9093502b7d2234252b2f143 ima-ng sha1:8cb57415f34031b50a7193c6a0176f1a74f2d03c /usr/lib64/libsigsegv.so.2.0.4 10 b7e0667a05d829ac3c3798d9fecc2b8c1921f059 ima-ng sha1:bb4c4b67723c732f12352e5449da1c1f0d2123d7 /usr/lib64/libreadline.so.7.0 10 d085818f8e9e0e7a8d2c24bc2e2b2e5af2d7c426 ima-ng sha1:e028f33771036ece3505a586b4f562992e188996 /usr/lib64/libtinfo.so.6.1 10 fc48f8598aae7723d0c9f7e3035f2eb900d772f0 ima-ng sha1:9403913e901469a4c97219ac9daf2fdb701bb483 /usr/share/locale/locale.alias 10 69f01172cf5cb9c2f6d5c17f3f065f2de924fee7 ima-ng sha1:692a1e79d6e453c99714ec37bb4999abb57f5b44 /usr/lib/locale/en_US.utf8/LC_CTYPE 10 77a0b37e3bc8d8fe55eb1ad3a8fa6f108be73bab ima-ng sha1:a82fb20c382fc9e5dec432ca5ee2f04684536e94 /usr/lib64/gconv/gconv-modules.cache 10 1393afae5534259ab7ad4ee7a759643ccccafc5c ima-ng sha1:aa5248b0ac085665477b2feb1266205b3bed557d /usr/lib/locale/en_US.utf8/LC_ADDRESS 10 eb7c76027b0b21e55693619d884995916d140058 ima-ng sha1:a1a2eabc4a2a2da47d7a90d36b4d0826aac03c73 /usr/lib/locale/en_US.utf8/LC_NAME ... 10 d02d14d189677572ce53d868447b1c9961b9faa6 ima-ng sha1:c5b36af5bfba8854a4209bb6e905c1a7eb47caad /usr/lib64/libidn2.so.0.3.6 10 3477e8cb1563071d5b09695709e7e2d31f567e90 ima-ng sha1:e6dc3965b93be37385e783727fbb0dfdabd4c7c4 /usr/lib64/libattr.so.1.1.2448 10 ce4a6d8b052af0c8d9cff507552d3f5839d9871f ima-ng sha1:47dab9dcb38148a217773302a9384bcfa21ba974 /usr/lib64/libdevmapper.so.1.02 10 f0289ee791a681cf00f512a66351fc295c1d0507 ima-ng sha1:f7259220100a2377ea7fb6ccda3c3037c63debfe /usr/lib64/libssl.so.1.1.1c 10 942fe25b8dd8b66c0f906af71b4c986fb96a67b6 ima-ng sha1:1b533c9e2826d8652e133c979dab4906ec37fdce /usr/lib64/libcrypto.so.1.1.1c 10 0ca2d280e7e3b714caf850203322d7eaa8fae9b8 ima-ng sha1:e6f458475bb286dfdcd98171550fbb0c38917cc6 /usr/lib64/libjson-c.so.4.0.0 10 0e964e27ce222afe78a3e4a030a06ff7eca96048 ima-ng sha1:d3fa4ee3c0ef05ac5b3bf4126facceb7ca59dced /usr/lib64/libpcap.so.1.9.0 10 323e40c5dd45055ece2032058ace3d604e6bc4f8 ima-ng sha1:340c7b865b132a0ef0ea51985435861e347fa2ab /usr/lib64/libunistring.so.2.1.0 10 a042f42bc53ec00abaf8de4efe536b4e50ac3ea9 ima-ng sha1:dec7b77de0dab91f28e9d9f265775c04642d70d5 /usr/lib64/libsepol.so.1 10 8820e6b8ea6d401c8e287b66a8e11bf4b50f435c ima-ng sha1:1802538af67a8306fbf85e12a1a9e18589de475b /usr/lib64/libudev.so.1.6.11 10 7e6a73ee70e9829795945ad3d02d94048cc64f5c ima-ng sha1:442d6bbeb51f089ccb9a0f31e1cf79d9eb8cb42c /usr/lib64/libz.so.1.2.11 10 0000000000000000000000000000000000000000 ima-ng sha1:0000000000000000000000000000000000000000 /tmp/sh-thd.YUyyzn 10 cfb6053df14de2785ef1caf7a77d8e84c7a2406d ima-ng sha1:5f30a9ee5335c8ef0aa17876ddc5d622daec8b15 /tmp/sh-thd.YUyyzn 10 0f60e2884367dee70864927ba04f2e657209355e ima-ng sha1:6207eaf514d465da23059e66759456dfe162349d /usr/bin/cat
- 12. The Leverage & Solution • container-selinux: SELinux Policy for Container Runtimes • New IMA rule: # ls -Z /usr/bin/runc system_u:object_r:container_runtime_exec_t:s0 /usr/bin/runc appraise func=BPRM_CHECK obj_type=container_runtime_exec_t appraise_type=imasig
- 13. What is appraise_type=imasig? • Use digital signature of the hash of a file as the “good” value • Store the digital signature in the xattr (extended attribute) of the file # evmctl ima_sign -a sha256 -k privkey.pem /usr/bin/runc # getfattr -m "." -d /usr/bin/runc getfattr: Removing leading '/' from absolute path names # file: usr/bin/runc security.ima=0sAwIEH+VEdQEAkN3YSXNhBLf...UJQ8tk7...LhYUZVy5wmEmtBXSkBHP9yTZFg== security.selinux="system_u:object_r:container_runtime_exec_t:s0"
- 14. How IMA appraise the file with imasig? # keyctl describe %keyring:.ima 574740493: -lswrv--swrv------------ 0 0 keyring: .ima # evmctl import x509_ima.der 574740493 # keyctl show %keyring:.ima Keyring 574740493 ---lswrv 0 0 keyring: .ima 4225879 --als--v 0 0 _ asymmetric: IMA Signing Key: e52f68af018134bfa644cbda02b5c8ef1fe54475 • Load the public key onto the .ima keyring (*) • IMA uses the public keys loaded onto the .ima keyring to verify the digsig (security.ima) of the hash of the file being appraised (*) The public key must be signed by the trusted certificate in the kernel
- 15. # docker run cve Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. HAX2: argv: /proc/self/fd/3 HAX2: fd: 4 HAX2: res: 13, 0 # strings /usr/bin/runc | tail -n 2 .debug_gdb_scripts cve-2019-5736 # docker run cve Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. ERRO[0000] Error removing container 6ed3c7ab94169e53352b3df9e461d4406d7851389948c4de39cc2cd0df7b4909 from runtime after creation failed Error: container create failed (no logs from conmon): EOF
- 16. References • https://blog.dragonsector.pl/2019/02/cve-2019-5736-escape-from-docker- and.html • Integrity Verification of Docker Containers for a Lightweight Cloud Environment, Future Generation Computer System 2019 • Leveraging Kernel Security Mechanisms to Improve Container Security: a Survey, ARES’19 • Linux Integrity Subsystem Update by Mimi Zohar, Linux Security Summit 2019 • The Keylime Project, https://keylime.dev
- 17. Q/A
- 18. Backups
- 19. BPRM_CHECK execve syscall -> exec_binprm (fs/exec.c) -> search_binary_handler (fs/exec.c) -> security_bprm_check (security/security.c) -> ima_bprm_check (security/integrity/ima/ima_main.c) MMAP_CHECK mmap_pgoff sys call -> vm_mmap_pgoff (mm/util.c) -> security_mmap_file (security/security.c) -> ima_file_mmap (security/integrity/ima/ima_main.c) FILE_CHECK openat2 sys call -> do_filp_open (fs/namei.c) -> path_openat (ds/namei.c) -> do_open (fs/namei.c) -> ima_file_check (security/integrity/ima/ima_main.c)
- 20. IMA Keyring: .crs or _crs? • Kernel config CONFIG_INTEGRITY_TRUSTED_KEYRING determines whether .ima (trusted keyring) or _crs (regular keyring) is used for IMA • If .ima was used, only those keys signed by a “kernel-trusted certificate” can be loaded onto the .ima keyring for appraisal • We can use CONFIG_SYSTEM_EXTRA_CERTIFICATE to reserve space in the kernel for embedding own certificate to the kernel • Then use our own certificate to sign the IMA signing public key • Details refer to the “Generate trusted keys” in http://linux-ima.sourceforge.net/evmctl.1.html • Use scripts/insert-sys-certs to insert the certificate into the kernel