containerd and CRI
6 likes•8,780 views
The document details the Container Runtime Interface (CRI) for Kubernetes, which allows integration of various container runtimes like containerd while ensuring Kubernetes remains extensible. It outlines the lifecycle of pods and containers, logging requirements, and the challenges faced when using Docker as the primary runtime. CRI aims to enhance flexibility by providing imperative operations for container management and support for various image formats, particularly through ongoing development projects.
containerd and CRI
- 2. Kubernetes: Container Runtime Interface (CRI) ● A new plugin interface for container runtimes ○ RuntimeService, ImageService ● A refactoring of organically evolved code ● Make Kubernetes more extensible ○ Empower arbitrary 3rd party runtimes without sending us a PR kubelet CRI shim container runtimegrpc client containercontainer container container containergrpc server
- 3. Pod Sandbox ● Pod is composed of a group of application containers in an isolated environment with resource constraints ● Pod Sandbox is the environment ● Isolation: interpreted flexibly by container runtimes ○ Network namespace ○ Virtual machine ● Resource constraints ○ Pod-level cgroup ● Container runtime is completely responsible for network setup
- 4. Imperative Container Operations ● CRI provides knobs for kubelet to implement higher-level features ○ lifecycle hooks ○ liveness/readiness checks ○ container restarts & backoff ● Why imperative container-level operations? ○ Flexibility vs. Feature velocity ○ User-facing Kubernetes API is still declarative
- 5. Container stdout/stderr Logs ● Requirements ○ Better log/disk management ■ Decouple the lifecycle of logs and containers ○ Support kubernetes logging features (e.g., kubectl logs --since) ■ Need understanding of log formats ● Solution ○ Instruct the runtime to store logs at a given path ■ /var/log/pods/<podUID>/<containerName>_<instance#>.log ○ Ask runtime to decorate the logs with a standard format 2016-10-06T00:17:09.669794202Z stdout The content of the log entry 1 2016-10-06T00:17:10.113242941Z stderr The content of the log entry 2 2016-10-06T00:17:11.241390016Z stderr The content of the log entry 3
- 6. Image Service ● Kubernetes supports only Docker images (as of release 1.6) ● Basic image management operations ○ ListImage, ImageStatus, PullImage, RemoveImage ● Refer to the images by name or digest ● Runtime Service needs to be able to locate and use the image to create a root filesystem
- 7. Pod and Container Lifecycle ● PodSandbox rpc calls ○ RunPodSandbox, StopPodSandbox, RemovePodsandbox ● Container rpc calls ○ CreateContainer, StartContainer, StopContainer, RemoveContainer, Exec,... ● Life of a pod ○ Create the Pod Sandbox first, then add containers to it ○ RunPodSandbox -> CreateContainer -> StartContainer -> StopContainer -> StopPodSandbox -> RemoveContainer -> RemovePodSandbox
- 8. Recap ● CRI is … ○ a plugin interface to support a wide variety of container runtimes in kubernetes ● CRI is NOT… ○ an interface for a full-fledged, all-inclusive container runtime ○ a user-facing API
- 9. CRI and Docker Engine ● CRI had to build on top of the Docker engine API ○ Docker is the default, and the only container runtime supporting all kubernetes features ○ Ensuring a smooth transition ● This brings extra constraints and limitations
- 10. CRI and Docker Engine - Challenges ● Adding one more hop from kubelet to runc ○ kubelet -> CRI shim -> docker daemon -> containerd -> runc ● Extra bells and whistles that are not used by kubernetes ○ Volumes, networking, etc. ○ Larger surface to validate and maintain ● Limited support for other image formats ○ OCI
- 11. CRI and Docker Engine - Challenges (cont.) ● Stdout/stderr logs ○ The lifecycle of logs are coupled with containers ○ CRI log format is not supported ○ Log path is not configurable ○ Temporary solution: continue using the JSON log driver ■ Support the Docker JSON format in kubelet ■ Use symbolic links to access the logs
- 12. A Better Match: containerd ● Provides only the core functionality we need ● Designed to be embedded into a larger system ● No tight dependence between images and containers ● Supports OCI images ● Eliminates one hop (docker daemon) kubelet CRI shim containerd containercontainer container container container
- 13. Near-Term plans ● Figure out the mapping of containerd to CRI ○ Make sure it has everything kubernetes needs ○ Help fill any gaps ● Proof-of-concept CRI shim ○ Likely challenges: logs, exec ● Community presentation and discussion
- 14. Current Status of CRI ● Kubernetes 1.5: v1alpha1 API ○ Still evolving ○ Missing container stats API, etc. ● Kubernetes 1.6 (Mar. 2017): Kubelet using CRI by default ○ Deprecating the old Docker implementation in the next release ● Ongoing CRI-compatible runtime projects ○ cri-o: oci-conformant runtimes ○ dockershim: the built-in docker-CRI integration ○ frakti: hypervisor-based runtimes ○ rktlet: the rkt container runtime ○ virtlet: a VM (QCOW) runtime