Extended and embedding: containerd update & project use cases

Extending and embedding:
containerd project use cases
A 2020 FOSDEM containerd project update
Phil Estes
Distinguished Engineer & CTO, IBM Cloud Platform
CNCF containerd project maintainer

What is containerd
● A “Container runtime”
○ Below platforms (Docker, Kubernetes)
○ Above lower level runtimes (runc, Kata, Firecracker, gVisor)
● Resource Manager
○ Container processes
○ Image artifacts
○ Filesystem snapshots
○ Metadata and dependencies
● Tightly scoped
○ 100% maintainer approval required to increase scope
○ Built-in CRI plugin only scope increase

State of containerd
● 5th project to graduate within the CNCF - February 2019
● Broad support and contribution from across the ecosystem
○ Over 200 individual contributors; represent > 100 companies
○ 13 maintainers represent 9 different companies
● All major cloud providers using containerd
● Supports Linux and Windows platforms, multiple architectures
● Added sub-projects to governance (Rust-based ttrpc; image encryption)

containerd 1.3
● Windows support for shim V2 API
● Device mapper snapshotter (Amazon Firecracker team contribution)
● New plugin interface for processing layers (encryption, compression)
● (CRI) Support for per-pod container shim

In progress
● Remote snapshotter for sharing snapshots in a cluster
● cgroups v2
● Windows CRI
● Mount and resource management
● Image encryption

Who is using containerd?
● Public Clouds
● Kubernetes Infra
● End Users
● DevOps Tools
● Custom Sandboxes

How is containerd used?
● Library
○ Go client API
■ oras, BuildKit, Weaveworks Ignite, IBM Cloud Functions, OpenFaaS
“faasd”, Alibaba PouchContainer
○ Extensibility
■ Amazon ECR resolver, Azure Teleport, remote snapshotters [cvmfs, stargz]
○ Imports/Subprojects (cri-o use of containerd/cgroups)
● Kubernetes Runtime
○ CRI-containerd
■ IBM Kubernetes Service, GKE, Ticketmaster, Alibaba, microk8s, KinD, k3s,
AWS Fargate
● Daemon
○ Docker, BuildKit

API - CRI
- CRI gRPC API exposed from containerd
- Kubelet can be configured to use containerd as runtime

API - containerd
- gRPC API, used by Go client
- Low level access to components
- Mirrors internal component interfaces
- Snapshots, Content, Containers, Task, Events, etc

Plugins - Backend
- No re-compilation required
- Proxy plugins for content store and snapshotters
- Runtime shims are separate binaries
implementing shim interface

Plugins - Client
1. Override services with service
options
2. Customize push and pull with
remote options
type RemoteOpt
func WithImageHandler(h images.Handler) RemoteOpt
func WithImageHandlerWrapper(w func(images.Handler) images.Handler) RemoteOpt
func WithResolver(resolver remotes.Resolver) RemoteOpt
type ServicesOpt
func WithContainerService(containerService containersapi.ContainersClient) ServicesOpt
func WithContentStore(contentStore content.Store) ServicesOpt
func WithDiffService(diffService diff.DiffClient) ServicesOpt
func WithEventService(eventService EventService) ServicesOpt
func WithImageService(imageService imagesapi.ImagesClient) ServicesOpt
func WithLeasesService(leasesService leases.Manager) ServicesOpt
func WithNamespaceService(namespaceService namespacesapi.NamespacesClient) ServicesOpt
func WithSnapshotters(snapshotters map[string]snapshots.Snapshotter) ServicesOpt
func WithTaskService(taskService tasks.TasksClient) ServicesOpt
2
1

Plugins - custom containerd binary
● Add a ﬁle with import to
cmd/containerd/ in your fork.
● Create your own main.go of
containerd
package main
import (
"fmt"
"os"
"github.com/containerd/containerd/cmd/containerd/command"
// import built-in plugins from cmd/containerd/builtins.go
_ "github.com/mygithub/customplugin"
)
func main() {
app := command.App()
if err := app.Run(os.Args); err != nil {
fmt.Fprintf(os.Stderr, "containerd: %sn", err)
os.Exit(1)
}
}

Example Snapshotter Proxy Plugin
// Snapshot service manages snapshots
service Snapshots {
rpc Prepare(PrepareSnapshotRequest) returns (PrepareSnapshotResponse);
rpc View(ViewSnapshotRequest) returns (ViewSnapshotResponse);
rpc Mounts(MountsRequest) returns (MountsResponse);
rpc Commit(CommitSnapshotRequest) returns (google.protobuf.Empty);
rpc Remove(RemoveSnapshotRequest) returns (google.protobuf.Empty);
rpc Stat(StatSnapshotRequest) returns (StatSnapshotResponse);
rpc Update(UpdateSnapshotRequest) returns (UpdateSnapshotResponse);
rpc List(ListSnapshotsRequest) returns (stream ListSnapshotsResponse);
rpc Usage(UsageRequest) returns (UsageResponse);
}
- implement Snapshotter gRPC API
- backend requests are proxied to plugin

External snapshotter
● Conﬁgure with proxy_plugins
● Build as an external plugin
[proxy_plugins]
[proxy_plugins.customsnapshot]
type = "snapshot"
address = "/var/run/mysnapshotter.sock"
package main
import(
"net"
"log"
"github.com/containerd/containerd/api/services/snapshots/v1"
"github.com/containerd/containerd/contrib/snapshotservice"
)
func main() {
rpc := grpc.NewServer()
sn := CustomSnapshotter()
service := snapshotservice.FromSnapshotter(sn)
snapshots.RegisterSnapshotsServer(rpc, service)
// Listen and serve
l, err := net.Listen("unix", "/var/run/mysnapshotter.sock")
if err != nil {
log.Fatalf("error: %vn", err)
}
if err := rpc.Serve(l); err != nil {
log.Fatalf("error: %vn", err)
}
}

Runtime shim v2 API
● Minimal and scoped to the execution lifecycle of a container
● Binary naming convention
○ Type io.containerd.runsc.v1 -> Binary containerd-shim-runsc-v1

Runtime Plugins - Task Service
service Task {
rpc State(StateRequest) returns (StateResponse);
rpc Create(CreateTaskRequest) returns (CreateTaskResponse);
rpc Start(StartRequest) returns (StartResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
rpc Pids(PidsRequest) returns (PidsResponse);
rpc Pause(PauseRequest) returns (google.protobuf.Empty);
rpc Resume(ResumeRequest) returns (google.protobuf.Empty);
rpc Checkpoint(CheckpointTaskRequest) returns (google.protobuf.Empty);
rpc Kill(KillRequest) returns (google.protobuf.Empty);
rpc Exec(ExecProcessRequest) returns (google.protobuf.Empty);
rpc ResizePty(ResizePtyRequest) returns (google.protobuf.Empty);
rpc CloseIO(CloseIORequest) returns (google.protobuf.Empty);
rpc Update(UpdateTaskRequest) returns (google.protobuf.Empty);
rpc Wait(WaitRequest) returns (WaitResponse);
rpc Stats(StatsRequest) returns (StatsResponse);
rpc Connect(ConnectRequest) returns (ConnectResponse);
rpc Shutdown(ShutdownRequest) returns (google.protobuf.Empty);
}

Extended and embedding: containerd update & project use cases

More Related Content

What's hot (20)

Similar to Extended and embedding: containerd update & project use cases (20)

More from Phil Estes (18)

Recently uploaded (20)

Extended and embedding: containerd update & project use cases