Delivering Docker & K3s worloads to IoT Edge devices
0 likes•313 views
The document discusses the integration of Docker and K3s with NVIDIA Jetson Nano for IoT edge computing, highlighting its capabilities for AI applications and GPU-accelerated containerized environments. It provides installation instructions and workflows for deploying applications, including examples of real-time systems for mask detection and drone delivery services. Key features of K3s as a lightweight Kubernetes distribution and Docker's support for multi-architecture are also emphasized.
![Enabling GPU access with Compose
Build on Open Source
● Compose services can define GPU device reservations
services:
test:
image: nvidia/cuda:10.2-base
command: nvidia-smi
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu, utility]
ia-smi
● Bring up GPU-enabled Docker container
$ docker-compose up
Creating network "gpu_default" with the default driver
Creating gpu_test_1 ... done
Attaching to gpu_test_1
test_1 |
+-----------------------------------------------------
------------------------+
test_1 | | NVIDIA-SMI 450.80.02 Driver Version:
450.80.02 CUDA Version: 11.1 |
test_1 |
|-------------------------------+---------------------
-+----------------------+
======================================================
==============|](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-11-320.jpg)
![Support
for
Multi-arch
Platform
Docker Compose v2.0
$ docker compose --project-name demo up
[+] Running 3/4
⠿ Network demo_default Created
4.0s
⠿ Volume "demo_db_data" Created
0.0s
⠿ Container demo_db_1 Started
2.2s
⠿ Container demo_wordpress_1 Starting](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-22-320.jpg)
![Mask Detection System running on Jetson Nano
[Confidential]
- A Jetson Nano Dev Kit running JetPack 4.4.1 or 4.5
- An external DC 5 volt, 4 amp power supply connected through the Dev Kit's
barrel jack connector (J25). (See these instructions on how to enable barrel
jack power.) This software makes full use of the GPU, so it will not run with
USB power.
- A USB webcam attached to your Nano
- Another computer with a program that can display RTSP streams -- we
suggest VLC or QuickTime.
$ sudo docker run --runtime nvidia
--privileged --rm -it
--env MASKCAM_DEVICE_ADDRESS=<your-jetson-ip>
-p 1883:1883
-p 8080:8080
-p 8554:8554
maskcam/maskcam-beta](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-33-320.jpg)
![CherryBot Systems - AI-Powered Payload Delivery System
[Confidential]
- AI-based payload delivery robot
- Low-cost autonomous robot system
- Equipped with NVIDIA Jetson Nano
board, a low-powered AI deployed as
Edge device, a sensor suite that
includes multiple cameras, GPS and
swappable batteries
- Uses deep learning to correctly
interpret data gathered from its
sensors and to make intelligent
decisions that ensure a fast, safe and
cost efficient delivery.
- It can correctly identify objects or
people or detect objects and
obstacles to avoid collisions in a safe
reliable manner.](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-35-320.jpg)
![CherryBot Systems - AI-Powered Payload Delivery System
[Confidential]](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-36-320.jpg)
![CherryBot Systems
[Confidential]
Food Delivery Swag Distribution Medicine Delivery](https://image.slidesharecdn.com/meetupdeliveringcontainers-basedappstoiotedgedevices-210619053151/85/Delivering-Docker-K3s-worloads-to-IoT-Edge-devices-37-320.jpg)
Delivering Docker & K3s worloads to IoT Edge devices
- 1. Delivering Docker & K3s to IoT Edge device Ajeet S Raina
- 2. - Docker Captain - ARM Innovator - Author @ collabnix.com - Docker Community Leader - DevRel at Redis Labs - Worked in Dell, VMware & CGI $whoami @ajeetsraina
- 3. Cloud-Native on IoT Edge (Jetson Nano) - Cloud-Native technologies offer the flexibility and agility needed for rapid product development and continual product upgrades. - Jetson brings Cloud-Native to the edge and enables technologies like containers and container orchestration which revolutionized cloud applications. - NVIDIA JetPack includes NVIDIA Container Runtime with Docker integration, enabling GPU accelerated containerized applications on Jetson platform. - Developers can package their applications for Jetson with all its dependencies into a single container that is guaranteed to work in any deployment environment.
- 4. NVIDIA Jetson Nano Edge - AI Computer 2 GB / 4 GB $59 / $99
- 5. Jetson Software for AI Edge Device
- 6. Getting Started ● microSD card (32GB UHS-1 minimum recommended) - used as a boot device & for main storage ● USB keyboard and mouse ● Computer display (HDMI or DP) ● Micro-USB power supply ● SD Card image - https://developer.nvidia.com/jetson-nano-sd-card-image
- 8. Docker support for NVIDIA Jetson Nano Docker comes with Jetson Nano, by default
- 9. Docker on NVIDIA Jetson Nano Build on Open Source ● Install the latest version of Docker curl https://get.docker.com | sh && sudo systemctl --now enable docker ● Setting NVIDIA Container Toolkit distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list ● Install nvidia-docker2 package $ sudo apt-get install -y nvidia-docker2 $ sudo systemctl restart docker ● Running Ubuntu ARM container docker run -it arm64v8/ubuntu /bin/bash
- 10. Docker Compose support for NVIDIA Jetson Nano $ export DOCKER_COMPOSE_VERSION=1.27.4 $ sudo apt-get install libhdf5-dev $ sudo apt-get install libssl-dev $ sudo pip3 install docker-compose=="${DOCKER_COMPOSE_VERSION}" $ apt install python3 $ apt install python3-pip $ pip install docker-compose
- 11. Enabling GPU access with Compose Build on Open Source ● Compose services can define GPU device reservations services: test: image: nvidia/cuda:10.2-base command: nvidia-smi deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, utility] ia-smi ● Bring up GPU-enabled Docker container $ docker-compose up Creating network "gpu_default" with the default driver Creating gpu_test_1 ... done Attaching to gpu_test_1 test_1 | +----------------------------------------------------- ------------------------+ test_1 | | NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.1 | test_1 | |-------------------------------+--------------------- -+----------------------+ ====================================================== ==============|
- 12. If nvidia-smi is no longer available, use tegrastats/jtop $ sudo docker run --rm -it --gpus all -v /run/jtop.sock:/run/jtop.sock ajeetraina/jetson-stats-nano jtop
- 13. Checking the Jetson CUDA Information - Hardware - OS version - CUDA version - CUDA arch - OpenCV - TensorRT - Linux Kernel Version There is no support for CUDA 11 for Jetson board as of now CUDA is an API created by NVIDIA to talk to NVIDIA GPU devices
- 14. CUDA Compiler & Libraries CUDA is an API created by NVIDIA to talk to NVIDIA GPU devices
- 15. Interesting? How shall I build those apps from scratch? - -
- 16. A Generic Docker Workflow BUILD SHIP RUN
- 17. A Generic Docker Workflow BUILD SHIP RUN $ docker run ajeetraina/hellowhale:latest
- 18. Identifying the Docker Image Arch > terminal $ docker run --rm mplatform/mquery ajeetraina/hellowhale:latest Image: ajeetraina/hellowhale:latest (digest: sha256:50e5d8b034ff3a0d537224e332da0ee74e393df36acefa6859daba5 8712ad1f4) * Manifest List: No (Image type: application/vnd.docker.distribution.manifest.v2+json) * Supports: linux/amd64
- 19. A Generic Docker Workflow (Edge) BUILD SHIP RUN
- 20. A Generic Docker Workflow (Edge) BUILD SHIP RUN $ docker run hello-world Hello from Docker !
- 21. Support for Multi-arch Platform > terminal $ docker run --rm mplatform/mquery hello-world:latest Image: hello-world:latest (digest: sha256:5122f6204b6a3596e048758cabba3c46b1c937a46b5be6225b835d091b90e46c ) * Manifest List: Yes (Image type: application/vnd.docker.distribution.manifest.list.v2+json) * Supported platforms: - linux/amd64 - linux/arm/v5 - linux/arm/v7 - linux/arm64/v8 - linux/386 - linux/mips64le - linux/ppc64le - linux/s390x - windows/amd64:10.0.17763.1935
- 22. Support for Multi-arch Platform Docker Compose v2.0 $ docker compose --project-name demo up [+] Running 3/4 ⠿ Network demo_default Created 4.0s ⠿ Volume "demo_db_data" Created 0.0s ⠿ Container demo_db_1 Started 2.2s ⠿ Container demo_wordpress_1 Starting
- 23. Do I need ARM-based devices to build ARM-based images? Not necessarily.
- 24. Buildx - A CLI Plugin to build Multi-arch Docker Images BUILD SHIP RUN $ docker buildx build --platform linux/arm/v7, linux/arm/v8 -t
- 26. K3s - A Lightweight Kubernetes Build on Open Source ● Fully compliant Kubernetes distribution ● Easy to install, half the memory ● Lightweight storage backend based on sqlite3 ● Support etcd3, MySQL, PostgreSQL ● All in a binary of less than 100 MB ● Install K3s on Edge $ curl -sfL https://get.k3s.io | sh - ● Add worker nodes $ sudo curl -sfL https://get.k3s.io | K3S_URL=https://pico1:6443 K3S_TOKEN= **K3s support both Docker & Containerd as runtime
- 27. Running deviceQuery on Docker with GPU Support
- 28. Running deviceQuery on Docker with GPU Support
- 29. Running deviceQuery on Containerd with GPU Support
- 30. Running deviceQuery on K3 Cluster
- 31. Running deviceQuery on K3 Cluster
- 32. Smart Camera system for Real-Time Crowd Face Mask detection Docker + Edge AI + Camera Real-time Implementations
- 33. Mask Detection System running on Jetson Nano [Confidential] - A Jetson Nano Dev Kit running JetPack 4.4.1 or 4.5 - An external DC 5 volt, 4 amp power supply connected through the Dev Kit's barrel jack connector (J25). (See these instructions on how to enable barrel jack power.) This software makes full use of the GPU, so it will not run with USB power. - A USB webcam attached to your Nano - Another computer with a program that can display RTSP streams -- we suggest VLC or QuickTime. $ sudo docker run --runtime nvidia --privileged --rm -it --env MASKCAM_DEVICE_ADDRESS=<your-jetson-ip> -p 1883:1883 -p 8080:8080 -p 8554:8554 maskcam/maskcam-beta
- 35. CherryBot Systems - AI-Powered Payload Delivery System [Confidential] - AI-based payload delivery robot - Low-cost autonomous robot system - Equipped with NVIDIA Jetson Nano board, a low-powered AI deployed as Edge device, a sensor suite that includes multiple cameras, GPS and swappable batteries - Uses deep learning to correctly interpret data gathered from its sensors and to make intelligent decisions that ensure a fast, safe and cost efficient delivery. - It can correctly identify objects or people or detect objects and obstacles to avoid collisions in a safe reliable manner.
- 36. CherryBot Systems - AI-Powered Payload Delivery System [Confidential]
- 37. CherryBot Systems [Confidential] Food Delivery Swag Distribution Medicine Delivery
- 40. Thank You