Kubernetes on the Edge / 在邊緣的K8S
- 1. 在邊緣的K8S Kubernetes on the Edge Stevennick
- 2. About Me •Stevennick •A programmer for: ▪Web ▪Video Streaming ▪Backend ▪System software •Work @ ITRI 2
- 3. Agenda •Challenges on Edge Computing •Proposed Edge Environment •Connect with the Cloud •Conclusion •Visit project site for more information: https://itrix-edge.github.io 3
- 4. Challenges on Edge Computing 4
- 5. New Chance of Edge Computing •Why Edge Computing? 1. Immediacy 2. Network bandwidth 3. Communication delay 4. Data security or policies Source: https://www.synopsys.com 5
- 6. Principle of Edge Computing 6 Source: https://www.alibabacloud.com/tc/knowledge/what-is-edge-computing
- 7. Principle of Edge Computing 7 Source: https://www.alibabacloud.com/tc/knowledge/what-is-edge-computing
- 8. Principle of Edge Computing 8 IoT Class Server Class
- 9. Principle of Edge Computing 9 IoT Class Server Class
- 10. Lacks of Edge Computing 10 •Non-standard installation •Totally different application deployment flow for Edge and Cloud •Difficult implementation for high availability support •Higher application maintenance costs
- 11. Lacks of Edge Computing 11 •How about use Kubernetes on the Edge? ▪Simplify infrastructure software stack ▪Similar deployment flow for cloud & edge ▪Application levels orchestration and HA support ▪Possible applications self-maintenance
- 12. Challenge of Bring Applications to the Edge Installation Configuration Deployment Management 12
- 13. Challenge of Bring Applications to the Edge •Installation ▪Environment limits, power, ethernet, etc. •Deployment ▪Application lifecycle, rolling update, rollback… •Configuration ▪OS, networking, device setup, backend synchronization, etc. •Management ▪Resource monitoring, audit trail, event management… 13
- 14. Challenge of Bring Applications to the Edge •Installation ▪Environment limits, power, ethernet, etc. •Deployment ▪Application lifecycle, rolling update, rollback… •Configuration ▪OS, networking, device setup, backend synchronization, etc. •Management ▪Resource monitoring, audit trail, event management… 14 The complexity with a large number of edges
- 15. Installation Considerations •Server-level clusters cost more power, bandwidth, and require a controlled environment ▪E.g., Cabinet servers •Tiny node reduces resource requirements and tolerance environment but with limited ability. May tolerate extreme environments ▪E.g., Single board computer •We focus on GPU powered node-level cluster ▪Smaller but enough power for edge applications ▪Good for clusters configuration 15
- 16. Configuration Considerations 16 •Containerize applications make more effective ways to manage OS & platform libraries ▪Split OS & application layer, reduce configuration manage costs •However, there still exists lots of work to do to make OS/container engine work: ▪Device driver maintenance ▪Networking setup ▪Device provisioning ▪more… •OOBM Controller and automation tools to help execute those tasks with unattended
- 17. Deployment Considerations 17 •Container orchestrator can help us deploy & manage applications inside the cluster •Version control deployment provide deploy tracking & rollback ability •Offline deployment may require for unstable network environment
- 18. Management Considerations •Monitoring tools involved for different facts of system management: ▪Hardware: events, fails ▪System: resource management, device status, logging ▪Application: resource usage, habit •Logging & debug trace tools for system and application tracking •Audit trail for policies and compliance •Certificates and trusts management help us prevent unauthorized operations 18
- 19. Challenge of Bring Applications to the Edge •Infrastructure maintenance •Application orchestration maintenance •Configuration & rollback •Factory reset •Network issue Deploy/Maintenance/Monitoring… HW failure/unreachable… Upgrade/patch/fixes 19
- 21. Edge Appliance for the edge •GPU Compute Node for edge applications •OOBM Controller provides physical compute node control •Internal network connection linked with 1Gbps Ethernet •Preset firewall rules on Internal Gateway for OOBM remote access 21
- 22. Reference Hardware Architecture •This demonstrates Jetson TX2 based reference hardware architecture ▪ Raspberry Pi as OOBM controller ▪ Compute node consists of Jetson TX2 ▪ Software also tested and passed on AGX Xavier, TX2 and Jetson Nano •More reliable components are required for production uses 22
- 23. Reference Edge Platform 23 ITRIX-EDGE TX2 Development Environment: 4 Compute Nodes
- 24. Reference Software Stack: Compute Node •Provisioning tool handles only OS, container runtime and Kubernetes •Edge maintenance tasks & support services are containerized, managed by Kubernetes •Application support services are also containerized and move on top of container runtime 24 OS KubernetesProvision Client Container Runtime Containerized Applications
- 25. Additional Components on top of the Node 25 •Application deployment agent ▪Run application deployments from remote •Container registry backup service ▪Ensure successful deployment an unstable network •Network connectivity service ▪Handles available external services and outbound IP for applications •Monitoring service
- 26. Application Registration Flow 26 1. Push application container images to edge accessible registry 2. Prepare deploy configuration files ▪ Deployments, Services, etc... 3. Register configurations with edge agent ▪ Once registration success, a hook key will return for this app deployment configuration 4. Use hook URL with given key starts app deployment
- 27. Application deployment flow 27 Model Training Pipeline 2. Trigger ImageShared folder Shared Repository Inference on Edge Devices A B 3.Pull 4. Inference
- 28. Connect with the Cloud 28
- 29. Cloud Integration for the Edge Application •Specialized container used for cloud directly integration ▪ Below are two integration samples for Azure Cloud and AWS ▪ This capability provides application migration possibility, suitable for requiring HA feature edge applications 29
- 30. Cloud Integration for the Edge Application •Use standard management tools for edge clusters •Choose cloud management tools for edge cluster management 30
- 31. Conclusion 31
- 32. Summary •Edge applications have totally different scale, from tiny devices to server-class clusters •Simple tasks become complex once the single deployment affects hundreds of targets •Duplicated resources enable error tolerable chances •Containerize, automation and disaster recovery mechanism helps developer life better 32
- 33. Summary 33 •New challenges for Edge K8S: ▪Solution design changes ▪Huge clusters management ▪Outbound network management integration
- 34. Reference •https://github.com/openbmc/docs •https://kubernetes.io/docs/home/ •https://github.com/NVIDIA/egx-platform •https://docs.aws.amazon.com/greengrass/latest/ developerguide/run-gg-in-docker-container.html •https://www.volterra.io/resources/blog/managin g-thousands-of-edge-kubernetes-clusters-with- gitops •Visit project site for more information: https://itrix-edge.github.io 34
- 35. Thank you! Q & A 35