Debugging Go in Kubernetes
3 likes1,665 views
1. The document discusses debugging Go applications running on Kubernetes. 2. It presents different approaches for debugging including running the debugger fully locally, fully remotely in Kubernetes, or using Telepresence to debug locally while the application runs in Kubernetes. 3. Examples are given for building debug Docker images, deploying them to Kubernetes, and using tools like Delve and Telepresence for interactive debugging.
![Build image:x.x.x-debug
Go App Static Binary
(built with `go build --gcflag "-N -l" ...`)
Delve Binary (COPY --from builder OR go get)
EXPOSE 2345 (debugger port)
Base Image (alpine/scratch OR golang)
CMD ["dlv", "exec", "--headless", "--api-version=2",
"--listen=:2345", "--log", "./todo-app"]](https://image.slidesharecdn.com/golangdebugk8s-190211181154/85/Debugging-Go-in-Kubernetes-14-320.jpg)
![# open VS Code editor
code .
# [code.terminal.#1]: swap todo-app deployment with bash
telepresence --namespace=todo-app --swap-deployment=todo-app
--expose 3000 --run bash
# [code.terminal.#1] run debugger (with .debug/runDelve.sh)
dlv debug --listen=localhost:2345
--headless=true --api-version=2 --log=true ./
-- -config-file="$TELEPRESENCE_ROOT/etc/todo-app/redis.config"
# [code.terminal.#2] invoke todo-app service in K8s cluster
http http://a8d6ad3e6169111e98c1d02b9ca3b647-624533560.us-
west-2.elb.amazonaws.com:9080/read/todo
# DEBUG SESSION ....... stop debug (dlv); exit (swap back)](https://image.slidesharecdn.com/golangdebugk8s-190211181154/85/Debugging-Go-in-Kubernetes-21-320.jpg)
Debugging Go in Kubernetes
- 1. Debugging Go on Kubernetes GopherCon 2019, Israel @alexeiled
- 3. Quick Survey 1. Who is using Docker today? 2. Who is using Kubernetes?
- 12. K8s Debug Models 100% local 100% remote Realism Low High Feedback Fast Slow Resource Footprint Heavy Lightweight
- 13. 100% Remote
- 14. Build image:x.x.x-debug Go App Static Binary (built with `go build --gcflag "-N -l" ...`) Delve Binary (COPY --from builder OR go get) EXPOSE 2345 (debugger port) Base Image (alpine/scratch OR golang) CMD ["dlv", "exec", "--headless", "--api-version=2", "--listen=:2345", "--log", "./todo-app"]
- 15. Deploy image:x.x.x-debug • Patch/Re-deploy K8s Deployment: image, SYS_PTRACE • Expose Delve server port (kubectl port-forward) • Rollback after debug session Every Commit
- 16. # build Docker image with delve debugger docker build -t alexeiled/todo-app-web:debug -f debug/Dockerfile . # push Docker image to registry docker push alexeiled/todo-app-web:debug # patch Kubernetes deployment and service kubectl patch deploy todo-app --type="json" -ntodo-app --patch "$(/bin/cat dbg/patch-debug.json)" # forward debugger port to the local machine kubectl port-forward pod/app-pod 2345 # DEBUG SESSION ....... # rollback the patch kubectl patch deploy todo-app --type="merge" --patch "$(/bin/cat k8s-deployment/todo-app-deployment.yaml)" -ntodo-app
- 17. Local + Remote
- 18. Telepresence (Datawire 👍) Debug Locally Instant Feedback Realistic Environment Minimal Resource Footprint Use your own tools
- 19. reverse tunnel (sshd) over a kubectl port-forward
- 21. # open VS Code editor code . # [code.terminal.#1]: swap todo-app deployment with bash telepresence --namespace=todo-app --swap-deployment=todo-app --expose 3000 --run bash # [code.terminal.#1] run debugger (with .debug/runDelve.sh) dlv debug --listen=localhost:2345 --headless=true --api-version=2 --log=true ./ -- -config-file="$TELEPRESENCE_ROOT/etc/todo-app/redis.config" # [code.terminal.#2] invoke todo-app service in K8s cluster http http://a8d6ad3e6169111e98c1d02b9ca3b647-624533560.us- west-2.elb.amazonaws.com:9080/read/todo # DEBUG SESSION ....... stop debug (dlv); exit (swap back)
- 22. Debug Go in K8s https://telepresence.io TODO: check this too
- 23. @alexeiled