GDG Cloud Southlake 29 Jimmy Mesta OWASP Top 10 for Kubernetes
- 1. Introducing the OWASP Top 10 for Kubernetes Jimmy Mesta CTO / Co-Founder, KSOC @jimmesta [email protected]
- 2. Nothing to see here Move along
- 4. Another Top Ten? Kubernetes is highly configurable, mature, and has no shortage of success in production. Security blind spots still exist and are presenting serious challenges for security teams at scale. owasp.org/www-project-kubernetes-top-ten @jimmesta
- 5. UNSECURED K8S CREDENTIALS CLUSTER PRIVILEGE ESCALATION OPEN API Tesla Shopify Dero Miner Abuse of Kubernetes at scale is already here RECENT K8S ECOSYSTEM CVES Clusternet CubeFS Jenkins plugin Crossplane NOTABLE BREACHES RBAC ISSUE MANAGING K8S AT SCALE OVERLY PERMISSIVE RBAC UNMASKED VALUES GOLANG LIBRARIES FOR CONTROL PLANE LEVERAGING RBAC FOR PERSISTENCE RBAC Buster runc CVE-2024-21626 - Critical Container Breakout GKE RBAC Privilege Escalation and Abuse
- 6. @jimmesta K00 | Introduction K01 | Insecure Workload Configurations K02 | Supply Chain Vulnerabilities K03 | Overly Permissive RBAC Configurations K04 | Lack of Centralized Policy Enforcement K05 | Inadequate Logging and Monitoring K06 | Broken Authentication Mechanisms K07 | Missing Network Segmentation Controls K08 | Secrets Management Failures K09 | Misconfigured Cluster Components K10 | Outdated and Vulnerable Kubernetes Components @jimmesta
- 7. 1. Insecure Workload Configurations Bring on the YAML
- 8. What’s the problem? Misconfigurations top the charts when it comes to security issues. The 2021 Kubernetes Security Survey from Redhat stated that nearly 60% of respondents have experienced a misconfiguration incident in their Kubernetes environments in the last 12 months. @jimmesta
- 9. How can we prevent it? @jimmesta
- 10. What tools are on oer to help us? @jimmesta
- 12. How can we prevent it? @jimmesta Software Bill of Materials (SBOM) Image Signing (hps://github.com/sigstore/cosign) Image Composition Image Runtime Verification (rad.security/catalog) Policy enforcement
- 13. How can we prevent it? @jimmesta
- 15. What is the problem? Role-Based Access Control enables fine grained access for users, groups, and service accounts within Kubernetes. RBAC can be extremely diicult to scope appropriately which opens up additional privileges. @jimmesta
- 16. @jimmesta
- 17. @jimmesta
- 18. @jimmesta
- 19. How can we prevent it? ▦ Reduce direct cluster access by end users when possible ▦ Don’t use Service Account Tokens outside of the cluster ▦ Avoid automatically mounting the default service account token ▦ Audit RBAC included with installed third-party components ▦ Utilize RoleBindings to limit scope of permissions to particular namespaces vs. cluster-wide RBAC policies @jimmesta
- 20. @jimmesta How can we prevent it?
- 21. 4. Lack of centralised policy enforcement @jimmesta
- 22. What is the problem? Distributing and enforcing security policies across multiple clusters, clouds, and risk tolerances quickly becomes unmanageable for security teams. The inability to detect, remediate, and prevent misconfigurations from a central location can leave clusters open to compromise. @jimmesta
- 23. Example aack scenario Container breakout in a single tweet! 🤯 @jimmesta
- 24. How can we prevent it? @jimmesta
- 26. What is the problem? A Kubernetes Environment has the ability to generate logs at a variety of levels from many dierent components. When logs are not captured, stored, or actively monitored aackers have the ability to exploit vulnerabilities while going largely undetected. @jimmesta
- 27. How can we prevent it? Start reviewing the Kubernetes audit logs! Centralised logging (events, containers, cloud logs, traces etc) Runtime detection using tools such as falco.org @jimmesta
- 29. What is the problem? Kubernetes supports a number of authentication mechanisms, however many these are likely only suitable for non-production or small clusters and can introduce significant security risks. @jimmesta
- 30. What is the problem? Kubernetes supports a number of authentication mechanisms, however many these are likely only suitable for non-production or small clusters and can introduce significant security risks. @jimmesta
- 31. @jimmesta
- 32. How can we prevent it? ▦ Avoid using certificates for end-user authentication ▦ Enforce MFA when possible ▦ Don’t use Service Account tokens from outside of the cluster ▦ Authenticate users and external services using short-lived tokens @jimmesta
- 34. What is the problem? A Wordpress pod is compromised on a cluster that has no network segmentation and the aacker is able to utilize built in networking utilities such as dig and curl to explore the network. They discover an internally accessible API running on port 6379 which is typically Redis. They are able to probe the Redis microservice which was intended to be internal and only used by backend APIs using curl. Data is stolen and modified. @jimmesta
- 35. How can we prevent it? ▦ Native Controls (Multi-Cluster) ▦ Native Controls (NetworkPolicies) ▦ Service Mesh @jimmesta
- 36. @jimmesta
- 38. What is the problem? An aacker compromises a web application running in a Kubernetes and is able to get a shell. They run the following command to ensure Kubernetes secrets are mounted: ls /var/run/secrets/kubernetes.io/serviceaccount The aacker installs kubectl in the compromised pod which by default will aempt to use the default service account located in the above directory. The aacker can then communicate with the Kubernetes API from the inside leveraging the default service account’s RBAC access. @jimmesta
- 39. How can we prevent it? ▦ Encrypt secrets at rest ▦ Ensure logging and auditing is in place ▦ Think about leveraging runtime detection @jimmesta
- 40. Useful tools for secrets @jimmesta hps://github.com/mozilla/sops hps://github.com/bitnami-labs/sealed-secrets hps://github.com/hashicorp/vault-k8s
- 42. What is the problem? An aacker compromises a web application running in a Kubernetes and is able to get a shell. They run the following command to ensure Kubernetes secrets are mounted: ls /var/run/secrets/kubernetes.io/serviceaccount The aacker installs kubectl in the compromised pod which by default will aempt to use the default service account located in the above directory. The aacker can then communicate with the Kubernetes API from the inside leveraging the default service account’s RBAC access. @jimmesta
- 43. What is the problem? The components the encompass Kubernetes itself are highly configurable. The Kubelet running on each node are an example of a critical piece of infrastructure that requires hardening. This is especially true in “DIY” clusters. @jimmesta
- 44. @jimmesta
- 45. @jimmesta How can we prevent it? https://github.com/ksoclabs/kbom
- 47. What is the problem? @jimmesta https://ksoc.com/blog/addressing-the-new-kubernetes-cves-in-ingress-nginx https://ksoc.com/blog/addressing-curl-vulnerabilities-cve-2023-3854-and-cve-2023-38545
- 48. Some questions to ask yourself… @jimmesta Are you using hardened base images? Are your images being scanned before being used? Are your images running as root? Are your images running as a consistent user and group? Do you have a labelling taxonomy for resources in your cluster? Are you enforcing policies using Admission Control? Have you audited the RBAC configuration of your cluster? Do you have a process for regularly upgrading Kubernetes? Do you have a process for regularly upgrading your third party tooling?
- 49. Thanks, happy to take questions … ksoc.com Jimmy Mesta, Co-founder & CTO @ KSOC @jimmesta [email protected]