Container security within Cisco Container Platform
0 likes293 views
The document discusses security within Cisco Container Platform. It provides an overview of the security model and features, including platform hardening through the Cisco Secure Development Lifecycle process, role-based access control for Kubernetes, and secure multi-tenancy capabilities in Kubernetes clusters. It also covers container and Kubernetes security best practices like encryption, authentication, and network policies that are supported in Cisco Container Platform. The presentation concludes with a demo of secure multi-tenancy in Kubernetes clusters.
Container security within Cisco Container Platform
- 1. #CLUS
- 2. #CLUS Sanjeev Rampal Principal Engineer, Cloud Platforms BU BRKCLD-2011 A comprehensive look at security within the Cisco Container Platform
- 3. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Agenda ⹠Introduction to Cisco Container Platform ⹠Security Model, Agile delivery, Sample Topology ⹠Platform Hardening & Cisco Secure Development ⹠Kubernetes & Container Security ⹠Kubernetes Secure Multi-tenancy ⹠Demo BRKCLD-2011 3
- 4. WEBEX TEAMS DOCUMENTS SPEAKER 2 SPEAKER 1 cs.co/ciscolivebot# Questions? Use Cisco Webex Teams to chat with the speaker after the session Find this session in the Cisco Live Mobile App Click âJoin the Discussionâ Install Webex Teams or go directly to the team space Enter messages/questions in the team space How Webex Teams will be moderated by the speaker until June 16, 2019. 1 2 3 4 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Cisco Webex Teams BRKCLD-2011 4
- 5. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Can a look at security ever be âcomprehensiveâ ? BRKCLD-2011 5
- 6. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Control Plane Data Plane VM VM Control Plane Kubernetes Automation Orchestration Operations HX Connect Cluster/ Machine Controllers VM VM VM Cluster 1 Kubernetes Cluster1 Workloads Cluster1 Ops Pod Pod Pod VM VM VM Cluster 2 Kubernetes Cluster2 Workloads Cluster2 Ops Pod Pod Pod Kubernetes Fluentd Prometheus Kibana Hyperflex Contiv Storage (HyperFlex / VMware) Networking (Nexus 9K) Compute Hardware (UCS) Hypervisor Layer (HyperFlex / VMware) Cisco Container Platform Architecture VM VM Istio BRKCLD-2011 6
- 7. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Ops Personas & Logical production layout Tenant cluster 1 Devops admin/ Dev K8s api, RBAC K8s data plane Tenant cluster 2 Devops Admin/ Dev K8s api, RBAC K8s data plane CCP Admin (IT Ops) CCP api, RBAC Full cluster & services life-cycle mgmt âImmutableâ infra Ubuntu K8s Add-ons Ubuntu K8s Add-onsUbuntu K8s CCP app CCP admin Web based Installer VM BRKCLD-2011 7
- 8. Security Model, Agile delivery, Sample Topology
- 9. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Software Layering & CCP Security Scope Hypervisor, Virtualization infra e.g. vSphere VMs, Instances, Node OS Kubernetes, Docker, Container infra plugins CCP Application Addons Physical Compute, Network, Storage Hypervisor, Virtualization infra e.g. vSphere VMs, Instances, Node OS Kubernetes, Docker, Container infra plugins End-user Applications Addons Physical Compute, Network, Storage CCP packaging & Security responsibility Physical Infra separate setup + responsibility End-user Application responsibility Control cluster Tenant cluster BRKCLD-2011 9
- 10. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Agile Delivery & Immutable infra based model ⹠Immutable Infrastructure ⹠Integrated provisioning and full lifecycle management of of infrastructure (VMs, Node OS etc) along with Kubernetes, container infra ⹠No additional software patching or maintenance needed for Node OS ⹠Centralized upgrades + patching of combined infra => No configuration drift or snowflakes ⹠Continuous Release and Delivery ⹠Bi-weekly internal releases, Monthly external releases, patch releases asap when needed => Improved overall product security, predictability & quick turnaround of security patches BRKCLD-2011 10
- 11. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Sample secure deployment: Private stub network 11BRKCLD-2011 K8S cluster K8S Pod IPs 192.168.0.0/16 K8S Service IPs 10.96.0.0/12 Cluster Node IPs Exposed k8s api and application IPs/ VIPs vSphere cluster CCP control cluster IP gateways External Routed n/w Inbound Proxy Outbound Proxy (optional) Non- containerized Oracle DB (for example) Firewall Private stub network w/ RFC 1918 addressing SNAT IPAM
- 12. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Multi-cloud deployment: Cisco CP + AWS EKS BRKCLD-2011 12
- 14. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS CCP Platform Hardening incl CSDL ⹠Cisco Container Platform is developed using the comprehensive security requirements defined in the Cisco Secure Development Lifecycle (CSDL) process ⹠Curated Ubuntu OS from Canonical ⹠Cisco performs additional hardening of containers (internally developed for CCP application as well as sourced from upstream) ⹠Frequent internal vulnerability scanning & fixing of every CCP release using a mix of external vendor container security tools as well as internal tooling BRKCLD-2011 14
- 15. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Internal DevSecOps: Secure Development Pipeline Validate IPC Developed code BOM created, reviewed and approved BOM to IPC (CCP Github) Releases into CCP CI file repo To CCO On-demand test deployment Static registry scan Run-time test Requirements Input + Vulnerability alert feeds Release built CCO Ubuntu, K8S ⊠Container artifacts Ex. Prometheus, NGINX etc CCP CI registry Vulnerability Scanning tools BRKCLD-2011 15
- 16. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Additional platform hardening features ⹠TLS communication for http traffic (Encrypted data in motion internal and external) ⹠Support for TLS 1.3 on CCP API/Dashboard ⹠Strong ciphers for internal encrypted data at rest ⹠ecdsa and ed25119 keys for ssh into cluster nodes ⹠Continuous monitoring of NVD and industry standard vulnerability intelligence streams and rapid turnaround of patch releases ⹠Recent industry CVEs fixed & delivered rapidly on CCP ⹠Example: Critical k8s patch delivered in 2 weeks ⊠CVE-2018-1002105: proxy request handling in kube-apiserver can leave vulnerable TCP connections BRKCLD-2011 16
- 17. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS CCP App Security: Role Based Access Control BRKCLD-2011 17
- 19. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Kubernetes Security is a Journey From: J Jalava: Kubernetes Security Journey BRKCLD-2011 19
- 20. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Tenant Kubernetes & Docker Security in CCP K8S Security related features on CCP Kubernetes clusters ⹠K8S dashboard protection ⹠K8S Authentication ⹠K8S Authorization ⹠K8S Cert manager ⹠K8S Encrypted secrets ⹠Kubernetes Ingress with TLS/ https ⹠Istio Ingress gateway + Service mesh ⹠K8S Network policy ⹠Secure Multi-tenancy, Admission controllers, Pod Security Policies, AppArmor, Kata (future) BRKCLD-2011 20
- 21. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS K8S dashboard protection Kubernetes dashboard locked in CCP "The hackers had infiltrated Tesla's Kubernetes console which was not password protected," - ArsTechnica BRKCLD-2011 21
- 22. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS K8S AuthN, AuthZ, Admission Control flow BRKCLD-2011 22
- 23. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS K8S AuthN, AuthZ on CCP K8S tenant clusters ⹠K8S Authentication options: ⹠X.509 Client certificates <Suggested for simple deployments only> ⹠If team has AWS account, can use AWS IAM with on-prem CCP K8S ⹠Integrate 3rd party identity solutions e.g. Tremolo ⹠Direct Kubernetes OIDC-LDAP integration (future) ⹠K8S Authorization options: ⹠ABAC: Disabled on CCP K8S ⹠RBAC: Role Based Access Control; enabled by default on CCP K8S ⹠Authorization webhooks (Tech preview; full support in upcoming release) ⹠Open Policy Agent (Tech preview; full support in upcoming release) BRKCLD-2011 23
- 24. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS AWS IAM Authentication for On-prem CCP K8S BRKCLD-2011 24
- 25. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Kubernetes cert-manager & encrypted secrets ⹠Kubernetes Cert-manager: ⹠Kubernetes project to automate generation of X.509 certificates ⹠Used in CCP to generate certs for internal communication & external API ⹠Kubernetes Encrypted secrets: ⹠Kubernetes state in etcd can be encrypted using Kubernetes encrypted secrets feature ⹠Note, this feature must be enabled via CCP api, not exposed to GUI yet ⹠Set etcd_encrypted=True to enable this capability per tenant k8s cluster BRKCLD-2011 25
- 26. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Admission Controllers & Secure Multi-tenancy ⹠New feature (Tech preview in CCP v3.2) ⹠Setting secure_multitenancy_enabled to True enables 1. Multiple built-in k8s admission controllers on the new cluster: ⹠PodSecurityPolicy ⹠LimitRanger ⹠ResourceQuota ⹠ValidatingAdmissionWebhook ⹠MutatingAdmissionWebhook 2. Privileged & restricted pod security policies and associated PodSecurity and RBAC policies and bindings (with AppArmor and Seccomp based tenant and container isolation) 3. Privileged-tenant & restricted-tenant as sample tenants BRKCLD-2011 26
- 27. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Kubernetes Network Policies Support ⹠Supported on CCP tenant clusters for all 3 CNI options ⹠Network microsegmentation tool within Application + across teams ⹠L3, L4 CNI network policies (ingress and egress) ⹠Extra network policy options when using ACI CNI ⹠L7 policies on K8S Ingress (Nginx) and Istio (Envoy) ⹠E-W http traffic encryption w/ Istio App-front App-db App-core1 App-front App-metrics BRKCLD-2011 27
- 28. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Technical Description ⹠Network policies of Kubernetes supported using standard upstream format but enforced through OpFlex / OVS using APIC Host Protection Profiles ⹠Kubernetes app configurations can be moved without modification to/from ACI and non-ACI environments ⹠Standard K8S Container Network policies + (optional) enhanced ACI container network policies Kubernetes Network Policies on CCP+ACI CNI Node OpFlex OVS Kubernetes ACI Policies Network Policy Node OpFlex OVS BRKCLD-2011 28
- 29. Demo: Secure Multi-tenancy in CCP K8S
- 30. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS More Information ⹠https://www.cisco.com/c/en/us/products/cloud-systems- management/container-platform/index.html ⹠https://www.cisco.com/c/en/us/support/cloud-systems- management/container-platform/tsd-products-support-series- home.html ⹠http://www.cisco.com/go/multicloud ⹠Webex space for this session cs.co.ciscolivebot#BRKCLD-2011 ⹠Or contact/ follow: [email protected] @sr2357 BRKCLD-2011 30
- 31. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Conclusion: Cisco Container Platform .. so that you wonât need to be this guy BRKCLD-2011 31
- 32. Complete your online session evaluation ⹠Please complete your session survey after each session. Your feedback is very important. ⹠Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live water bottle. ⹠All surveys can be taken in the Cisco Live Mobile App or by logging in to the Session Catalog on ciscolive.cisco.com/us. Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS BRKCLD-2011 32
- 33. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Continue your education Related sessions Walk-in labs Demos in the Cisco campus Meet the engineer 1:1 meetings BRKCLD-2011 33
- 34. Thank you #CLUS
- 35. #CLUS
- 36. Backup content
- 37. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS K8S Security: Some key standards & initiatives ⹠CIS Docker Benchmark https://www.cisecurity.org/benchmark/docker/ ⹠CIS Kubernetes Benchmark https://www.cisecurity.org/benchmark/kubernetes/ ⹠NIST SP 800-190 https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdf ⹠AppArmor http://wiki.apparmor.net/index.php/Main_Page ⹠SELinux https://selinuxproject.org/page/Main_Page ⹠CRI-O https://github.com/cri-o/cri-o ⹠Kata containers https://katacontainers.io/ ⹠And more ⊠⹠These are in addition to common infrastructure security & compliance related standards such as Common Criteria, FIPS, PCI-DSS, GDPR, HIPAA BRKCLD-2011 37
- 38. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Preconfigured Pod Security Policy: ârestrictedâ apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name:restricted annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNa mes:'docker/defaultâ apparmor.security.beta.kubernetes.io/allowedProfileNam es:'runtime/defaultâ seccomp.security.alpha.kubernetes.io/defaultProfileNam e:'docker/defaultâ apparmor.security.beta.kubernetes.io/defaultProfileNam e:'runtime/defaultâ spec: privileged:false allowPrivilegeEscalation:false requiredDropCapabilities: -ALL volumes: -'configMapâ -'emptyDirâ -'projectedâ -'secretâ -'downwardAPIâ -'persistentVolumeClaim' hostNetwork:false hostIPC:false hostPID:false runAsUser: rule:'MustRunAsNonRootâ seLinux: rule:'RunAsAnyâ supplementalGroups: rule:'MustRunAsâ ranges: - min:1 max:65535 fsGroup: rule:'MustRunAsâ ranges: - min:1 max: 65535 readOnlyRootFilesystem:false BRKCLD-2011 38
- 39. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Calico/ Contiv Overlay Container Networking K8S master nodes/ VMs 1..3 K8S compute nodes/ VMs 1..M VMWare VM Port group 100 Physical L3 gateways Contiv VXLAN overlays Non-contiv VLAN traffic K8S compute nodes/ VMs 1..M BRKCLD-2011 39
- 40. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS HX vSphere Cluster CCP â CP1 K8S-Red K8S-Blue vCenter PG10 PG20 PG30 10.1.1.0/28 10.1.2.0/24 10.1.3.0/24 ASR1K or any L3 GW Leaf e.g. N93xx Spine e.g. N95xx 100.1.x.x DHCP server* (for pre-3.0 releases) Secure On-Premises Deployment Topology BRKCLD-2011 40