Deep dive into Kubernetes Networking
17 likes10,010 views
The document provides an overview of Kubernetes networking concepts including single pod networking, pod to pod communication, service discovery and load balancing, external access patterns, network policies, Istio service mesh, multi-cluster networking, and best practices. It covers topics such as pod IP addressing, communication approaches like L2, L3, overlays, services, ingress controllers, network policies, multi-cluster use cases and deployment options.
![Network control policy example2
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: hello-allow-from-product
spec:
policyTypes:
- Ingress
podSelector:
matchLabels:
app: reviews
ingress: []
Product Reviews
Details Ratings](https://image.slidesharecdn.com/kubernetesnetworking-180803174415/85/Deep-dive-into-Kubernetes-Networking-18-320.jpg)
Deep dive into Kubernetes Networking
- 1. Deep dive into Kubernetes Networking Sreenivas Makam Partner Engineer @Google Cloud @Container conference, 3rd Aug 2018
- 2. Agenda ● Single pod networking ● Pod to pod Networking(East-West traffic) ● Service discovery and load balancing ● External access(North-South traffic) ● Network policy ● Istio service mesh ● Multi-cluster and hybrid cloud ● Best practises
- 3. Pods ● Small group of tightly-coupled containers ● Every pod has a unique IP ● Share same namespace and volume ● Share same lifecycle ● Use cases for multi-container pods - Sidecars, proxies/adapters Example: File puller & web server Consumers Content Manager File Puller Web Server Volume Pod
- 5. Communication across pods in different nodes Kubernetes supports different networking approaches to do pod networking Plugins can be added using CNI Type/Features L2 L3 Overlay Cloud Summary Pods communicate using L2 Pod traffic is routed in underlay network Pod traffic is encapsulated and uses underlay for reachability Pod traffic is routed in cloud virtual network Underlying technology L2 arp, broadcast Routing protocol like BGP VXLAN Pre-programmed fabric using controller(eg: GKE) Encapsulation No double encapsulation No double encapsulation Double encapsulation No double encapsulation Examples Calico Flannel, Weave GKE, ACS, EKS
- 6. L2 approach
- 7. L3 approach
- 9. Services ● A service is a group of endpoints (usually pods) grouped by selector ● Services provide a stable VIP (except Headless) ● VIP automatically routes to backend pods ● VIP to backend pod mapping is managed by kube-proxy, implemented using iptables Client Virtual IP apiVersion: v1 kind: Service metadata: name: productpage spec: selector: app: productpage type: ClusterIP ports: - port: 80 targetPort: 8080 protocol: TCP
- 10. DNS ● Service Discovery achieved by SkyDNS ● SkyDNS runs as a pod in the cluster apiserverwatch etcd kube2skyskyDNS
- 11. Service communication ● Service IP is accessible only from within the cluster ● Type “ClusterIP” is used for internal communication within the cluster ● Traffic sent to Service IP gets load balanced to pods that belong to service IP ● Source NAT is used for pods to communicate to external world ● IPtables are used extensively for load balancing and NAT ● Nodeport, Network load balancer and Ingress controller are service types for external world to reach services inside the cluster.
- 12. Service external reachability options Feature Nodeport Load balancer Ingress Summary Service is exposed using a reserved port in all nodes of cluster(Default: 32000-32767) Typically implemented as network load balancer Typically implemented as http load balancer IP address Node IP is used for external communication Each service needs to have own external IP Many services can share same external IP, uses path based demux Use Case Demo purposes L3 services L7 services Examples GKE Network load balancer GKE http load balancer, nginx, Istio
- 13. Nodeport apiVersion: v1 kind: Service metadata: name: productpage labels: app: productpage spec: type: NodePort ports: - port: 30000 targetPort: 9080 selector: app: productpage
- 14. Load balancer apiVersion: v1 kind: Service metadata: name: productpage labels: app: productpage spec: type: LoadBalancer ports: - port: 80 targetPort: 9080 selector: app: productpage
- 15. Ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gateway spec: backend: serviceName: productpage servicePort: 9080 rules: - host: mydomain.com http: paths: - path: /productpage backend: serviceName: productpage servicePort: 9080 - path: /login backend: serviceName: productpage servicePort: 9080 - path: /* backend: serviceName: productpage servicePort: 9080
- 16. Network control policy Controls communication between Pods and Services ● Traffic is by default allowed without network policy ● Kubernetes network policies are allow policies, there is no deny policy ● Empty pod selector selects all pods ● Ingress and egress rules are available to control traffic ● Traffic is allowed if atleast 1 policy allows traffic ● Network plugins like Calico, Weavenet support network policies. GKE uses calico
- 17. Network control policy example1 kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: hello-allow-from-product spec: policyTypes: - Ingress podSelector: matchLabels: app: reviews ingress: - from: - podSelector: matchLabels: app: productpage Product Reviews Details Ratings
- 18. Network control policy example2 kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: hello-allow-from-product spec: policyTypes: - Ingress podSelector: matchLabels: app: reviews ingress: [] Product Reviews Details Ratings
- 19. Book review App
- 20. Istio is an open framework for connecting, securing, managing and monitoring services Traffic control Observability Fault-injectionSecurity Hybrid cloud What is Istio? What can Istio do?
- 21. Istio Architecture Traffic transparently proxied — unaware of proxies Pilot Mixer Discovery & config data to proxies TLS certs to proxies Policy checks, telemetry Proxy Frontend Proxy Payments Citadel Istio Control Plane
- 22. Book review App(with Istio)
- 23. Compare Network control policy with Istio
- 24. Multi-cluster Use- cases: ● 1 cluster serving as DR for another cluster ● CI/CD between Dev and production clusters ● Global load balancing across multiple clusters ● Ability for 1 cluster to consume services from another cluster Requirements: ● Common control plane across clusters ● Service discovery across clusters ● Load balancing traffic across replicas in different clusters ● Common policy management across clusters - RBAC, Quota etc
- 25. Multicluster/Hybrid cloud use cases S1 S2 S1 S2 Disaster Recovery S1 S2 S1 S2 Dev/Prod setup CI/CD S1 S2 S1 S2 Load balance across regions LB Region1 Region2 S1 S2 S1 S2 Cloud bursting Consume services across cluster S4 S3
- 26. Multi-cluster deployment options ● Kubernetes Federated cluster ○ Have a single Kubernetes control plane that spans multiple clusters ○ Controlled using Kubefed ○ Service discovery works across clusters ● Istio Multicluster ○ Istio control plane that spans across multiple clusters ○ Kubernetes control plane limited to single cluster ○ Service discovery works across clusters ○ Istio ingress takes care of load balancing external traffic across clusters ● Managed service from cloud providers (eg: GCP Cloud services platform)
- 27. GCP Hybrid cloud Kubernetes solution
- 28. Kubernetes Networking - Best practises ● Use namespaces for multi-tenancy ● Use Ingress rather than Load balancer to expose services. This is to preserve resources ● Do not expose cluster to outside world if its not needed ● Create security policies like IAM, RBAC, Network control in the beginning rather than later ● Put emphasis on resource control including quotas, priorities/preemption etc
- 29. Demo Demo code and instructions: https://github.com/smakam/gcp/tree/master/gke/containerconfbglr18
- 30. References ● The Ins and Outs of Kubernetes and GKE networking presentation ● Understanding Kubernetes Networking ● Kubernetes networking concepts ● Enforcing network policies with Kubernetes ● Building hybrid clouds with Istio ● Kubernetes Nodeport vs Loadbalancer vs Ingress ● https://github.com/thesandlord/Istio101 Note: Some pictures used were borrowed from above blog posts