4K–Kubernetes with Knative, Kafka and Kamel
1 like•5,886 views
This document contains details about an upcoming DevNation schedule of events on Kubernetes topics, including dates and times. It also provides background information on the presenters Burr Sutter and Kamesh Sampath. Additionally, it outlines an agenda for a session on understanding Kubernetes, Knative, Apache Kafka and Camel-K as four pillars of cloud data streaming.
4K–Kubernetes with Knative, Kafka and Kamel
- 1. bit.ly/kubemaster1 1 4K - Kubernetes dn.dev/4k-kubernetes with Knative, Kafka and Kamel
- 2. dn.dev/4k-kubernetes Upcoming DevNation Schedule India (English) UTC: 7:00 Local 12:30 France (French) UTC 9:00 France (French) UTC 11:00 Local 13:00 Brazil (Portuguese) UTC 14:00 Local 11:00 USA (English) UTC 14:00 Local 10:00 Eastern USA (English) UTC 16:00 Local 12:00 Eastern Mexico (Spanish) UTC 18:00 Local 13:00 Central Standard May 4 2020 Kubernetes Beginner Kubernetes Beginner Kubernetes Beginner Kubernetes Beginner Kubernetes Beginner May 6 2020 Kubernetes Elementary Kubernetes Elementary Kubernetes Elementary Kubernetes Elementary Kubernetes Elementary May 11 2020 Quarkus Basics Quarkus Basics Quarkus Basics Quarkus Basics Quarkus Basics May 12 2020 Istio Istio Istio Istio Istio May 13 2020 Quarkus Cloud Native Quarkus Cloud Native Quarkus Cloud Native Quarkus Cloud Native Quarkus Cloud Native May 18 2020 Ops & Admins I Ops & Admins I Operator SDK I Ops & Admins I May 20 2020 Ops & Admins II Ops & Admins II Operator SDK II Ops & Admins II
- 4. dn.dev/4K-kubernetes @burrsutter ● [email protected] ● Featured speaker at technology events around the globe ● Java Champion since 2005 ● Former President of the Atlanta Java User Group ● Founded the DevNexus conference ● Always looking for technologies that enable developers to deliver better software ever faster
- 5. dn.dev/4K-kubernetes Kamesh Sampath(devadvocate.yaml) apiVersion: bio/v1 kind: DeveloperAdvocate spec: title: “Director of Developer Experience @ Red Hat” gde: expert: cloud year: 2019 javaDeveloper: true opensourceContributions: - Eclipse Che - Quarkus - Supersonic Subatomic Java - kubernetes/minikube: Run Kubernetes locally - Camel K social-handles: linkedin: https://linkedin.com/in/kameshsampath github: kameshsampath (Kamesh Sampath) twitter: Kamesh Sampath (@kamesh_sampath) blogs: developers.redhat.com(Kamesh Sampath) kubectl email --address [email protected]
- 6. dn.dev/4k-kubernetes Agenda Understand four pillars of Cloud Data Streaming ○ Kubernetes ○ Knative ○ Apache Kafka ○ Camel-K(Kamel)
- 8. dn.dev/4k-kubernetes Love Thy Mono Old School New School
- 9. dn.dev/4k-kubernetes A Challenge Server Hardware Operating System Application Server .war or .ear Java Virtual Machine Custom Configuration Linux Kernel Version & Distribution Java 1.6.6_45 or Java 1.7.0_67 Weblogic 10.x.y, Tomcat 6.x.y, JBoss EAP 6.x.y JDBC driver, datasource, JMS queue, users Have you ever had “/” vs “” break your app? Or perhaps needed a unique version of a JDBC driver? Or had a datasource with a slightly misspelled JNDI name? Or received a patch for the JVM or app server that broke your code? Containerize Your App
- 10. dn.dev/4k-kubernetes Email MyApp.war has been tested with the following On my Windows 7 desktop JDK 1.8.43 Wildfly 9 Configuration: Datasource: MySQLDS Tested with: mysql-connector-java-5.1.31-bin.jar Production Environment Red Hat Enterprise Linux 6.2 JRE 1.7.3 WebSphere 8.5.5 Oracle 9
- 11. dn.dev/4k-kubernetes Dockerfile Container Guest OS Java App Server custom configuration FROM centos/wildfly COPY xyz.xml /opt/myserver/configuration/ COPY mysql-connector-java-5.1.31-bin.jar /opt/wildfly/standalone/deployments/ COPY mysql-sample-ds.xml /opt/wildfly/standalone/deployments/ COPY myapp/target/your.war /opt/wildfly/standalone/deployments/ your.war dependencies
- 12. dn.dev/4k-kubernetes DevOps Challenges for Multiple Containers ▪ How to scale? ▪ How to avoid port conflicts? ▪ How to manage them on multiple hosts? ▪ What happens if a host has trouble? ▪ How to keep them running? ▪ How to update them? ▪ Rebuild Container Images? Node Node Node Node Node Logger Node
- 14. dn.dev/4k-kubernetes ▪ Greek for “Helmsman,” also the root of the word “Governor” (from latin: gubernator) ▪ Container orchestrator ▪ Supports multiple cloud and bare-metal environments ▪ Inspired by Google’s experience with containers ▪ Open source, written in Go ▪ Manage applications, not machines Meet Kubernetes
- 15. dn.dev/4k-kubernetes History of Microservices Continuous Integration via XP 1999 AWS EC2 2006 DropWizard May 2011 Agile Manifesto Feb 2001 NETFLIX to AWS 2010 Ribbon March 2012 Hystrix March 2012 Eureka July 2012 Microservices Assess Thoughtworks Radar March 2012 Spring Boot Sept 2013 Microservices Defined Thoughtworks Fowler, Lewis March 2014 Kubernetes June 2014 Java EE6 2009 DevOps 2009 Docker March 2013 Vert.x June 2011 2 0 1 5
- 18. dn.dev/4k-kubernetes Pods A group of whales is commonly referred to as a pod and a pod usually consists a group of whales that have bonded together either because of biological reasons or through friendships developed between two or more whales. In many cases a typical whale pod consists of anywhere from 2 to 30 whales or more.* *http://www.whalefacts.org/what-is-a-group-of-whales-called/
- 20. dn.dev/4k-kubernetes Pod Replicaset/ Deployment Service Label ✓ 1+ containers ✓ Shared IP ✓ Shared storage (ephemeral) ✓ Shared resources ✓ Shared lifecycle ✓ The Desired State - replicas, pod template: health checks, resources, image ✓ Grouping of pods (acting as one) has stable virtual IP and DNS name ✓ Key/Value pairs associated with Kubernetes objects (env=production) Kubernetes Terms Persistent Volume ✓ Network available storage ✓ PVs and PVCs
- 21. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - Nodes kubelet kubelet kubelet kubeletkubeletkubelet
- 22. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - Declarative kubelet kubelet kubelet kubeletkubeletkubelet image: repo/mytomcat:v1 replicas: 4
- 23. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - 4 Tomcats kubelet kubelet kubelet kubeletkubeletkubelet
- 24. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - Pod Fail kubelet kubelet kubelet kubeletkubeletkubelet X
- 25. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - Correcting kubelet kubelet kubelet kubeletkubeletkubelet
- 26. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - Node Fail kubelet kubelet kubelet kubeletkubeletkubelet X
- 27. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Dev Ops api etcd scheduler controllers kubelet kubelet kubeletkubeletkubelet Kubernetes Cluster - Pods Replaced
- 28. dn.dev/4k-kubernetes Master Node Node Istio proxy Node Node Node Dev Ops api etcd scheduler controllers Kubernetes Cluster - New Node kubelet kubelet kubeletkubeletkubelet Node kubelet
- 31. dn.dev/4k-kubernetes Serverless Computing “Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions, are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment.”
- 32. dn.dev/4k-kubernetes Why we need Serverless ?? Agility in any cloud environment
- 33. dn.dev/4k-kubernetes Serverless Computing “Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions, are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment.”
- 34. dn.dev/4k-kubernetes Short History of Serverless AWS S3 March 2006 AWS Lambda November 2014 AWS EC2 August 2006 Serverless Described Thoughtworks August 2016 Serverless Coined October 2012 Serverless Assess Thoughtworks Radar April 2016 AWS Dynamo DB January 2012 Firebase September 2011 Iron.io July 2011 Google Cloud Function Feb 2016 Azure Functions Mar 2016 Pivotal Riff Dec 2017 Oracle Functions Sept 2016 IBM Bluemix OpenWhisk Feb 2016 Oracle Fn Project Oct 2017
- 36. dn.dev/4k-kubernetes When to choose what ? • • • • • • • •
- 37. dn.dev/4k-kubernetes Microservices Serverless Functions Your Control Long-Lived Processes Known Programming Model Often Sync Request-Response Mature: IDE Integration Debuggers Tracers Monitoring CI/CD Cloud Control Short-Lived Processes New Programming Model Event-Driven Async Immature: ?
- 38. dn.dev/4k-kubernetes Microservices Serverless Functions Your Control Long-Lived Processes Known Programming Model Often Sync Request-Response Mature: IDE Integration Debuggers Tracers Monitoring CI/CD Cloud Control Short-Lived Processes New Programming Model Event-Driven Async Immature: ? Knative
- 40. dn.dev/4k-kubernetes Capabilities ● Rapid application deployment ● Scale-to-zero ○ No pod == no memory ● Scale-from-zero ○ Traffic spike starts N pods ● Configurations and Revisions ○ Deployment snapshots ○ Built-in Blue/Green ● Routing and Network configurations
- 43. dn.dev/4k-kubernetes Exploring Knative Eventing ● Eventing Sources ○ Producers of events e.g. Github, Camel, GCP Pub/Sub,CronJob ● Eventing Channels ○ Event forward or persistence layer e.g. Apache Kafka Channel, GCP Pub/Sub , InMemory ● Brokers ○ Event mesh ● Triggers ○ Connecting Broker and Subscriber ○ Filtering Events
- 44. dn.dev/4k-kubernetes Exploring Knative Eventing (contd.,) ● Event Registries ○ Repository of available event types that can be used with Broker ● Event Sink ○ Kubernetes Service ○ Knative Service ○ Channel ○ Broker
- 45. dn.dev/4k-kubernetes CNCF, CloudEvents and Serverless
- 49. dn.dev/4k-kubernetes Apache Kafka ? ● Developed at Linkedin back in 2010, open sourced in 2011 ● Designed to be fast, scalable, durable and available ● Ability to handle huge number of consumers ● Distributed by nature ● Messages are sent to and received from a topic
- 50. dn.dev/4k-kubernetes DevOps challenges - Apache Kafka ● Installation and Configuration is not simple ● Configuring a new cluster: ○ Configuring single brokers ○ Pay attention to conflicting configuration ● Updating a cluster: ○ Updating every single broker ○ Rolling restart of Brokers ○ Rollout and synchronization of Apache Zookeeper
- 51. dn.dev/4k-kubernetes http://strimzi.io/ Strimzi
- 52. dn.dev/4k-kubernetes Topic & User operators Zookeeper Cluster Kafka Cluster Cluster Operator Kafka Custom Resource Deployment Apache Kafka on Kubernetes
- 53. dn.dev/4k-kubernetes When to use Apache Kafka ? ● Kafka is a great tool for event-driven architectures ○ Backbone for event driven systems ○ Streaming platform ● Events can have different forms ○ Absolute: Temperature is 21 degrees celsius ○ Deltas: Temperature increased by 1 degree celsius ● Event streams ○ Event sourcing ○ Audit trails ○ Log and metrics analytics
- 57. dn.dev/4k-kubernetes What is Apache Camel •The swiss knife of integration •>10 years of development - still one of the most active Apache projects •Java-based integration framework that can be installed on multiple runtimes: Spring-Boot, JEE, JBoss, Karaf, Standalone, ... •Based on Enterprise Integration Patterns •Supports 250+ components •Uses a powerful DSL •Can integrate anything
- 60. dn.dev/4k-kubernetes Apache Camel-K ? ● Lightweight integration platform to run Apache Camel integrations as Kubernetes applications ● Run integrations as serverless applications ● Ability to integrate with Knative Channels
- 64. dn.dev/4k-kubernetes Resources Tutorials •Kubernetes Tutorial - dn.dev/kube-tutorial •Knative Tutorial – dn.dev/knative-tutorial •Istio Tutorial - dn.dev/istio-tutorial •Quarkus Tutorial - dn.dev/quarkus-tutorial •Tekton Tutorial - dn.dev/tekton-tutorial Master Courses - dn.dev/master DevNation Tech Talks - dn.dev/upcoming