Building Event Streaming Microservices with Spring Boot and Apache Kafka | Jan Svoboda, Confluent
Download as PPTX, PDF1 like1,796 views
This document summarizes Jan Svoboda's presentation on building event streaming microservices with Spring Boot and Apache Kafka. Jan discusses his motivation for learning Kafka and microservices development. He outlines a typical journey from strangling monoliths to building event-driven architectures using Kafka. Jan explains how Spring and Kafka integrate well together. The remainder of the presentation demonstrates building a sample application using these techniques and dives into common architecture patterns like event sourcing, CQRS, scaling stateful services, and reverse proxy routing.
Building Event Streaming Microservices with Spring Boot and Apache Kafka | Jan Svoboda, Confluent
- 1. Building Event Streaming Microservices with Spring Boot and Apache Kafka Jan Svoboda, Solution Engineer, May 2021
- 2. Jan Svoboda https://www.linkedin.com/in/svobodajan/ • Confluent, Solutions Engineer 2020 - now • Event Streaming, Event-Driven Microservices • Apache Kafka, Kafka Streams • Pivotal, Platform Architect, 2016 - 2020 • PaaS, Microservices • Cloud Foundry, Spring, Rabbit MQ • IBM, Solutions Architect, 2011 - 2016 • DevOps, Agile • Bluemix, Rational
- 3. Agenda 3 Motivation My journey to learning Kafka Spring Microservices Architecture Why Spring + Kafka = ♥ Events Streaming Microservices Concepts Look Ma No Hands! Event Sourcing and Data Liberation Removing Database Dependency Microservices Decoupling Integration Pattern State Caching Storing State of a Microservice Demo See it to believe it...
- 4. My Motivation for Kafka and Microservices Why should a microservices developer care about learning Kafka?
- 5. Building Microservices aka Cloud Native Applications Modernizing Legacy Applications with New Technology Stack • 3-Layer Model-View-Controller Pattern has been very popular last 15 years • Organizations look for migration shortcuts (no budget=no fun) • New Toy Syndrome (just buy new technology and we are done…) 5 App PaaS D B UI UI App Service Reg. LB My session from Cloud Foundry Summit https://www.youtube.com/watch?v=9moTgBiEauE
- 6. What is your Microservices Journey? 6 James Watters (Pivotal SVP), Kafka Summit 2019 Keynote, https://youtu.be/9I3CDfHKfNY?t=582 ● Stage 1 -> Strangling the Monolith ○ DB based integration ● Stage 2-> Provide Independent Data Sources ○ REST based integration ● Stage 3 -> Build Event Streaming ○ Kafka based integration
- 7. Spring ♥Kafka Like Peanut Butter and Jelly...
- 8. Spring? Spring Kafka? Spring Cloud Services? 8 ● Spring Kafka -> provides a core Kafka libraries ● Spring Cloud Stream + Kafka Binders -> provides an abstraction for building Kafka pipelines ● Spring Integration -> provides Kafka as an integration adapter (EIP) https://spring.io/projects/spring-cloud-stream https://spring.io/blog/2015/11/29/how-not-to- hate-spring-in-2016
- 9. (Re)Building Spring based Microservices with Kafka Look Ma, No Hands!
- 10. Demo Application - Architecture and Dataflow • Spring Boot (2 jars) • Apache Kafka (2 topics) • Spring Cloud Streams + Kafka Streams Binder (RocksDB as a state store) Jan:Google,http://google.com Jan:Google,http://google.com JanGoogle: Google, http://google.com JanGoogle,http://google.com https://github.com/griga23/bookmarksDemo
- 11. Event Streaming Microservices Architecture Patterns Getting hands dirty...
- 12. 1. Event Sourcing - Removing Database Dependency My Data Liberation ● From Data sourcing to Event sourcing ● From synchronous Request/Reply to asynchronous Publish/Subscribe ● Store the latest key/values in Kafka Streams State Store ● Use Kafka Streams Interactive Queries to query the State Store ● Make data free for everyone to read https://docs.confluent.io/platform/current/streams/developer-guide/interactive-queries.html =>
- 13. 2. CQRS - Microservices Decoupling Command Query Responsibility Segregation ● Pattern separates read and update operations for a data store. ● Implementing CQRS can maximize performance, scalability, and security. ● Allows a system to better evolve over time and prevents update commands from causing merge conflicts at the domain level. https://docs.microsoft.com/en- us/azure/architecture/patterns/cqrs
- 14. 3. Scaling – Stateful Microservices • Multiple State Stores with different partitions • HTTP REST calls to provide State Stores inter communication Jan:Google,http://google.com Jan:Yahoo,http://yahoo.com JanGoogle:Google,http://google.com JanYahoo:Yahoo,http://yahoo.com JanGoogle:Google,http://google.co m JanYahoo:Yahoo,http://yahooo.co m https://docs.confluent.io/platform/current/streams/developer-guide/interactive-queries.html#querying-remote-state-stores-for-the-entire-app
- 15. 4. Reverse Proxy – Routing to the Right State Store https://medium.com/bakdata/queryable-kafka-topics-with-kafka-streams-8d2cca9de33f • Spring Cloud Gateway for incoming HTTP traffic to redirect to proper State Store • State Store Host Awareness Jan:Google,http://google.com Jan:Google JanYahoo:Yahoo,http://yahoo.com