Raleigh Kafka Meetup - DDD, ES, and CQRS
0 likes•116 views
The document discusses the integration of domain-driven design, event sourcing, and command query responsibility segregation (CQRS) in scaling software products. It emphasizes the importance of organizing software to harmonize communication and facilitate migration to microservices while addressing challenges such as consistency and change. Additionally, it highlights the benefits of separating commands and queries to allow for independent scaling and architectural flexibility.
Raleigh Kafka Meetup - DDD, ES, and CQRS
- 1. Scaling Products: People and Technology What do Domain-Driven Design, Event Sourcing, and CQRS have to do with each other?
- 2. “Who am I? Why am I here?” – Admiral James Stockdale
- 3. Who is Jeff? Why is he here? Data Platform Architect for HPE InfoSight Technologist / Generalist Work with many teams across HPE
- 4. Who are you? Why are you here? Free beer and pizza at the Kafka Meetup! Monolith to Microservices? Focus shift from one component to entire solution? Would like to migrate your DB, but you’re stuck? Reactive, responsive, real-time insights for your customers? Struggling with data engineering and ETL for your data warehouse? Successful product needs to scale?
- 5. Challenges We Face Scaling is hard! Why? Consistency is easy for singletons (e.g. monoliths) Tech: Service, Database, Server, Data Center People: Team, Geography, Code Repo Change is hard! I don’t know why, it just is. Scaling and change are not inherent goals…just inevitable
- 6. TL;DR Domain-Driven Design - organizing an evolving software model in a way that harmonizes communication within the software and among the people that develop it Event Sourcing - organizing the publishing of a complete set of information for an evolving set of consumers Command Query Responsibility Segregation (CQRS) Separate and scale transactions separately from queries In conjunction with Event Sourcing, democratizes data
- 8. Domain-Driven Design How to organize software and services based on the objectives and actual model of the business? Complementary with Agile methodologies Collaboration between technical and subject matter experts to map the business into the software Principled approach for migration to Microservices Learn about DDD before you (re-)architect your system
- 9. Domains and Services Services are part of domains Terminology: Domain has a consistent Model within a Bounded Context, described by a Ubiquitous Language, manifested in the software itself Core Domain is unique and critical to the business Sub-Domains are more generic, less important
- 10. Device Ident & Auth Federated ID Entitlement CRM Cases Core Domain Installed Base Email Rule Registry API Gateway Device Shadow Health Alerts
- 11. Data Access within Domains Services respond to Commands and Queries Commands can change state, cause side effects Queries share data with no side effects Return data from the service DB Proxy requests to other services Return application-level joins of the above two Complex queries across data models?
- 12. “Microservices shall have their very own database.” – People on the Internet https://xkcd.com/386
- 13. “Oh, crap. My data is everywhere.” – Microservice Adopter
- 14. “Trust me. MICRO-services are the dumbest services in the world. Sad.” – Microservice Hater
- 15. Event Sourcing
- 16. Event Sourcing Event is something that has happened Truth is represented as a time-ordered stream of events The Log - classic blog post by Jay Kreps Transaction logs - e.g. DB and storage replication Views (e.g. DB tables) can be populated from event stream
- 17. Producers and Consumers Microservices and Streaming Jobs can both be Producers and Consumers of Event Streams
- 18. IoT Core Domain API Query DB Data Warehouse or Data Mart Shadow Device Aggregator Microservice Device Shadows Device Streaming Load Streaming Load Stateless API Microservice Data Science Kafka Subscribe to Event Streams from other Domains Domain-Produced Event Stream Config Rule Checking Device Device Shadow Commands / Queries Status, Metrics, Events, Logs Commands, Desired State Streaming Analytics / ML
- 19. Kafka Device Ident & Auth Federated ID Entitlement CRM Cases Core Domain Installed Base Email Rule Registry API Gateway Device Shadow Health Alerts Tagging
- 21. CQRS Why segregate Commands and Queries? Traditional services query the same transactional DB Separating them allows different architectures and to scale independently Commands still support transactions, but Queries are Eventually Consistent
- 22. CQRS
- 23. CQRS / Event Sourcing
- 24. Transactional Event Sourcing CQRS/ES only makes sense in Services with Entities that support Commands (i.e. transactions) Does not apply to Gateway Services (“Salesforce says…”) Not applicable for Stream Processing Frameworks For transactions across Services, see Saga Pattern
- 25. Questions?