Decomposing the Monolith using Microservices that don't give you pain
Download as PPTX, PDF•6 likes•1,578 views
The document discusses the complexities of distributed computing, particularly focusing on microservices architecture and its benefits and challenges. It highlights the importance of reliable networks, operational maintenance, and advanced monitoring along with specific design principles like loose coupling and eventual consistency. Additionally, it outlines a variety of techniques and practices for managing microservices, such as event sourcing, asynchronous projections, and the use of API documentation.
Decomposing the Monolith using Microservices that don't give you pain
- 1. Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver
- 3. Also known as the Microservices Premium
- 4. Hybrid of Front-end Technologies Multiple patterns for the same problems Long compile time Large source control repositories Long-running unit tests Too much coupling Limited team scalability Devs need to know everything NoIsolationProductivity drops over time.
- 6. • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology doesn't change • There is one administrator • Transport cost is zero • The network is homogeneous. https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
- 7. The First Law of Distributed Computing
- 8. Very scalable, very reliable Very difficult to debug distributed problems Unreliable network requires complicated techniques Requires message bus/broker End-to-end testing very unpractical Very mature DevOps culture is a prerequisite Advanced monitoring and visualization is a must Security can not be an afterthought High operational maintenance. Can be deployed/upgraded independently.
- 10. Monolith Microservice Microservice Microservice HTTP API HTTP API HTTP API OWIN component released through MyGet/Nuget No network I/O at all Can have its own database instance, shared with monolith, or shared storage service Microservice owns the schema Separate repos with distinct owners. Runs in-process, thus easier debugging New version requires redeployment of the monolith. Loose coupling through HTTP Less scalability options HTTP is not the fastest serialization format Relies on libraries, not frameworks.
- 12. Client IIS Console WinSvc Unit Test HTTP request HTTP response Middleware environment handler next Middleware IDictionary<string, object> Func<IDictionary<string, object>, Task> Task Func< Func<IDictionary<string, object>, Task>, Func<IDictionary<string, object>, Task> >
- 13. Monolith Microservice HTTP API Service Registry Contract documented through Swagger Webhooks Microservice Microservice Provides OWIN- aware HttpClient to connect to service Event payload exposed as Swagger (e.g. SwaggerHub) Subscribers register URLs or in-process OWIN message handler API/payload consumers use liberal JSON interpretation Circuit Breaking to handle unreliable consumers At-least-once payload delivery Subcribers can request replays. Payload includes ID to help idempotency Each request requires correlation ID that flows through all APIs, webhooks and services
- 15. Aggregates Command Handlers Queries Commands Events Upconverters Value Objects Query Handlers Schema Migrations Projectors Data Importers Query API Jobs Projections HTML CSS JS API Controllers Projectors Projections Registrations Webhook API Command API Event Storage Payload Definitions Outer layers depend on inner layers Domain is persistency ignorant Can be tested independently For eventual consistent business rules Master data To support domain queries For microservices that provide a (partial) UI or management screen Based on Domain Driven Design Supported by LibLog, TinyIoc Designed according to the 12 Factor principles
- 16. UI / HTTP API Command Service ChangeUser EmailHandler User Unit of Work User Projector DAL Read DB Write DB ChangeUserEmailCommand Execute query Get<User>(identity) Invoke method Event Store Load(events) Apply Get changes Dispatcher Events Handle(UserEmailChangedEvent) Submit changes History Dennis Doomen | @ddoomen | The Continuous Improver
- 17. Monolith Microservice HTTP API Event Store Events Payload Projector Subscription Request Payload ‘event store’ Projectors use LiquidProjections Have their own checkpoints Projects fine-grained events into payload projections Projector Payload schema is documented using Swagger Projector Projector Webhook Projectors Separate instance per subscription Subscriber HTTP API ‘Project’ payload ‘event store’ into HTTP calls Projected Data Uses the incoming payloads as an ‘event store’ Payload Use Circuit Breakers to handle slow/unavailable subscribers SQL DB Schema changes using FluentMigrator or using NoSQL DB
- 19. Domain Event Store Events App RDBMS Events Projection EventsProjection Projection Projector Optimized for specific queries Separate projections database NoSQL Projector Projection-specific storage Projector HTML Raw SQL or Dapper Run asynchronously Great for sharding Dennis Doomen | @ddoomen | The Continuous Improver
- 20. Events Transaction 6 Transaction 5 Transaction 4 Transaction 3 Transaction 2 Transaction 1 Temporal Projector Read Store Time Projected until this point Immutable, thus auditable and SOX compliance Dennis Doomen | @ddoomen | The Continuous Improver
- 21. Domain Event Store Events App Events Projection EventsProjection Projection Projector Application projections RDBMS Reporting Projector Traditional reporting model Asynchronous OLAP Dennis Doomen | @ddoomen | The Continuous Improver
- 22. StreamId: User-Dedo, Rev: 5 Persisted State Changes StreamId: User-Dedo, Rev: 4 Grant Role “Editor” Change Password 2nd StreamId: User-Dedo, Rev: 4 StreamId: User-Dedo, Rev: 3 Create User Grant Role “Owner” Add Phone Number Change Password 1st StreamId: User-Dedo, Rev: 5 Revoke Role “Owner” StreamId: User-Dedo, Rev: 6 Grant Role “Editor” Time StreamId: User-Dedo, Rev: 7 Changed Password 2nd Dennis Doomen | @ddoomen | The Continuous Improver
- 23. Events 1 2 3 6297 6298 6298 Projections Documents Groups Users Objects Folders Microservice V2 Changes Queries Events 1 2 3 6299 6300 6301 Projections Documents Groups Users Objects Folders Microservice V1 ChangesQueries Migration Process Dennis Doomen | @ddoomen | The Continuous Improver
- 24. Events Aggregate Root Entity Entity Value Object Aggregate Root Aggregate Root Aggregate Root Entity Value Object Value Object Dennis Doomen | @ddoomen | The Continuous Improver
- 25. • Asynchronous projections • Idempotency of projections • Self-tracking • Projection autonomy -> more duplication • Projection tracking and prediction.
- 26. • Functional archiving • Clear separation between critical and auxiliary data -> prioritization • Partitioning • Dynamic rebuilding (after bugs, schema changes, etc).
- 27. • Projections that crash – Column constraints – Changes in data invariants – Unexpected projection dependencies – Lookups vs stream dependencies. • (Unexpected) projection dependencies – Synchronous -> asynchronous and existing bugs – Aggregrate dependencies. • Bugs – Causing large event streams.
- 29. Prerequisites - Microservice owns ‘schema’ - Supports upgrading and downgrading - NoSQL preferred, but RDBMS can work too.
- 30. Rebuild schema in-place – Rebuilding using event store – Requires tracking API – Involves down-time.
- 31. Disallow breaking changes – Only new ‘columns’ and ‘tables’ – Partial rebuilding using event store – Requires forwards compatibility.
- 32. Rebuild side-by-side – Rebuilding using event store – Requires tracking API – Requires compatibility with previous ‘schema’ – Previous ‘schema’ is removed in the next release.
- 34. Microservice Bunch of classes that should be used directly X No inheritance Uses composition over inheritance internally Convenience classes that don’t hide the magic Library Abstract Package Interfaces Delegates DTOs Only depend on abstract packages… Stable Package …or depend on more stable packages Auxiliary Package Classes that are not used together do not belong togetherOptional Dependency Dependency Package Consumers should not be faced with optional dependencies
- 35. • Delegates • Source-only Packages • Merge dependencies
- 37. Monolith Microservice Microservice Microservice HTTP API HTTP API HTTP API Apply routing conventions Versioning using Accept Headers, Response Headers or Routes Optimize body using AVRO or Protobuf Cross-service tracing using OpenTracing Replace OWIN with (upcoming) in- process gRPC Common contracts for diagnostics, monitoring Add KPI dashboards, e.g. Sumologic
- 39. 1 Build a (container-based) cloud capability. 2 Deploy monolith to the cloud. 3 Automate your delivery pipeline. 4 Full end-to-end responsibility 5 Break-up delivery teams and code 6 Evaluate status-quo on micro-services (products, platforms, etc) 7 Slide-and-dice microservices one-by- one.
- 41. • Liquid Projections – The Example https://github.com/liquidprojections • Bliki: MonolithFirst https://dzone.com/articles/martin-fowler%E2%80%94bliki • In-process gRPC https://github.com/grpc/grpc-go/issues/906 • Protobuf in WebAPI http://www.infoworld.com/article/2982579/application- architecture/working-with-protocol-buffers-in-web-api.html • Ingredients for well-design OWIN components http://www.continuousimprover.com/search/label/OWIN%20Recip es • The good, the bad and the ugly of Event Sourcing http://www.continuousimprover.com/search/label/event%20sourci ng