Decomposing the Monolith using modern-day .NET and a touch of microservices
Download as PPTX, PDF4 likes379 views
The document discusses the challenges and considerations in software architecture, particularly around microservices and monolithic systems. It emphasizes the importance of scalability, reliability, and design principles such as dependency inversion and eventual consistency, while also highlighting various tools and techniques for implementation. Additionally, it explores event sourcing, data management, and the need for mature DevOps practices in deploying and maintaining complex systems.
![IAppBuilder builder = ….
builder.UseStatistics(stats);
GET http://localhost/projectionStats/CountByDocument
{
"projectorId": "CountByDocument",
"lastCheckpoint": 1000,
"lastCheckpointUpdatedUtc": "2018-05-10T10:39:00Z",
"properties": [{
"key": "some setting key",
"value": "some value",
"lastUpdatedUtc": "2018-05-10T10:39:00Z"
}],
"eventsUrl": "http://localhost/projectionStats/CountByDocument/events"
}
Dennis Doomen | @ddoomen | The Continuous Improver](https://image.slidesharecdn.com/decomposingthemonolithdotnetdays2020-200301180055/85/Decomposing-the-Monolith-using-modern-day-NET-and-a-touch-of-microservices-35-320.jpg)
![public interface IEventStore
{
IDisposable Subscribe(
long? lastCheckpoint,
Func<Transaction[]>, Task> handler,
string subscriptionId);
}
public delegate IDisposable CreateSubscription(
long? lastCheckpoint,
Func<Transaction[]>, Task> handler,
string subscriptionId);](https://image.slidesharecdn.com/decomposingthemonolithdotnetdays2020-200301180055/85/Decomposing-the-Monolith-using-modern-day-NET-and-a-touch-of-microservices-68-320.jpg)
Decomposing the Monolith using modern-day .NET and a touch of microservices
- 1. Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver
- 2. @ddoomen
- 5. Hybrid of Technologies Multiple patterns for the same problems Long compile time Large source control repositories Long-running unit tests Too much coupling Limited team scalability Devs need to know everything NoIsolation.Productivity drops over time. Proof of success!
- 7. Application Functional Module Functional Module Functional Module Functional Module Composition Each module owns/exposes its master data Does not depend on anything except for curated shared APIs Owns schema and storage (may use shared database abstraction) DRY within module, not outside May use an internal container Does not touch other module’s data directly Shared Services Connects the modules and serves as anti- corruption layer Only services you really think should not be duplicated Designed according to Dependency Inversion Principle Covers front-end and back- end is the scope of unit & integration tests Align folder structure with boundaries Defines the contracts for the host as well as any services it exposes Aligned with Bounded Context.
- 11. What is a microservice? From https://martinfowler.com/articles/microservices.html
- 12. Very scalable, very reliable Very difficult to debug distributed problems Unreliable network requires resiliency techniques Requires message bus/broker/ gateway End-to-end testing very unpractical Very mature DevOps culture is a prerequisite Advanced monitoring and visualization is a must High operational maintenance Can be deployed/upgraded independently Explosion of network activity.
- 13. The First Law of Distributed Computing
- 15. Monolith Functional Module Functional Module Functional Module HTTP API HTTP API HTTP API Released as NuGet packageNo network I/O at all Own database instance, / shared with monolith / or shared storage service Owns the schema Separate repos with distinct owners Runs in-process, thus easier debugging New version requires redeployment of the monolith. Loose coupling through HTTP Less scalability options HTTP is not the fastest serialization format Designed for minimum dependencies
- 17. Client IIS Console WinSvc Unit Test HTTP request HTTP response Middleware environment handler next Middleware IDictionary<string, object> Func<IDictionary<string, object>, Task> Task Func< Func<IDictionary<string, object>, Task>, Func<IDictionary<string, object>, Task> > Func<IDictionary<string, object>, Task>
- 18. Client IIS Console WinSvc Unit Test HTTP request HTTP response Middleware context handler next Middleware HttpContext Task Func< HttpContext, Func<Task>, Task > Func<HttpContext, Task> Func<HttpContext, Task>
- 19. Monolith Functional Module HTTP API Module Registry Contract documented through OpenAPI Webhook Functional Module Functional Module Provides HTTP abstraction to connect to module Event payload exposed as OpenAPI Subscribers register HTTP abstraction Consumers use liberal JSON interpretation Subcribers can request resending history. Payload includes unique ID to help idempotency Each request requires correlation ID that flows through all APIs, webhooks and module
- 21. Dennis Doomen | @ddoomen | The Continuous Improver Application Domain NoSQL / RDBMS OR/M / DAL Web UI, HTTP API, etc Lucene Index Document Projector Web UI, HTTP API, etc Web UI, HTTP API, etc Domain Commands Events Event StoreProjections Projectors Uses Event Sourcing Uses traditional CRUD architecture Indexing-based architecture Subcribe to webhooks Coarse- grained HTTP requests. Bus Subscribe Publish coarse- grained event
- 23. Domain is persistency ignorant For modules that provide a (partial) UI or management screen Aggregates Command Handlers Commands Events Upconverters Value Objects Query Handlers Schema Migrations Projectors Data Importers Query API Projections HTML CSS JS API Controllers Projectors Projections Registrations Webhook API Command API Event Storage Domain Services Payload Definitions Can be tested independently Based on Domain Driven Design Outer layers depend on inner layers Master data
- 24. Application Command Service Correct Customer Email Handler Customer Unit of Work Projector Data Access Layer Read Database Write Database CorrectCustomerEmailCommand HTTP API / In-process invocation Get<Customer>(identity) Correct Email Event Store Load(events) Apply Get changes CustomerEmailCorrectedEvent Submit changes History Dennis Doomen | @ddoomen | The Continuous Improver NES Aggregates.NET SimpleDomain NStore EventStore NEventStore SQLStreamStore NStore NES Marten Handelier Brighter MediatR (*) Projac LiquidProjections (*)
- 25. Monolith Functional Module HTTP API Event Store Events Payload Projector Subscription Request Payload ‘event store’ Have their own checkpoints Projects fine-grained events into payload projections Projector Projector Projector Webhook Projectors Separate instance per subscription Subscriber Callback API ‘Project’ payload ‘events’ into HTTP calls Projected Data Uses the incoming payloads as an ‘event store’. Payload SQL DB Schema changes using code or using NoSQL Subscription Request
- 27. - Module owns ‘schema’ - Supports upgrading and downgrading - NoSQL preferred, but RDBMS can work too.
- 28. Event Store Projector Application Version 2 Owns schema (e.g. FluentMigrator) Version 1 X Involves down-time until projections are rebuild Dennis Doomen | @ddoomen | The Continuous Improver
- 29. Event Store Projector Application Application Network Load Balancer Event Store Version 1 Version 2 events Projection ProjectorProjection bring off-line Returns HTTP 503 (Service Unavailable) Dennis Doomen | @ddoomen | The Continuous Improver Returns HTTP 503 (Service Unavailable)
- 32. Dennis Doomen | @ddoomen | The Continuous Improver
- 33. var mapBuilder = new EventMapBuilder<MyContext, DocumentProjection, string>(); mapBuilder .Map<WarrantAssignedEvent>() .AsCreateOf(anEvent => anEvent.Number) .Using((document, anEvent) => { document.Type = "Warrant"; document.Kind = anEvent.Kind; document.Country = anEvent.Country; document.State = anEvent.InitialState; }); mapBuilder .Map<StateTransitionedEvent>() .When(anEvent => anEvent.State != "Closed") .AsUpdateOf(anEvent => anEvent.DocumentNumber) .Using((document, anEvent) => document.State = anEvent.State); Dennis Doomen | @ddoomen | The Continuous Improver
- 34. var stats = new ProjectionStats(() => DateTime.UtcNow); stats.StoreProperty("CountByDocument", "some setting key", "some value"); stats.LogEvent("CountByDocument", "some significant thing that happened"); stats.TrackProgress("CountByDocument", currentCheckpoint); float? transactionsPerSecond = stats.GetSpeed("CountByDocument"); TimeSpan? eta = stats.GetTimeToReach("CountByDocument", targetCheckpoint); public void Configure(IAppBuilder builder) { builder.UseStatistics(stats); } Dennis Doomen | @ddoomen | The Continuous Improver
- 35. IAppBuilder builder = …. builder.UseStatistics(stats); GET http://localhost/projectionStats/CountByDocument { "projectorId": "CountByDocument", "lastCheckpoint": 1000, "lastCheckpointUpdatedUtc": "2018-05-10T10:39:00Z", "properties": [{ "key": "some setting key", "value": "some value", "lastUpdatedUtc": "2018-05-10T10:39:00Z" }], "eventsUrl": "http://localhost/projectionStats/CountByDocument/events" } Dennis Doomen | @ddoomen | The Continuous Improver
- 37. Service Host Monolith Functional Module Functional Module Functional Module HTTP API HTTP API HTTP API Apply routing conventions Versioning using Accept Headers, Response Headers or Routes Optimize body using AVRO or Protobuf Cross-service tracing using OpenTracing Replace HTTP with gRPC Common contracts for diagnostics, monitoring HTTP Proxy Child process of monolith (e.g. LitteForker) Hides the externalized state of the module (e.g. ProxyKit)
- 39. 1 Build a (container-based) cloud capability. 2 Deploy monolith to the cloud. 3 Automate your delivery pipeline. 4 Full end-to-end responsibility 5 Evaluate status-quo on micro-services (products, platforms, etc) 6 Slide-and-dice microservices one-by- one 7 Break-up delivery teams and code.
- 40. Dennis Doomen @ddoomen Dennis Doomen | @ddoomen | The Continuous Improver
- 41. • Liquid Projections https://www.liquidprojections.net • Bliki: MonolithFirst https://dzone.com/articles/martin-fowler%E2%80%94bliki • In-process gRPC https://github.com/grpc/grpc-go/issues/906 • Ingredients for well-designed OWIN components https://www.continuousimprover.com/search/label/OWIN%20 Recipes • The good, the bad and the ugly of Event Sourcing https://www.continuousimprover.com/search/label/event%20s ourcing
- 42. • Aggregates.NET https://github.com/volak/Aggregates.NET • EventFlow https://github.com/eventflow/EventFlow • OWIN in ASP.NET Core https://docs.microsoft.com/en- us/aspnet/core/fundamentals/owin?view=aspnetcore-2.1 • Nested ASP.NET Core applications http://dhickey.ie/2018/06/09/aspnet-core-nested-apps/ • LittleForker https://github.com/damianh/LittleForker • ProxyKit https://github.com/ProxyKit/ProxyKit
- 44. Domain Event Store Events App RDBMS Events Projection EventsProjection Projection Projector Optimized for specific queries Separate projections database NoSQL Projector Projection-specific storage Projector HTML Raw SQL or Dapper Run asynchronously Great for sharding Dennis Doomen | @ddoomen | The Continuous Improver
- 45. Events Transaction 6 Transaction 5 Transaction 4 Transaction 3 Transaction 2 Transaction 1 Temporal Projector Read Store Time Projected until this point Immutable, thus auditable and SOX compliance Dennis Doomen | @ddoomen | The Continuous Improver
- 46. Domain Event Store Events App Events Projection EventsProjection Projection Projector Application projections RDBMS Reporting Projector Traditional reporting model Asynchronous OLAP Dennis Doomen | @ddoomen | The Continuous Improver
- 47. Dennis (v4) Persisted State Changes Dennis (v3) Role Granted PasswordChangedEvent Dennis (v4) Dennis (v3) User Created Role Granted Phone Number Added Password Changed Dennis (v5) Role Revoked Dennis (v6) Role Granted Time Dennis (v7) Password Changed Dennis Doomen | @ddoomen | The Continuous Improver
- 48. Events 1 2 3 6297 6298 6298 Projections Documents Groups Users Objects Folders Microservice V2 Changes Queries Events 1 2 3 6299 6300 6301 Projections Documents Groups Users Objects Folders Microservice V1 ChangesQueries Migration Process Dennis Doomen | @ddoomen | The Continuous Improver
- 49. Events Aggregate Root Entity Entity Value Object Aggregate Root Aggregate Root Aggregate Root Entity Value Object Value Object Dennis Doomen | @ddoomen | The Continuous Improver
- 51. Designing your domain based on ownership
- 52. Relying on eventual consistent projections
- 53. Bad choice in functional keys (e.g. username vs ID)
- 54. Running business logic after an domain event is raised
- 55. Domain-specific value types in events
- 57. Ask questions about consistency Understand the real world
- 58. Driven by invariants, not composition Only consistent within aggregate Small aggregates Reference by identity
- 60. • Column constraints (e.g. data truncation) • Changes in data invariants (null vs non- null in event versions) • Unexpected projection dependencies • Partial replays.
- 61. • Causing duplicate child records • Causing large event streams • Incorrect caching strategy (e.g. on lookups) • Identity case-sensitivity • Incomplete SQL-backed event store reads.
- 62. • Side by side rebuilding • Functional archivability and archiving • Projection tracking and ETAs • Prioritization • Partitioning.
- 63. • After bugs, schema changes, etc • Manual or automatic (e.g. hashes)
- 65. Microservice Bunch of classes that should be used directly X No inheritance Uses composition over inheritance internally Convenience classes that don’t hide the magic Library Abstract Package Interfaces Delegates DTOs Only depend on abstract packages… Stable Package …or depend on more stable packages Auxiliary Package Classes that are not used together do not belong togetherOptional Dependency Dependency Package Consumers should not be faced with optional dependencies
- 68. public interface IEventStore { IDisposable Subscribe( long? lastCheckpoint, Func<Transaction[]>, Task> handler, string subscriptionId); } public delegate IDisposable CreateSubscription( long? lastCheckpoint, Func<Transaction[]>, Task> handler, string subscriptionId);
- 69. e.g. LibLog, TinyIoc, FluidCaching
- 72. • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology doesn't change • There is one administrator • Transport cost is zero • The network is homogeneous. https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing