Wojtek Suwała, Head Architect, ASC LAB
CQRS and Event Sourcing
Intro For Developers
Agenda
• What CQRS is
• Why CQRS?
• Is CQRS an architecture?
• Examples of implementation (evolutionary approach)
• CQRS pros & cons
• What Event Sourcing is
• Event Sourcing pros & cons
• Should I use CQRS/ES frameworks?
• Advices
CQRS
Command-Query Responsibility Segregation - a pattern that
tells us to separate operations that mutate data from the
ones that query it.
It is derived from Command Query Separation (CQS) principle
devised by Bertrand Mayer - author of Eiffel.
CQS states that there can be only two kind of methods on a
class: the ones that mutate state and return void and the
ones that return state but do not change it.
CQRS - example
interface PolicyService {
void ConvertOfferToPolicy(ConvertOfferRequest convertReq);
PolicyDetailsDto GetPolicy(long id);
void AnnexPolicy(AnnexRequestDto annexReq);
List<PolicyInfoDto> SearchPolicies(PolicySerachFilter filter);
void TerminatePolicy(TerminatePolicyRequest terminateReq);
void ChangePayer(ChangePayerRequest req);
List<PolicyInfoDto> FindPoliciesToRenew(RenewFilter filter);
}
CQRS - example
interface PolicyComandService {
void ConvertOfferToPolicy(ConvertOfferRequest convertReq);
void AnnexPolicy(AnnexRequestDto annexReq);
void TerminatePolicy(TerminatePolicyRequest terminateReq);
void ChangePayer(ChangePayerRequest req);
}
interface PolicyQueryService {
PolicyDetailsDto GetPolicy(long id);
List<PolicyInfoDto> SearchPolicies(PolicySerachFilter filter);
List<PolicyInfoDto> FindPoliciesToRenew(RenewFilter filter);
}
Why CQRS?
What CQRS enables:
Most of the time data needed to change state are different in
form or quantity than data needed by the user to make a
decision. Having the same model to handle both queries and
commands result in a model that is bloated with things
needed only by one type of operations, also model
complexity is increased and aggregate size is usually bigger.
CQRS enables us to have different models for mutating
state and different models to support queries.
Why CQRS?
Usually write operation are less frequent than reads.
Having separate models and classes allows us to
independently scale the query side and to better handle
concurrent access as reads no longer block writes or the
command side (in opposite case).
Why CQRS?
Having separate models for commands and queries let us
assign these responsibilities to different teams with different
skills.
For example you can assign command handler to highly
skilled OOP developers while query side can be implemented
by SQL developers.
CQRS let you scale your team and
let you best devs focus on the core stuff.
Is CQRS an architecture?
People often get it wrong:
CQRS IS NOT A TOP LEVEL / SYSTEM LEVEL ARCHITECTURE.
Example of architecture styles are: layered, ports & adapters.
CQRS is a pattern you apply on “the insides” of your service /
application and you may apply it only to portion of your
service.
Examples of
implementation -
evolutionary
approach
No CQRS
Separate Commands and Queries
Separate Models Commands and Queries
Separate Models Commands and Queries
We usually use RDBMS with ORM in our applications.
With this setup we can:
• use our entities as command model
• define views in a database, map it with ORM and use it as
the query model or simulate “materialized views” by
programmatically updating tables designed just for
queries
Separate Models Commands and Queries
Query models:
• One table/view per screen or widget
• Denormalized data model
• Relation between tables like relations between screen
elements
• “View tables” contain columns per each field displayed
on the screen
• Read model does not do any computation, calculate data
in command model, store precomputed data in query
model
• Don’t be afraid of duplicates
Separate Models Commands and Queries
Query models:
Separate Models Commands and Queries
When we have two separate models
we need to synchronize them.
• If database views are used RDBMS takes care for us.
• Otherwise query model must be updated as a result of
updating command model.
This can be done by:
– synchronously in the same transaction using something like
Application Events in Spring
– synchronously in the same transaction in command handler
– asynchronously using some kind of in-memory event bus,
resulting in eventual consistency
– asynchronously using some kind of queue middleware like
RabbitMQ, resulting in eventual consistency
Separate Models Commands and Queries
Command models:
• Requires task based UI, it will be hard to do CQRS with CRUD UI
• Before sending command do initial validation in UI to maximize
chances of success
• One command usually maps to one aggregate
• Domain model acts on actual system state not on what user see
on the screen
• Command should usually after validation retrieve domain object
(aggregate root) using repository and pass it command message
for processing
• Domain model performs business logic, rises domain events,
calls domain services, enforces consistency within aggregate
root
Separate Models Commands and Queries
Command models:
• Model gets simplified: some associations used only for
reads no longer necessary, some fields no longer
necessary
• In most cases as one command usually interacts with one
aggregate root we should no longer need many-to-many
and most of one-to-many relationship
• We may consider abandoning RDBMS for domain model
storage: we only need to load it by business id, maybe
object database or document store is more sufficient
• Use Domain events to notify and synchronize (bounded
contexts)
Separate Storage Engines
Event Sourcing
Examples of implementation
• Separate class for Queries and for Command handlers
• Separate classes for each query and each command
handler
• Separate models for Queries and for Command handlers
• Different storage engines for Queries models and for
state change models (ElasticSearch/JPA,
ElasticSearch/Document Db, Document Db/Storing
aggregates as JSON in RDBMS…)
• Separate modules / microservices for query and for state
changes
• Use of Event Sourcing for the write side
CQRS pros & cons
Pros:
• better performance and scalability of your system
• better concurrent access handling
• better team scalability
• less complex domain model and simple query model
Cons:
• read and write models must be kept in sync
• maintenance and administration costs if you choose two
different engines for read and for write side
• eventual consistency is not always allowed
What Event Sourcing is
Event Sourcing is a pattern for managing state.
Instead of keeping only current state of the system by
altering previous state, we append events (changes) to
sequential list of past events (changes).
This way we not only know the current state of the system
but we can easily track how did we reach this state.
What Event Sourcing is
Traditional
state
management
__________________________________________________
Event Sourcing
based state
management
Event Sourcing pros & cons
Pros:
• append only model is great for performance, scalability
• no deadlocks!
• events (facts) are well understood by the business expert, some domains are
inherently event sourced: accounting, healthcare, trading
• audit trail for free
• we can get object state at any point in time
• easy to test and debug
• data model decoupled from domain model
• no impedance mismatch (Object model vs Data Model)
• flexibility - many different domain models can be constructed from the same
stream of events
• we can use this model with reversing events, retroactive events
Event Sourcing pros & cons
Cons:
• not natural way for developers to manage state and construct aggregates,
takes time to get used to
• querying beyond one aggregate is a bit harder (you have to construct
projections for each type of query you want to add to the system)
• event schema changes is much harder than in case of relational model (lack
of schema migration tools)
• you must consider versioning handling from the beginning
Should I use CQRS/ES frameworks?
Example CQRS/ES frameworks:
Axon, Lagom, Eventuate, EventFlow
Should I use it?
If you are not experienced with CQRS you should not start
with any framework.
Focus on your domain and core functionality.
Use simple events store (a file ) / in-memory bus.
Once things start to work evaluate frameworks / other
infrastructure components.
Advices
There two camps: one says you should always use CQRS/ES
the other says you should use only for parts of your solution
and only when you need in highly concurrent systems with
high performance/availability/scalability systems.
You should always evaluate your choices in the context of
your requirements.
It is safe to use CQRS simplest form to handle search screens
- use database views/materialized views or dedicated search
engines like ElasticSearch to simplify querying and to allow
better scaling of searches.
Advices
CQRS may results in a very complex technical solution if you
choose to use different storage engines, event buses and
other technical components.
Only some complex scenarios and scalability requirements
justify this complexity (if you run on Netflix scale). At the
same time you can also apply CQRS using simple technical
solution and benefit from this pattern -
you don’t need Kafka to do CQRS.
If your command model and read model are very similar it’s
better to use one model.
Links
CQRS by Martin Fowler
Simple CQRS by Vaughn Vernon
CQRS, Task Based UIs, Event Sourcing agh! by Greg Young
Types of CQRS by Vladimir Khorikov
When to Avoid CQRS by Udi Dahan
Event Sourcing by Martin Fowler
Event Sourcing Pattern by Microsoft
Why Use Event Sourcing by Mirosław Pragłowski
Why Use Event Sourcing by Greg Young
32
altkomsoftware.pl
github.com/asc-lab
@AltkomSC
asc-lab@altkom.pl

Cqrs and Event Sourcing Intro For Developers

  • 1.
    Wojtek Suwała, HeadArchitect, ASC LAB CQRS and Event Sourcing Intro For Developers
  • 2.
    Agenda • What CQRSis • Why CQRS? • Is CQRS an architecture? • Examples of implementation (evolutionary approach) • CQRS pros & cons • What Event Sourcing is • Event Sourcing pros & cons • Should I use CQRS/ES frameworks? • Advices
  • 3.
    CQRS Command-Query Responsibility Segregation- a pattern that tells us to separate operations that mutate data from the ones that query it. It is derived from Command Query Separation (CQS) principle devised by Bertrand Mayer - author of Eiffel. CQS states that there can be only two kind of methods on a class: the ones that mutate state and return void and the ones that return state but do not change it.
  • 4.
    CQRS - example interfacePolicyService { void ConvertOfferToPolicy(ConvertOfferRequest convertReq); PolicyDetailsDto GetPolicy(long id); void AnnexPolicy(AnnexRequestDto annexReq); List<PolicyInfoDto> SearchPolicies(PolicySerachFilter filter); void TerminatePolicy(TerminatePolicyRequest terminateReq); void ChangePayer(ChangePayerRequest req); List<PolicyInfoDto> FindPoliciesToRenew(RenewFilter filter); }
  • 5.
    CQRS - example interfacePolicyComandService { void ConvertOfferToPolicy(ConvertOfferRequest convertReq); void AnnexPolicy(AnnexRequestDto annexReq); void TerminatePolicy(TerminatePolicyRequest terminateReq); void ChangePayer(ChangePayerRequest req); } interface PolicyQueryService { PolicyDetailsDto GetPolicy(long id); List<PolicyInfoDto> SearchPolicies(PolicySerachFilter filter); List<PolicyInfoDto> FindPoliciesToRenew(RenewFilter filter); }
  • 6.
    Why CQRS? What CQRSenables: Most of the time data needed to change state are different in form or quantity than data needed by the user to make a decision. Having the same model to handle both queries and commands result in a model that is bloated with things needed only by one type of operations, also model complexity is increased and aggregate size is usually bigger. CQRS enables us to have different models for mutating state and different models to support queries.
  • 7.
    Why CQRS? Usually writeoperation are less frequent than reads. Having separate models and classes allows us to independently scale the query side and to better handle concurrent access as reads no longer block writes or the command side (in opposite case).
  • 8.
    Why CQRS? Having separatemodels for commands and queries let us assign these responsibilities to different teams with different skills. For example you can assign command handler to highly skilled OOP developers while query side can be implemented by SQL developers. CQRS let you scale your team and let you best devs focus on the core stuff.
  • 9.
    Is CQRS anarchitecture? People often get it wrong: CQRS IS NOT A TOP LEVEL / SYSTEM LEVEL ARCHITECTURE. Example of architecture styles are: layered, ports & adapters. CQRS is a pattern you apply on “the insides” of your service / application and you may apply it only to portion of your service.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
    Separate Models Commandsand Queries We usually use RDBMS with ORM in our applications. With this setup we can: • use our entities as command model • define views in a database, map it with ORM and use it as the query model or simulate “materialized views” by programmatically updating tables designed just for queries
  • 15.
    Separate Models Commandsand Queries Query models: • One table/view per screen or widget • Denormalized data model • Relation between tables like relations between screen elements • “View tables” contain columns per each field displayed on the screen • Read model does not do any computation, calculate data in command model, store precomputed data in query model • Don’t be afraid of duplicates
  • 16.
    Separate Models Commandsand Queries Query models:
  • 17.
    Separate Models Commandsand Queries When we have two separate models we need to synchronize them. • If database views are used RDBMS takes care for us. • Otherwise query model must be updated as a result of updating command model. This can be done by: – synchronously in the same transaction using something like Application Events in Spring – synchronously in the same transaction in command handler – asynchronously using some kind of in-memory event bus, resulting in eventual consistency – asynchronously using some kind of queue middleware like RabbitMQ, resulting in eventual consistency
  • 18.
    Separate Models Commandsand Queries Command models: • Requires task based UI, it will be hard to do CQRS with CRUD UI • Before sending command do initial validation in UI to maximize chances of success • One command usually maps to one aggregate • Domain model acts on actual system state not on what user see on the screen • Command should usually after validation retrieve domain object (aggregate root) using repository and pass it command message for processing • Domain model performs business logic, rises domain events, calls domain services, enforces consistency within aggregate root
  • 19.
    Separate Models Commandsand Queries Command models: • Model gets simplified: some associations used only for reads no longer necessary, some fields no longer necessary • In most cases as one command usually interacts with one aggregate root we should no longer need many-to-many and most of one-to-many relationship • We may consider abandoning RDBMS for domain model storage: we only need to load it by business id, maybe object database or document store is more sufficient • Use Domain events to notify and synchronize (bounded contexts)
  • 20.
  • 21.
  • 22.
    Examples of implementation •Separate class for Queries and for Command handlers • Separate classes for each query and each command handler • Separate models for Queries and for Command handlers • Different storage engines for Queries models and for state change models (ElasticSearch/JPA, ElasticSearch/Document Db, Document Db/Storing aggregates as JSON in RDBMS…) • Separate modules / microservices for query and for state changes • Use of Event Sourcing for the write side
  • 23.
    CQRS pros &cons Pros: • better performance and scalability of your system • better concurrent access handling • better team scalability • less complex domain model and simple query model Cons: • read and write models must be kept in sync • maintenance and administration costs if you choose two different engines for read and for write side • eventual consistency is not always allowed
  • 24.
    What Event Sourcingis Event Sourcing is a pattern for managing state. Instead of keeping only current state of the system by altering previous state, we append events (changes) to sequential list of past events (changes). This way we not only know the current state of the system but we can easily track how did we reach this state.
  • 25.
    What Event Sourcingis Traditional state management __________________________________________________ Event Sourcing based state management
  • 26.
    Event Sourcing pros& cons Pros: • append only model is great for performance, scalability • no deadlocks! • events (facts) are well understood by the business expert, some domains are inherently event sourced: accounting, healthcare, trading • audit trail for free • we can get object state at any point in time • easy to test and debug • data model decoupled from domain model • no impedance mismatch (Object model vs Data Model) • flexibility - many different domain models can be constructed from the same stream of events • we can use this model with reversing events, retroactive events
  • 27.
    Event Sourcing pros& cons Cons: • not natural way for developers to manage state and construct aggregates, takes time to get used to • querying beyond one aggregate is a bit harder (you have to construct projections for each type of query you want to add to the system) • event schema changes is much harder than in case of relational model (lack of schema migration tools) • you must consider versioning handling from the beginning
  • 28.
    Should I useCQRS/ES frameworks? Example CQRS/ES frameworks: Axon, Lagom, Eventuate, EventFlow Should I use it? If you are not experienced with CQRS you should not start with any framework. Focus on your domain and core functionality. Use simple events store (a file ) / in-memory bus. Once things start to work evaluate frameworks / other infrastructure components.
  • 29.
    Advices There two camps:one says you should always use CQRS/ES the other says you should use only for parts of your solution and only when you need in highly concurrent systems with high performance/availability/scalability systems. You should always evaluate your choices in the context of your requirements. It is safe to use CQRS simplest form to handle search screens - use database views/materialized views or dedicated search engines like ElasticSearch to simplify querying and to allow better scaling of searches.
  • 30.
    Advices CQRS may resultsin a very complex technical solution if you choose to use different storage engines, event buses and other technical components. Only some complex scenarios and scalability requirements justify this complexity (if you run on Netflix scale). At the same time you can also apply CQRS using simple technical solution and benefit from this pattern - you don’t need Kafka to do CQRS. If your command model and read model are very similar it’s better to use one model.
  • 31.
    Links CQRS by MartinFowler Simple CQRS by Vaughn Vernon CQRS, Task Based UIs, Event Sourcing agh! by Greg Young Types of CQRS by Vladimir Khorikov When to Avoid CQRS by Udi Dahan Event Sourcing by Martin Fowler Event Sourcing Pattern by Microsoft Why Use Event Sourcing by Mirosław Pragłowski Why Use Event Sourcing by Greg Young
  • 32.

Editor's Notes

  • #4 Note: 1)That’s it. This is just a pattern which is very simple but has powerful consequences. 2) Eiffel - the best OOP language that nobodys is using :(
  • #7 Example: adding a product to order requires only productId and price but user must see also product name, description etc... Big aggregates result in worse performance, limited scalability and is potential deadlock source
  • #9 Note: junior devs can write projections and query handlers also
  • #10 An architectural style describes the pattern that is followed by an entire application or service (possibly even a microservice), and not just some small portion of the the overall application or service
  • #15 with this setup read side implementation is just select * from MyView where id=:id
  • #18 https://www.baeldung.com/spring-events if you go async you have to deal with failures!!
  • #21 Events can be emitted from Aggregate or from Command Handler
  • #30 some people say ES should be used everywhere applicability is always based on context