Event Driven Architecture

Wildfire Interactive, Inc. | 167 Hamilton Ave, Palo Alto CA 94301 Suite 200 | (888) 274-0929
Event Driven Architecture
Lourens Naudé – SAPO Codebits 2010

About
• Developer @ http://www.wildfireapp.com
• Ruby / C
• Well versed full stack
• Active interest in Event Driven technologies
and related business processes

Agenda
• Business and systems growth
• Events
• (Lack of) control
• Event Driven Architecture
• Domain Driven Design primer
• Command and Query Responsibility
Segregation
• Complex Event Processing

Maintenance cost is
multiple times more than
development cost

http://en.wikipedia.org/wiki/Open/closed_principle
“Systems should ideally be
open for extension, but
closed for modification”

Driving forces for change
• Fulfillment of business requirements
• Business expansion: rollout of new
processes to drive sales, market penetration etc
• Mergers and acquisitions
• Legislation

Modification risks
• Introduces regression testing and a QA
burden – never compromise a working
system for a functional requirement
• Profiling overhead for (soft) realtime
systems
• Excessive “refactoring” of existing stable
interfaces
• May fracture a business domain

EDA imposes a system
design that's geared for
extension, interoperability
and unanticipated use.

Interoperability VS Integration
• Two or more systems performing a
task / process together
• Integration implies a formal link / bridge
between two systems - high coupling
• Building with interoperability in mind is
the easiest way for long-term integration
• Should be accounted for in system lifecycle as
well: delays, cost, communication etc.

Governance (lay down the law)
• Technology standards, SLAs, development
processes
• Cannot achieve interoperation without
governance
• Poor governance: inappropriate policies,
people (habits) and business structures
• Policies should be enforced at runtime
in an auditable manner
• Otherwise like criminal law without
police and courts

Business layers

Business Layers
• Physical world: sensors and users as
actors
• Transactional: physical world interactions
with the business / domain layer
• Intelligence: reporting tier and insights,
most notably sales and marketing
performance
• This is where EDA really shines: visibility
into business processes
• Transparent operation for easier
compliance and lower support costs

Events

Event Definition
• Anything that happened (or didn't
happen)
• A change in state
• A condition that triggers a notification
• An event is always named in the past tense
and is immutable
CustomerAddressChanged
InventoryRelocated
OrderShipped

Three levels of detail
• Basic fact that an event occurred
• Event definition required in order to
recognize an event
• Event detail / context
• Always attempt to capture events at the point
of greatest information value (context)
• This reduces downstream lookup and
correlation overhead

Event Types
• Notification: informational
• Alert: notification that should cause a
response
• Event Object: record of an event - just data,
no behavior
• Business Event: meaningful for conducting
trade activities
• Complex: abstraction of one or more other
events that represents summary level insights

Notification Types
• Observational
• Reports an event
but
does not change
anything
LowStock
CartCleared
• Transactional
• Reports an event
and
causes change
• Reliable transport
• Originates from
within the org.
OrderShipped
CardProcessed

(Lack of) Control

Event Driven Programming
• Control flow is driven by external
events
• NO central controller (main) - contrary
to what we're taught starting off as devs
• No well defined flow of data

EDA Components
• Event producers
• Event consumers / listeners
• Event processors
• Event reactions
• Messaging infrastructure

Lack of control in EDA
• An EDA is driven by extensibility and
not controllability
• Transaction state is managed by
events, not some central controller
• Loosely coupled, asynchronous and stateless
• Tell downstream components what
happened, but not what to do.
• Business events enable a system to operate
under control, but without central
control

Reduced coupling
• There's always some coupling
• We strive to reduce the degree of
coupling
• Always coupled to data

Stateless Order process
• Modifying processes managed by a central
controller requires changes to a component
as well as the controller
• Move state / data from the central controller
to events

Event Driven Architecture

Use cases
• Government: security, border control,
taxes etc.
• Health: stream processing + research
• Compliance: guard against violations in
cross country laws
• Track and trace: Fedex, UPS etc.
• Service Level Agreements

When not to use an EDA
• If the potential for reuse is very little
• Problem being solved is lower down the
application stack
• Network bandwith and capacity is
limited – serialization overheads and tends
to be chatty
• Security boundaries: immutability of
events that carry sensitive data

EDA Workloads
• Simple event processing: narrow design
• Stream processing: high message
volume, but only relevant events is filtered
• Complex Event Processing (CEP): able
to correlate multiple streams

Interaction Types
• Time-driven: actions initiated at specified
times, eg. batch workflows etc.
• Request-driven: initiated by client and
completes when the provider replies, eg.
client server
• Event-driven: initiated by events and
participants is open ended

Hybrid sytems
Most event driven systems
also include time and request
driven components.

Components
• Event producers
• Event consumers / listeners
• Event processors
• Event reactions
• Messaging infrastructure

Nervous System Analogy
• Hand is the event producer
• Spinal cord is the ESB
• Brain is the event listener
• Brain is the event processor
• Hand is the event reaction – we pull away

Event Producers
• Generates event data OR transforms it
into a format that the EDA can
understand
• Data isn't always generated to feed an
EDA, especially if the producer is a legacy
component

Event Consumers / Listeners
• Knows how to differentiate an event
from data it receives
• Can only “hear” events it's supposed
to
• Should be able to detect and interpret an
event

Event Processors
• Able to asses event importance and
derives the next action to take
• No value in not handling a perceived event
• "Do nothing" is a valid reaction as
well

Event Reactions
• Automated responses
• Notifications to participants or people
• Human reactions / interventions for
offline processes

Messaging Infrastructure
• Communication link between EDA
participants
• Doesn't have to be a single piece of
infrastructure - distinct backbones
should be able to share information
• Embrace standards and be as universal as
possible
• Promote decoupling between
participants - pub/sub

Data in Event Driven
Systems

Event Data
• Business event data held in an event
object
• Context of what happened
ProductCategorized('dsaed32wed', 'Coffee')
OrderCancelled('ko90323', 'No funds')

State Data
• Core system data that changes as a
result of operational activity
Order('ko90323', 'Tom Jones', 13.45 'open')
Product('dsaed32wed', 'Delta', 'Coffee', 5.8)

Reference Data
• Stable / somewhat permanent - could
change, but a lot less frequently than
state data
• Reference data would be replaced whereas
state data would change incrementally.
SupportedCurrency('USD')
ProductCategory('Coffee')
Address('Some Street', 'CA', 'USA')

Lifecycle of Data
• New state data = state data + event
data applied
• State data is the accumulation of event
data over time
• Event data as deltas
Product('dsaed32wed', 'Delta', 'Coffee', 5.8)
ProductCategorized('dsaed32wed', 'Tea')
Product('dsaed32wed', 'Delta', 'Tea', 5.8)

Domain Driven Design
primer

Domain Driven Design
• Suitable for behavior driven systems
• Modeling is often data and not
behavior driven
• A Domain Model is a representation of
relationships between Entities and Values
• Try to have a business analyst review
processes before introducing new
architecture

Common Language
• Participants: Developers, Experts and
Stakeholders
• Nouns: things with / without identity
• Verbs: actions, hints at state changes
• Domain models evolve through language
• We often write different to how we speak and
think
• Most failures of systems is communication
failures of people

Entities
• Has an identity within the system
• State can change
Product
Customer
Order

Value Objects (Values)
• No identity
• Immutable
• Represented by it's structure - structural
equivalence
Money
OrderStatus
Timezone

Domain Services
• Actions, operations and activities
• Business driven intent
• Stateless
• Should handle behavior that doesn't map
well to any particular entity
ShippingQuotesService
PaymentProcessingService

Aggregates
• A group of Entities that belong
together
• An instance of a process
• A single unit of work – consistency /
transactional boundary
• eg. Order and Order items
• Order item is useless without an Order
• Remove Order item: reconstitute the Order,
find item via Order, remove item
• Usually identified by cascading delete
requirements

Rental administration example

Command and Query
Responsibility Segregation

Either perform an action,
or return a result
Not both.

Overview
• CAP theorem: only can have 2 of 3
• Consistency steps aside for Availability and
Partitionability
• Somewhat stale data is often acceptable
• Business requirements define by how much
• Split systems into distinct command and query
components (read and write)

Conceptual view

Benefits
• Split teams: high level + good communicators
on the write side, junior devs on the read side
• Ability to scale command and reporting
sides independently
• Auditable system (dependent on history)
• Selectively disable functionality during
upgrades on the write side whilst the reporting
side is still available
• Lends itself well to behavior driven extension
• Easy to test

CQRS Components
• Commands: drives change in the system
• Internal events: captures intent (event
store)
• External events: republished to let other
components know
• Queries: examines state changes

Eventual Consistency
• All data is stale, unless the whole
system uses pessimistic locking
• Most web apps request data *before*
they render screens
• Add 250ms to 500ms on average for
rendering and additional server side
blocking operations
• Users don't react immediately to UI
changes
• Strictly consistent on the write side, relaxed
(eventual) consistency on the read side

Rethinking UI / adaptors
• Users don't think in data, but in tasks
and processes
• Cannot get benefits of CQRS without
considering the UI first
• Focus on capturing user intent
explicitly
• User experience should be an integral part of
any business process

Ambigious UI
UpdateComment / CommentUpdated

Task Driven UI
ApproveComment / CommentApproved

Commands
• A well defined task with parameters
• Represents a process, not an entity
• Distinct and transactional unit of work:
wholly accepted or rejected
• The only way to change data in a system
• Just the right amount of granularity and intent
to not cause concurrency conflicts of updates
on large entities
• Rejectable with exception or fault event
• Idempotent – should never be reprocessed

Rethinking Validations
• There's a fundamental difference
between a validation and a business rule
• Validation: well structured and formatted etc.
• Business rule: uniqueness constraints etc.
• Validate in client and in command handler -
not all clients are well behaved
• Valid commands can fail, most often
through optimistic concurrency conflicts
• Common for offline processes to reject requests

Queries and Reports
• Read only view of data - examines state
in the system
• Never goes through the domain model
• Table per UI view (assuming relational
storage)
• Single call to the reporting store returns
all (or most) data for a screen
• Minimal / no transformation - reporting
tier is a cache

Aggregates and Events
---------- begin consistency boundary ----------
=> Commands::CategorizeProduct("blends",
"1517fd7efbffacdf928056656bbb61cf")
rebuild
Aggregates::Product(1517fd7efbffacdf928056656bbb61cf)
from Events::ProductCreated({:category=>"coffee",
:product_id=>"1517fd7efbffacdf928056656bbb61cf",
:sku=>"XYZ", :name=>"Delta"})
Aggregates::Product @ version 1
<= Events::ProductCategorized({:category=>:blends,
:product_id=>"1517fd7efbffacdf928056656bbb61cf"})
Aggregates::Product @ new version 2
---------- end consistency boundary ----------

Code example
class Aggregates::Product
......
def rename(name)
apply Event(:ProductRenamed, uuid, name)
end
private
def apply_product_renamed(event)
@name = event.name
end
......
end

Internal Events
• Domain / Event store is append only: no
deletes, ad hoc reads.
• Compensating events to delete
• Event store is a history of intent
• Replay: events allow for reproducing error
conditions by rebuilding an aggregate via
snapshot up to just before a problem occurred
• Republishes any handled events to other
components

External Events
• Events handled by our domain that's
been republished to the rest of our
system
• Communicates any state changes to
the reporting tier
• Integration / interoperation hook for
business growth

ORM Issues
• Suitable for structural models only
• Structural models tend to loose
information
• Mapping objects to structed data is difficult
to optimize
• CRUD – what changes on UPDATE ?

Testing
• Command -> Domain -> [Event, Event …]
• Able to assert behaviors in two ways
• What happened ?
• What didn't happen ?
• Same pattern for testing the reporting side
• Event -> View -> assert state from getters

Assertion example
def test_relocate_inventory
Domain << Command(:CreateInventory, uuid,
123, 'Delta', 'A', 'X',10)
Domain << Command(:RelocateInventory, uuid,
'B', 'Y')
assert_published :InventoryCreated,
:InventoryRelocated
end

Complex Event Processing

Overview
• Connecting the dots
• Summary level information gathered
from base events
• Patterns identified by business experts, but
enforced at runtime by CEP software
• Value is proportional to accuracy and
relevance
• Accurate data can however still be
irrelevent
• Allows for Management by Exception

Consciousness / awareness
• Without consciousness can't react to
situations without human intervention
• SOA's the nervous system
• EDA brings awareness
• The core of Military systems – situational
awareness on the battlefield derived from input
streams

Event Stream
• Linear (ordered by time) sequence of events
• Produced by a single component / system
• Little aggregation required
• High throughput
• Minimal insights into what happened
• Example: market data feed

Event Cloud
• Collection of Event Streams within a system
• Aggregates streams into event sets
• Much lower throughput
• Very good situational awareness
• Example: trading strategies that's
dependent on multiple market feeds,
realtime news sources etc.

Event Cloud layout

CEP in Retrospective
• Adds value to Event Driven systems
• “Complex” is misleading – it simplifies input
and streams
• Matches real-time events from an Event
Cloud with patterns from historical data
and detect or predict situations
• Event -> correlate -> assess -> decide -> action

Summary
• Restist modification
• Tasks and intent – behavior matters
• Communicate
• Reduce coupling and avoid central
control

@methodmissing
http://www.github.com/methodmissing
CQRS Example
https://github.com/methodmissing/aftermath
Thanks !
Questions ?

Event Driven Architecture

More Related Content

What's hot

Similar to Event Driven Architecture

More from Lourens Naudé

Recently uploaded

Event Driven Architecture