RedisConf18 - Scalable Microservices with Event Sourcing and Redis

Editor's Notes

• #3: While adopting a microservices, or service based architecture comes with a lot of benefits including ease of deployment, resiliency, reliability, and performance, it also comes with it’s own set of challenges that we don’t have with a monolithic application. Today I will talk about how we can address some of those challenges by making our services event-driven with the help of Redis. I’ll talk about some useful patterns for implementing event-driven services with Redis, and then do a brief walkthrough of a demo application that shows these patterns in practice.
• #4: One of the main issues we face with microservices is how do services communicate with one another. Often, communication is implemented with HTTP calls between services. So, for example… Three main problems: Services are tightly coupled, The Order Service must know about the Customer Service and it’s API; If Customer service is down, we are forced to either have inconsistent data or to implement distributed transactions and give the user an error; Performance issues, UI must wait for all of the various API calls to complete successfully.
• #6: Decouples services, now Order Service doesn’t need to know anything about Customer Service or Inventory Service Writes are now inherently atomic as they only need to write to the message queue and we don’t need to worry about distributed transactions. The system can maintain consistency and availability although it is now “eventually consistent”. If Customer service goes down, it can still process it’s events waiting in the queue once it comes back up. All updates to other services are now asynchronous. The client does not have to wait for multiple calls to multiple services. However, eventual consistency must be kept in mind here too. Updates to data in Customer Service or Inventory Service won’t be immediate.
• #8: Redis pub/sub is very basi. It does not persist events, so if there is no subscriber at the time the event is written, the data will be lost. Therefore, we have to implement a more reliable mechanism to prevent data loss. 1. An event subscriber registers itself in a Subscribers List in Redis. 2. When an event is published, event data is written to a Published List for each subscriber and a Redis message is published in an atomic operation. The event does not contain any data itself — it is simply notifying subscribers that there is event data waiting to be processed in the Published List. If a subscriber goes down, data will continue to be written to its Published List and can be picked up and processed by the subscriber once it comes back up. 3. When a subscriber receives an event notification, it pops the event data from the Published List and pushes it to another Processing List in an atomic operation using the RPOPLPUSH command. This makes it possible to have multiple instances of a subscriber running, but ensures that only one of them will actually pick up the event data and process it. All instances will receive the event notification, but only one of them can pick up the event data and process it. 4. Once processing is complete, event data is removed from the Processing List. If an error is encountered during processing, data remains in the Processing List and processing can be retried at a later time.
• #9: ““Instead of storing just the current state of the data in a domain, use an append-only store to record the full series of actions taken on that data. The store acts as the system of record and can be used to materialize the domain objects. This can simplify tasks in complex domains, by avoiding the need to synchronize the data model and the business domain, while improving performance, scalability, and responsiveness. It can also provide consistency for transactional data, and maintain full audit trails and history that can enable compensating actions.” The fundamental primitive in the EventStore is a domain object or Aggregate. Going back to the example above this would be something like an Order. The state of the Aggregate object is stored as a series of events. For example, there may be something like OrderCreated, QuantityChanged, OrderDeleted, etc.
• #10: Within the event store, data is stored using a combination of data types offered by Redis: A Redis List stores the stream of events for an Aggregate. Redis Hashes store the actual data associated with each event. Each Aggregate recieves a unique Aggregate ID in the form of a GUID. Each Aggregate then has a Commit List associated with it that is keyed off of the unique Aggregate ID. Each event that is stored for a particular Aggregate recieves a unique Commit ID. The Commit List for the Aggregate simply stores all of the Commit IDs associated with the Aggregate in the order that they were created. A Redis List is used for this because it maintains the order of the values as they are inserted and adding values to the end of a list is very fast as it is an O(1) operation. For each Commit ID within the Commit List, there is a corresponding entry in a Redis Hash that uses the Commit ID as the key and stores the JSON formatted event data as the value. Retrieving a single entry for a Hash is also a fast O(1) operation. In order to retrieve all events for an Aggregate, the entire Commit List can be retrieved using the Redis LRANGE command. The list can then be iterated through and an HGET command issued to retrieve the data for each event. We use a CRC32 hash of the Commit ID and a modulo operation to partition the event data into multiple Redis Hashes.
• #11: CQRS means "Command-query responsibility segregation". We segregate the responsibility between commands (write requests) and queries (read requests). The write requests and the read requests are handled by different objects. This makes sense with Event Sourcing and event driven systems in general because all of our commands are publishing events and all of our queries are run against a “read model” that is written to by our event subscribers.