Utilizing Point-in-Time Queries in Event-Based Systems, Bobby Calderwood | Current 2022
0 likes419 views
Bobby Calderwood discusses the utilization of point-in-time queries in event-driven systems to maintain consistency and transparency in decision-making. He emphasizes the importance of a stable basis for queries through various data storage solutions, enabling features like time travel queries and error reconciliation. The document highlights the integration of technologies like Datomic, CRDTs, and Kafka to support a versioned event model repository in the Onote application.
Utilizing Point-in-Time Queries in Event-Based Systems, Bobby Calderwood | Current 2022
- 1. Bobby Calderwood — October 2022 Absolute Consistency: Utilizing Point-in-Time Queries in Event-Based Systems
- 2. The goal and the problem • My name is Bobby, and my team makes https://oNote.com • Our goal is to help software teams design, implement, and operate event-driven systems • Design with your team on our collaborative Event Modeling canvas • Implement via schema design, code generation, and functional domain modeling • Operate systems whose state is consistent and evident at all times
- 3. Systems whose state is consistent and evident at all times
- 4. Correlating Business Events with Versioned Read Models
- 5. Correlating Business Events with Versioned Read Models
- 6. Correlating Business Events with Versioned Read Models
- 7. Examples of Point-in-Time Queryable Data Stores • Git (content-addressed) • Datomic/XTDB (EAVT fact-based) • Certain CRDTs • Materialize/CockroachDB/Snow fl ake/BigQuery • Traditional RDBMSs via temporal tables and SQL FOR SYSTEM_TIME and AS OF • Domain speci fi c (e.g. our own Event indexing service)
- 8. Characteristics of PiT Queryable Data Stores • Immutable/append only, often log based • New facts supersede/succeed old ones in a speci fi c way • Often (but not always) able to evict data for regulatory/compliance reasons, while maintaining causal integrity • Time/causal order has a fi rst-class representation, a system “clock”
- 9. Why would I need this?
- 10. Why would I need Git?
- 11. — Rich Hickey, Database as a Value, GOTO 2012 If you have a business, you can’t make decisions if you don’t know what happened before yesterday.
- 12. Stable Basis for Decisions • Issue the same query and get the same results, even as the system makes ongoing progress • Multiple distributed participants in the system can agree on and make decisions based on consistent point-in-time state • System basis/clock can be serialized, sent, stored, etc. to coordinate reads across participants and time
- 13. Multiversion Concurrency Control • With a stable basis for query, we can ensure that writes happen against last-seen state, or else abort • GET /foos -> query DB at latest basis, include that basis in e.g. form data on page • PUT /foos/bar -> write to DB unless basis is greater than the one contained in request • Ensures Command processing of PUT request emits Events based on immediate predecessor of relevant state or not at all • No more clobbering other users’ (or browser tabs’) writes with decisions based on stale data!
- 14. Support Different Data Access Patterns • Some data access pattern workloads are long-running and/or tolerate staleness • Other data access patterns require more current data • These queries put different load on OLTP data stores, might affect availability differently
- 15. Full Context, Traceability, and Transparency • Why did we make this decision at that point in the past? • What changed from then to now? • Time travel queries! • What caused this particular problem? • Audit, compliance, BI, etc. • Analytics often aren’t enough to detect qualitative changes
- 16. Business recovery or reconciliation of errors • Like git blame, but for data • Speeds up time to recovery from business errors • Detect patterns to preemptively catch similar errors in future • Example: Service Member Civil Relief Act
- 17. How does it work in practice?
- 18. Datomic and XTDB (EAVT fact-based) • Clock: monotonically increasing integer identifying each transaction • Transactions are fi rst-class entities, so we can add Event ID as metadata on the transaction • Correlation and traceability • Enables query as-of a particular Event ID!
- 19. Datomic and XTDB (EAVT fact-based) • TODO: code listing
- 20. OpSet-based CRDT • Clock: map of actor ID -> last counter seen for actor • Can interpret the value based on a clock-de fi ned subset of the OpSet
- 21. OpSet-based CRDT • TODO: code listing
- 22. Domain Specific: Event Indexing Service • Clock: map of stream name -> count of events on stream, shortens to overall count of events across all streams • Can query state of any stream at a given clock state • Supports MVCC on write: assert that a given stream must be at a particular revision, else fail appending the event
- 23. Domain Specific: Event Indexing Service • TODO: code listing
- 24. Putting it all together: oNote Event Model Repository
- 25. Event Clock and CRDT Clock • Event Clock from our event indexing service • Map of stream -> event count • Used for MVCC and to fetch fi ne-grained streams per entity • CRDT Sync operation • Appends an Event with a CRDT patch based on client clock and server clock to Event Stream • Responds with patch to bring client up to latest server clock • CRDT query: fetch Event Model at a particular clock state
- 26. Syncing with Git Hosts • Addition of CRDT patch fi le to con fi gured directory syncs CRDT state with speci fi c Git SHA • Changes can come from either: • oNote UI as Sync operations • Git host via branch merges, etc. • Kafka-based event processor ensures all changes are merged and made available in oNote app and in con fi gured Git repo
- 27. Conclusion • Event Log + versioned Read Models provide: • Fully consistent point-in-time state that results from each business event • Transparency, analytics, audit trail for each state transition in system • Kafka + Event indexing provide a meta-Read Model for Events on streams, enabling fi ne-grained streams and MVCC for event-based applications • Kafka + CRDT + Git hosting enables versioned and history-preserving Event Model repository for oNote
- 28. Thank you!