![Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Catalog Service
Product
SDL
* SDL is
document based.
no direct SQL
sdl.insert(request.getProduct.mapTo[ProductDomain])](https://image.slidesharecdn.com/java-240529143613-b56ff532/85/Beyond-Event-Sourcing-Embracing-CRUD-for-Wix-Platform-Java-IL-47-320.jpg)
![Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Product
SDL
Catalog Service
Product
SDL
sdlBuilder =
_.withDomainEventsEnabled(_.mapTo[Product])](https://image.slidesharecdn.com/java-240529143613-b56ff532/85/Beyond-Event-Sourcing-Embracing-CRUD-for-Wix-Platform-Java-IL-48-320.jpg)
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
- 1. Beyond Event Sourcing May 2024 Embracing CRUD for Wix's High-Growth Platform
- 2. Beyond Event Sourcing Hi, I’m Natan Backend Infra Tech Lead @Wix Yoga enthusiast Speaker Blogger natansilnitsky www.natansil.com @NSilnitsky
- 3. Beyond Event Sourcing @NSilnitsky
- 4. Beyond Event Sourcing @NSilnitsky ~4B Daily HTTP Transactions 4000± Microservices in production ~70B Kafka messages a day
- 5. Beyond Event Sourcing @NSilnitsky Agenda Event sourcing & CQRS Framework for CRUD services Dev velocity 1 Scalability 2 Performance 3 Resilience 4 * stores
- 6. Beyond Event Sourcing @NSilnitsky Event Sourcing & CQRS Wix Stores Example
- 7. Beyond Event Sourcing @NSilnitsky Add/Update product QueryProduct Product Catalog
- 8. Beyond Event Sourcing @NSilnitsky WriteProduct Event sourcing and CQRS Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 ReadProduct Name Price Description Stock-level Append-Only Events Snapshot
- 9. Beyond Event Sourcing @NSilnitsky Event Store CreateProduct: Catalog Write API Event sourcing and CQRS Product Created
- 10. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 Event Store CreateProduct: Catalog Write API Product Created
- 11. Beyond Event Sourcing @NSilnitsky Replay GetProduct Catalog Read API Event sourcing and CQRS Replay events Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 Event Store CreateProduct: Catalog Write API Product Created
- 12. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS – Advantages Debug with “time travel” GetProduct Catalog Read API Replay events Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 Event Store CreateProduct: Catalog Write API Product Created
- 13. Beyond Event Sourcing @NSilnitsky Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 ….. 5 Product Changed (stock-level) 30 Event Store Event sourcing and CQRS
- 14. Beyond Event Sourcing @NSilnitsky Product 123: Product Created 1 Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) 4 ….. 5 Product Changed (stock-level) 30 Event Store Product 123 Snapshot Snapshot Repository Event sourcing and CQRS
- 15. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Advantages Catalog Read API Inventory snapshot Product Snapshot Product Created Product Changed (price) 2 Product Changed (description) 3 Product Changed (stock-level) Product 123: 1 4
- 16. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Product DB Create/Read Product Catalog CRUD Event Store CreateProduct: Catalog Write API Product Created Product 123 Snapshot Snapshot Repository Catalog Read API
- 17. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Product Created (price 4$) 1 Product Changed (price 6$) 2 Product 123: Snapshot Repository Delayed Product snapshot Consumer Catalog Write API
- 18. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Product 123: Snapshot Repository Product Created (price 4$) 1 Product snapshot Consumer Catalog Write API Product Changed 2 Product Created (price 4$) 1 Product Changed (price 6$) 2 Delayed
- 19. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Product Changed (descriptionA) 3 Product Changed (descriptionB) 4 Product 123: Snapshot Repository Product Created (price 4$) 1 Product Changed (descriptionB) 2 Catalog Write API Product Changed (descriptionA) 3 Product Created (price 4$) 1 Product Changed (price 6$) 2 Delayed Product snapshot Consumer
- 20. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Product Changed (descriptionA) 3 Product Changed (descriptionB) 4 Product 123: Snapshot Repository Product Created (price 4$) 1 Product 123 Snapshot Product Changed (descriptionB) 2 Product 4$ DescriptionA Product Changed (descriptionA) 3 Catalog Write API Product Created (price 4$) 1 Product Changed (price 6$) 2 Delayed Product snapshot Consumer
- 21. Beyond Event Sourcing @NSilnitsky Read your own writes Event sourcing and CQRS - Disadvantages
- 22. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Confetti Cannon events: Product Changed (price -1$) Read your own writes Catalog Write API Event Store
- 23. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Confetti Cannon events: Snapshot Repository Confetti Cannon Snapshot Physical Item 3$ DescriptionA Product Changed (price -1$) Read your own writes Catalog Write API Event Store
- 24. Beyond Event Sourcing @NSilnitsky Event sourcing and CQRS - Disadvantages Confetti Cannon events: Snapshot Repository Confetti Cannon Snapshot Physical Item 3$ DescriptionA Product Changed (price -1$) Read your own writes Catalog Write API Event Store Product Changed (price -1$) +
- 25. Beyond Event Sourcing @NSilnitsky Product DB Update Product Catalog CRUD Event sourcing and CQRS - Disadvantages Read your own writes
- 26. Beyond Event Sourcing @NSilnitsky Read your own writes Event sourcing and CQRS - Disadvantages
- 27. Beyond Event Sourcing Event sourcing and CQRS - Disadvantages Complexity Eventual consistency only Massive scale Corrupted snapshot Read your own writes
- 28. Beyond Event Sourcing @NSilnitsky APIs + Standardization
- 29. Beyond Event Sourcing @NSilnitsky RPC Product Catalog API Cart Service Internal Wix Developer Open Platform - API First * Center 4000, clear, focused, comprehensive
- 30. Beyond Event Sourcing @NSilnitsky HTTP/SDK Product Catalog API Internal Wix Developer PoS App External App Developer Open Platform - API First
- 31. Beyond Event Sourcing @NSilnitsky JavaScript method Product Catalog API Internal Wix Developer Custom Filter External App Developer External Wix Site Developer (Velo) Open Platform - API First
- 32. Beyond Event Sourcing @NSilnitsky Product Catalog API Internal Wix Developer External App Developer External Wix Site Developer (Velo) Open Platform - API First * CRUD on ES, consistent, hard in standard way. Users fast experience
- 33. Beyond Event Sourcing @NSilnitsky Stores Bookings Events Forms Loyalty Rewards Tickets Policies Checkout Time Slots Schemas Sub missions Guests Coupons Calendar Orders Waitlist Cart Catalog Programs Open Platform - API First Internal Wix Developer External App Developer External Wix Site Developer (Velo)
- 34. Beyond Event Sourcing @NSilnitsky Wix’s Open Platform CRUD Event sourcing Was Independent “startups” Now Single Open Platform
- 35. Beyond Event Sourcing @NSilnitsky Wix’s Open Platform CRUD Event sourcing API First APIs - TDD + FE driven Was Independent “startups” Now Single Open Platform
- 36. Beyond Event Sourcing Framework for CRUD services Open Platform API Schema Validations & Permissions Scaffold Persistence + Event publishing Data projections (Queries)
- 37. Beyond Event Sourcing @NSilnitsky CreateProduct Product Document Store Catalog API Product Catalog - CRUD ReadProduct UpdateProduct DeleteProduct Unified!
- 38. Beyond Event Sourcing @NSilnitsky CreateProduct ReadProduct UpdateProduct DeleteProduct Catalog API API First - platformized CRUD
- 39. Beyond Event Sourcing @NSilnitsky CreateProduct ReadProduct UpdateProduct DeleteProduct Catalog API API First - platformized CRUD
- 40. Beyond Event Sourcing @NSilnitsky CreateProduct ReadProduct UpdateProduct DeleteProduct Catalog API service ProductService { option (service_entity).message = "...v3.Product"; rpc CreateProduct (CreateProductRequest) returns (CreateProductResponse) ... rpc GetProduct (GetProductRequest) returns (GetProductResponse) ... rpc UpdateProduct (UpdateProduct) returns (UpdateProduct) ... … } message Product { option (entity) = {fqdn: "...v3.product"}; google.protobuf.StringValue id = 1; google.protobuf.Int64Value revision = 2; ... repeated Inventory inventory = 25; ... } API First - platformized CRUD
- 41. Beyond Event Sourcing @NSilnitsky API First - platformized CRUD CreateProduct ReadProduct UpdateProduct DeleteProduct Catalog API service ProductService { option (service_entity).message = "...v3.Product"; rpc CreateProduct (CreateProductRequest) returns (CreateProductResponse) ... rpc GetProduct (GetProductRequest) returns (GetProductResponse) ... rpc UpdateProduct (UpdateProduct) returns (UpdateProduct) ... … } message Product { option (entity) = {fqdn: "...v3.product"}; google.protobuf.StringValue id = 1; google.protobuf.Int64Value revision = 2; ... repeated Inventory inventory = 25; ... }
- 42. Beyond Event Sourcing @NSilnitsky API First - platformized CRUD
- 43. Beyond Event Sourcing @NSilnitsky API First - platformized CRUD
- 44. Beyond Event Sourcing @NSilnitsky Event Driven Architecture Automatic Domain Events FTW
- 45. Beyond Event Sourcing @NSilnitsky EDA - Domain Events CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Product Document Store Catalog Service * DE describes…
- 46. Beyond Event Sourcing @NSilnitsky EDA - Domain Events CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Catalog Service Product SDL * SDL is document based. no direct SQL
- 47. Beyond Event Sourcing @NSilnitsky EDA - Domain Events CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Catalog Service Product SDL * SDL is document based. no direct SQL sdl.insert(request.getProduct.mapTo[ProductDomain])
- 48. Beyond Event Sourcing @NSilnitsky EDA - Domain Events CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Product SDL Catalog Service Product SDL sdlBuilder = _.withDomainEventsEnabled(_.mapTo[Product])
- 49. Beyond Event Sourcing @NSilnitsky EDA - Domain Events Data warehouse/Lake CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Catalog Service Product SDL * debugging corruption
- 50. Beyond Event Sourcing @NSilnitsky EDA - Domain Events ebay-bridge Service CreateProduct UpdateProduct DeleteProduct Catalog Service Product Created Product Updated Product Deleted Product SDL
- 51. Beyond Event Sourcing @NSilnitsky EDA - Domain Events CreateProduct UpdateProduct DeleteProduct Product Created Product Updated Product Deleted Catalog Service Product SDL
- 52. Beyond Event Sourcing @NSilnitsky Data consistency in Wix’s EDA Resilient Producers and consumers
- 53. Beyond Event Sourcing @NSilnitsky Make DB Update & Event Producing Atomic Catalog Service Ebay Bridge Service * atomic, otherwise
- 54. Beyond Event Sourcing @NSilnitsky Produce event to S3 Resilient Producer Catch Unsent Events Catalog Service
- 55. Beyond Event Sourcing @NSilnitsky Produce to Kafka Poll Produce event to S3 Resilient Producer Fallback to S3 and Heal Catalog Service Healer Service
- 56. Beyond Event Sourcing @NSilnitsky Consumer retries + DLQ Make DB Update & Event Producing Atomic Catalog Service Ebay Bridge Service
- 57. Beyond Event Sourcing @NSilnitsky Alternative - use outbox pattern and/or CDC Transaction Outbox Table Insert Product Table Insert Update Delete Database Instant read-your-own-writes consistency in Catalog service Write to database CDC Read from Outbox Table Kafka Connect Debezium connector Publishes messages to brokers Kafka Broker Eventually consistent data exchange with Ebay Bridge Service Catalog Service Ebay Bridge Service
- 58. Beyond Event Sourcing @NSilnitsky Data Projection & query optimization Materializer
- 59. Beyond Event Sourcing @NSilnitsky Query latency - naive CreateProduct ReadProduct DeleteProduct Catalog Service Product SDL FilterProductWithInventory Inventory Service Inventory SDL RPC 1 2
- 60. Beyond Event Sourcing @NSilnitsky Multi-step Query latency - naive CreateProduct ReadProduct DeleteProduct Catalog Service Product SDL FilterProductWithInventory Inventory Service Inventory SDL RPC price < 100 and stock > 4 2 1
- 61. Beyond Event Sourcing @NSilnitsky Query latency - DB join CreateProduct ReadProduct DeleteProduct Catalog Service Product SDL FilterProductWithInventory Inventory Service Inventory SDL DB level Join
- 62. Beyond Event Sourcing @NSilnitsky Query Latency - Materializer Product + Inventory Materializer Inventory Inventory updated Event Catalog Service Inventory Service FilterProductWithInventory
- 63. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 64. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 65. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 66. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 67. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer * no consistency
- 68. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 69. Beyond Event Sourcing @NSilnitsky Simplicity Onboarding new team member Write performance Read performance consistency Audit log/time machine Projections/queries Comparing Event sourcing to Wix’s CRUD based solution Event Sourcing CRUD SDL+Domain Events Materializer
- 70. Beyond Event Sourcing @NSilnitsky Summary Wix successfully shifted its vast distributed system entirely to CRUD-based microservices, moving away from a CRUD/event sourcing hybrid. . . .
- 71. Beyond Event Sourcing @NSilnitsky Summary Wix successfully shifted its vast distributed system entirely to CRUD-based microservices, moving away from a CRUD/event sourcing hybrid. This transformation was driven by a commitment to standardization, managed infrastructure with automated code generation, and a decoupled architecture.
- 72. Beyond Event Sourcing @NSilnitsky Summary Advanced tools were also implemented to boost development speed, ensure system resilience, and optimize for scale and performance. Domain Events Resilient Producer Materializer Simple Data Layer
- 73. Beyond Event Sourcing Q & A Thank you natansilnitsky www.natansil.com @NSilnitsky 👉 slideshare.net/NatanSilnitsky