JOSA TechTalks - Data Oriented Architecture
Download as PPTX, PDF0 likes132 views
The document discusses data-oriented architecture (DOA) as an alternative to traditional monolithic and service-oriented architectures (SOA). It highlights DOA's characteristics, such as its reliance on a monolithic data access layer, stateless components, and minimized inter-component interaction, which favor schema considerations over API definitions. The document also contrasts the advantages and challenges of DOA, emphasizing its suitability in specific complex integration scenarios while acknowledging the difficulties in schema design.
JOSA TechTalks - Data Oriented Architecture
- 1. Data-Oriented Architecture Eyas Sharaiha <[email protected]> Jun 23, 2021 by Paul Downey via Flickr
- 2. Agenda Setting Context Common Approaches Monolithic Architecture Service-Oriented Architecture (SOA) Drawbacks of SOA Data Oriented Architecture (DOA) Definition Theory Trade-offs
- 4. System Architecture vs Software Design For the purposes of this talk: • System Architecture: Design & principles of • … entire system of programs • … usually in the backend • Software Design: Design & principles of • … single programs
- 5. Our Running Example: Trading System • You are a large financial institution that offers your clients to place orders to trade Consolidated Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm
- 7. Monolithic Architecture • In many ways, the “default” architecture • ~1 back-end program does everything • … fetches data from 1 or more database
- 8. Monolithic Architecture • Monolithic Architecture is not always a mess! • The monolithic program can • … be well designed • … impose separation of concerns • … be modularized/componentized • But, there’s no forced API boundary • … except between: • … client & server, or • … server & database
- 9. API Boundaries in Monolithic Architecture HTTP, RPC, etc. SQL, RPC, etc.
- 10. Running Example: Monolithic Architecture • You are a large financial institution that offers your clients to place orders to trade A Single Program Consolidated Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm
- 13. Service-Oriented Architecture • Organize backend into “Services” • Services are • Loosely coupled • Have a clear API (usually RPC/HTTP interface)
- 14. Microservices Architecture • A type of Service-Oriented Architecture • Everyone defines it differently • For me, services are • … smaller than SOA • … more isolated than SOA • When designed well: • separates concerns at the Service level
- 15. API Boundaries RPC/HTTP SQL/GraphQL/RPC/etc. Each service in this diagram must be “addressable”; individual services must have a URI (or some other path) to the service it’s calling.
- 16. Summary: Properties of SOA • Each Service defines its own API • usually exposed as an RPC Interface, or HTTP API • Every (callable) Service is Addressable
- 17. Summary: Advantages of SOA • Each Service can be developed & understood independently • Since the system is loosely coupled, new services are free to depend on existing services • Easier to Mock with clear API boundary • Easier to Debug by manually calling & inspecting individual methods
- 18. Running Example: Microservices • You are a large financial institution that offers your clients to place orders to trade Consolidated Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Bank A Connector Bank B Connector Bank C Connector Customer Connector Pricing Server Order Book
- 20. When Service-Oriented Architecture Fails to Scale
- 21. Problems from Addressability • Every Service needs the address of each Service it calls • Splitting a Service requires modification to all N callers • Some “Core” services might be depended on by many others • Creating an Isolated Sandbox is hard • Creating separate environments is hard • Reasoning about data consistency is hard
- 22. Problems of Scale • 𝑂 𝑁2 growth in integration complexity as the number of components (𝑁) grows • Complex topology of dependency graph, hard to reason about a priori
- 23. Problems of Scale: Illustrated • You are a large financial institution that offers your clients to place orders to trade Consolidated Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Bank A Connector Bank B Connector Bank C Connector Customer Connector Pricing Server Order Book Quote Server Bank D Connecto r
- 25. Data-Oriented Architecture • Like SOA: organized in small, loosely-coupled components • Unlike SOA: 1. Organized around a monolithic Data Access Layer 2. Components are Always Stateless 3. Component-to- Component interaction is minimized
- 26. Monolithic Data Access Layer • In Data Oriented Architecture, effort thinking about APIs becomes effort thinking about Schema • All state is data • All data is inspectable and queryable
- 27. Components are Stateless • In some SOA designs, each Service can have its own DB • **or set of tables only it looks at • In DOA, state is data & data is global
- 28. Minimized Inter-Component Interaction • Rather than RPCs, model state, requests, and results as data • Communication largely can be moved into the data space
- 29. Orders Trade Orders Orders Prices Running Example: Data Monolith Consolidated Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Prices Place Order Trade Confirm Bank A Connector Bank B Connector Bank C Connector Customer Connector Pricing Server Order Book Data Access Layer (Monolith) Prices Table Prices Prices Prices Prices Orders Orders Orders Trade Trade Trade Trade Source Security Bid Price Ask Price A EUR/US D 1.1861 1.1865 A USD/CH F 0.9226 0.9229 B EUR/US D 1.1862 1.1864 C USD/JPY 110.21 110.26 Prices Prices Target Customer Security Price Status Parent Order ac1de88a435f EUR/US D 1.1865 OPEN NULL Order dac8e87920f3 EUR/US D 1.1865 FILLED NULL Order f2ada1bec5c0 EUR/US D 1.1864 OPEN NULL A EUR/US D 1.1864 OPEN <acd7e8> Orders Trade
- 30. The DOA Trade off Easier Business Logic, Harder Schema Design
- 31. DOA Scale, Illustrated Bank D Connecto r Prices Orders Trade Quote Server Target Customer Security Typ e Size Status Quoter ac1de88a435f EUR/US D Buy 1MM OPEN Quoter dac8e87920f3 EUR/US D Buy 1MM ACCEPT Quoter f2ada1bec5c0 EUR/US D Sell 5MM REJECT Quote Request Source Request Price Quoter <bfe388> 1.1865 Quoter <e9f873> 1.1864 Quoter <00f9e9> 1.1864 Quote Quote Request Quote Quote Request Quote Prices
- 33. DOA, In Practice
- 34. Knowledge Graphs By Fuzheado via Wikipedia (ShareAlike)
- 36. Is Data-Oriented Architecture just SOA? • Sure, but with constraints. • If you have a SOA system with • Schema as the primary API, • Event/Service bus as main method of communication, • (Almost) no segregated pockets of data … then you have a Data-Oriented Architecture system
- 37. Didn’t you just reinvent ___?
- 38. DOA, In Theory
- 39. DOA: Theoretical Intuition Building • Where possible, model business logic as a series of pure data transformations: Quote = 𝑓 Quote Request, Price1, Price2, Price3, … Even further: • Can your business logic entirely be modeled as a series of Map, FlatMap, Filter, Reduce, & other Monadic operations? • In each case, these pure transformations (e.g. 𝑓 above) are a Component.
- 40. DOA: Theoretical Intuition Building, contd. • A DOA Component, being free of state, is a function; transforming • a bunch of inputs (that it queries or subscribes to), into • output objects (that it creates/INSERTs) • with no other side-effects • Sometimes, a complex operation can be modeled as more than one component • thus making intermediate values visible & inspectable
- 41. Minimizing RPCs Between Components Instead of RPCs between components, DOA encourages: • Data Request & Response: Can an RPC Request & Response be meaningfully modeled as data in its own right? • e.g. Quote Request is a request, and Quote is a response • Data Events: If one component triggers another when something (X) happens, can the component instead meaningfully create an object describing X? If these approaches are not meaningful, use directly addressable RPCs.
- 42. Data-Oriented Architecture is only sometimes the right architecture by Jernej Furman via Flickr
- 43. Not for the faint of heart… • Getting the schema right is hard & time consuming • DOA is not always the right architecture
- 44. When is DOA right? • Complexity of Integration > Complexity of Schema Design • Reasoning about Data Isolation
- 45. /fin
- 46. Follow me around the Web https://eyas.sh/ https://blog.eyas.sh/ [email protected]