How Orwell built a geo-distributed Bank-as-a-Service with microservices
2 likes•804 views
The presentation outlines Orwell Group Holding Limited's innovative banking technology, focusing on its geo-distributed banking-as-a-service model that utilizes microservices and kappa architecture for enhanced efficiency and flexibility. It highlights the company's strategies to overcome legacy challenges, manage risk, and ensure operational continuity while leveraging MariaDB for data consistency and change data capture. The document emphasizes the importance of a robust, scalable, and adaptable infrastructure to meet evolving user expectations and regulatory demands.
How Orwell built a geo-distributed Bank-as-a-Service with microservices
- 1. The material in this presentation has been prepared by Orwell Group Holding Limited as general background information only about Orwell’s activities current as at the date of this presentation. Information in this presentation, including forecast financial information, should not be considered as advice or a recommendation in relation to holding, purchasing or selling securities in Orwell Group Holding Limited or any subsidiaries. This presentation may contain forward looking statements including statements regarding our intent, belief or current expectations with respect to Orwell’s businesses and operations, market conditions, results of operation and financial condition, and risk management practices. Readers are cautioned not to place undue reliance on forward looking statements. Forecasts, estimates and examples are subject to uncertainty and contingencies outside of our control. Orwell Banking Platform Payment Processing Cash management Compliance Payment Network as as Service How did we build a geo-distributed Bank- as-a-Service with microservices Manoj Chugh Chief Architect Raul Rodrigo Barco Engineering Head, Madrid
- 2. About Orwell 01 02 03 04 05 Regulated Cash Management with Borderless Banking Model Breakthrough Payments Technology Open Banking and PSD2 compliant with New Payments Architecture White Label for Banks Suite of Treasury, Payments and Loyalty solutions for Corporates
- 3. Challenge
- 4. Challenges Complex estates that have been running since the late 60’s and 70’ How to manage change in these large estates to migrate to newer technology. Legacy Financial Services • User value experience and speed. • Users have different behaviour • Users expect availability of their data 24x7x365 Users • Secure and control data access and movement • Reduce risk (liquidity, credit, market, fraud, counterparty) Security, Risk & Controls • Anti-money laundering, Sanctions and Fraud • Reporting, strong security, safeguarding of money Regulators • Create niche products • Reduce the cost and Increase agility • Business continuity Product & Business Owners
- 5. How can we address the challenges and provide business agility with bleeding edge technology at best possible cost The journey is as important as the end state.Legacy
- 6. Solution
- 7. Key principles Append-only immutable log is the central layer for the distributed architecture Kappa Architecture Domain Driven, Single Responsibility with Ubiquitous Language and Bounded Context Domain Driven Microservices Command Query Responsibility Segregation & Event Sourcing Storing events rather than state. CQRS allows highly performant architecture. Failure is expected. Every component must be continuously available. Continuous Availability 01 02 03 04 Orwell’s adoption of technology Principles
- 8. Kappa Architecture Style • Kappa Architecture is a software architecture pattern. Rather than using a relational DB like SQL or a key-value store like Cassandra, the canonical data store in a Kappa Architecture system is an Apache Kafka append-only immutable log. • From the log, data is streamed through a computational system and fed into auxiliary stores for serving.
- 9. Microservices Design Domain Based Microservices Awareness of Microservices’ Trade- off Must use some of the microservices principles that allows isolation but must avoid the possible silos that could be created. Awareness of Legacy function The system must support and maintain the same level of functionality of exiting institutions or legacy systems will never be turn off. Awareness of Legacy Data Old systems usually means bad data quality of missing data. Any new system must support this fact.
- 10. Data Infrastructure Design Our Data Infrastructure is based upon separation of concerns Write efficient (Consistency) Read efficient (Availability) Financial Institutions on average will have above a 90% reads to 10% writes. Payments are write operations but not updates. Customer don’t change their details frequently, no one moves address multiple time time a day. Read/write performance is all about trade-offs and balance
- 11. Data Model Design This allows the definitions of products with customized characteristics that can be built and test in a matter of days or weeks. The model is designed to allow complex relationships between participants e.g. a company having users that manage their own self-service banking products Data Model is based upon Master Data Management & Party-Contract-Product Held principles
- 12. Geo-distributed multi-cloud Local Registry System Make Payment USA Instance ID Local Registry System UK Instance ID Account Key (64Bit) Instance ID Account Naming Structure to allow Routing Automated Discovery of Network participants Local Registry System Broadcast – 128bit Account Number Hash Germany Instance ID Make Payment DHT Based Network UUIDs: That are customized identifiers Like email The system is designed to have multiple instances across multiple geographies and multiple public clouds. Each system is instance aware.
- 13. How do we use MariaDB
- 14. Why MariaDB in our technology stack Accuracy Flexible information model Common understanding Write Performance We needed a flexible information model for customer and product information, that humans easily understand. Consistency and accuracy of financial information is highly important. The form of information to consume in microservice is dependent on the consumption model but we all need a common understanding. We don’t have a high performance requirement for this dataset as customers don’t change telephone number 50000 a second.
- 15. MariaDB Setup
- 16. MariaDB Change Data Capture Orwell’s unique solution
- 17. Orwell CDCClient implements MaxScale CDC Protocol - Connection andAuthentication • Client connects to MaxScale CDC protocol listener. • Send the authentication message which includes the user and the SHA1 of the password Registration • SendingUUID • Specify the output format (AVRO or JSON) for data retrieval. Data Request • SendCDC commands to retrieve router statistics or to query for data events CDC client features: • One client manage all changes events in different tables (configured in zookeeper) • Handle not existing avro-schema files • High availability (Leader-election pattern) MariaDB Change Data Capture
- 20. - Prometheus: Monitoring / Alerting - Grafana: Dashboard Node Cluster Definition: • 3 Instances • Monitoring Status instances • Monitoring Leader/Followers Leader/Followers • Monitoring Leader Elections CDC Resilience
- 21. - Prometheus: Monitoring - Grafana: Dashboard CDC Instance Nodes: • Tables • Events processed • OK • Expired • Duplicates CDC Metrics
- 22. Conclusion
- 23. Conclusion Orwell combined best of two worlds (MariaDB SQL & NoSQL) in microservices CQRS architecture to delivering a hyperscale & highly-available solution Our approach of using Domain based microservices allows a loosely coupled component architecture, all of which can have different scalability profiles Use of Master Data Management techniques & JSON has allowed flexibility and creation of Metadata to support dataset changes Flexible information model has reduced time to market for creating new products MariaDB has been a useful General purpose RDBMS not just for the data consistency requirements but also for use with Hadoop applications such as HIVE Metastore, Apache Ranger and Apache Ambari. With Orwell’s unique CDC solution, we can not only keep SQL & NoSQL in sync but also recover the failures with no down time to the service
- 24. The material in this presentation has been prepared by Orwell Group Holding Limited as general background information only about Orwell’s activities current as at the date of this presentation. Information in this presentation, including forecast financial information, should not be considered as advice or a recommendation in relation to holding, purchasing or selling securities in Orwell Group Holding Limited or any subsidiaries. This presentation may contain forward looking statements including statements regarding our intent, belief or current expectations with respect to Orwell’s businesses and operations, market conditions, results of operation and financial condition, and risk management practices. Readers are cautioned not to place undue reliance on forward looking statements. Forecasts, estimates and examples are subject to uncertainty and contingencies outside of our control. Thanks
Editor's Notes
- #3: About Orwell Orwell Group is a regulated payments services provider specializing on one layer of the banking services stack: the cash management function. This includes current accounts, liquidity-related bank accounts and all types of payment services and their related value-added service. The company offers its regulated service in white label, allowing banks to expand geographically using cutting-edge technology. We, at Orwell, look at providing solutions in the financial sector namely high efficiency banking systems that address todays challenges within finance. We provide series of services that focus on providing the best value for our customer and customers customers. This is not only a our vision but our own mission statement that dove the design of the most efficient banking system possible.
- #5: Orwell Group set out on the mission to build a new kind of banking system that would enable cross border payments with new functionalities and cash flow management for high retail banking customers and corporate banks. It needed a way of scaling to much bigger enterprises with multiple millions of clients. There are many challenges with the finance sector and these are due to many reason but complex estates that bank have been going for many years the cost of change management make it very hard for existing institution to face the digital challenges they are facing. Complexity in the IT estate is not the only factor adding business complexities like product management and speed to market makes it very expensive to move to a more lean world. The leap must to move these institutions must provide the right cost incentives as well as manage the complexities of moving to this lower cost model. The journey is as important as the end state.
- #8: Cloud Portability Self service Open Source products
- #10: Microservices are implemented using CQRS pattern with shared database model (as against one database per service) Saga pattern is used for rollback/undo Talk of anti-patterns Microservices are not a goal and these are no magic pixie dust that can be sprinkled
- #11: Orwell group selected MariaDB as the core foundation of the platform for a consistent database with a high write efficiency. We built microservices and event driven architecture around MariaDB Galera cluster. The data architecture is built around master data management and reference data management methodologies. This brought the capability of having a highly resilient geo distributed however still consistent architecture.
- #12: Why is Master Data Management useful? Talk about JSON as the flexible model. Information is everything, like water, flows and you want to store as much as possible. Models evolve - Don't think tables, Think characteristics and relationships. There is no one final answer but some concepts are very constant so we can employ some common sense.
- #13: Cost saving for instrument payments (scheme & network settlement services) Global network
- #15: We look at banking in a different way not as a number on a database but as a set of changes on accounts. A balance is just the state at a certain point in time that can vary faster or slower depending on operations. Not important for a process run faster but how parallel can we run operations there is where cost saving lies.
- #16: 3 node cluster across AZ Active Active (Automatic failover)
- #17: Introduce Raul Why Orwell built this unique solution (gaps in the existing implementations – authentication, transport security) CQRS
- #18: Start with MariaDB solution (example)
- #19: With MariaDB, no database limitations with the use of different back-end storages like RocksDB that we are exploring.
- #21: CDC resilience starts from the beginning and to duplicates must be avoided
- #24: Orwell Group has been able to build a secure distributed architecture with a high cost efficiency. despite doing the development of a cutting edge, modern banking infrastructure. Having a high accuracy and consistency in data has allowed high quality data pipelines from Transaction to Analytics and compliance with regulations (including GDPR).