Automating Processes in Modern Architectures
0 likes259 views
1) Automating long-running processes in modern architectures can be achieved through using a workflow engine. Workflow engines provide capabilities like persistence of state, retries, and error handling. 2) While event-driven and reactive architectures are popular, relying solely on event choreography can result in lost visibility of larger processes and future problems. 3) Orchestration of services and tasks using a workflow engine embedded within a service provides process automation while avoiding tight coupling between services. This allows decentralized orchestration while maintaining state.
Automating Processes in Modern Architectures
- 1. Automating Processes in Modern Architectures @berndruecker
- 3. Change is the only constant „ We don‘t know what we will need tomorrow. But we do know that we will need something. We have to be able to move quickly! CIO of a German insurance company in 2019
- 4. And you?
- 5. Credit Card Handling Payment REST Order Fulfillment REST DB PaymentId Amount Status 1 100 $ Collected 2 125 $ Open Reporting of overdue payments {script} Fixing open payments Scheduler to process open payments Damn it – we need a date! …to be continued…
- 6. 8 Using a workflow engine instead Workflow Engine Scheduler Durable State Glue Code Credit Card Service REST Workflow Definition Scope of your solution, application or microservice Workflow Engine: Is stateful Can wait Can retry Can escalate Can compensate Provides visibility
- 8. [email protected] @berndruecker http://berndruecker.io/ Bernd Ruecker Co-founder and Chief Technologist of Camunda
- 9. Camunda • Source-Available & Developer Friendly • Supports BPMN • Mature tooling, widely adopted • On-prem or cloud • http://camunda.com/
- 10. Live hacking
- 15. The era of APIs API API API API API API API API @berndruecker
- 16. The era of APIs API API API API API API API External Services Standard Software API Functions @berndruecker
- 17. Connecting the dots… API API API API API API API Microservices External Services Standard Software API Functions @berndruecker
- 18. Going hipster! https://www.reactivemanifesto.org/
- 21. But there is also trouble in the event-driven land! Event chains that implement processes Implementing long running behavior
- 24. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 25. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 26. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
- 27. Phil Calcado at QCon NYC 2019
- 28. Notification Checkout Payment Inventory Shipment Pinball Machine Architecture „What the hell just happened?“ @berndruecker
- 29. Peer-to-peer event chains Checkout Payment Inventory Shipment Order placed Payment received Goods shipped Goods fetched Fetch the goods before the payment @berndruecker
- 30. Peer-to-peer event chains Checkout Payment Inventory Shipment Fetch the goods before the payment Goods fetched Order placed Payment received Goods shipped @berndruecker
- 31. What we wanted Photo by Lijian Zhang, under Creative Commons SA 2.0 License and Wikimedia Commons / CC BY-SA 4.0 @berndruecker Choreography
- 32. Why is it so tempting? Service A Event Bus A B Service B C Service C D E F Service D G Service … Service … Service … Service … Service … Service … Service … @berndruecker
- 33. Why is it so tempting? Service A Event Bus A B Service B C Service C D E F Service D G Service … Service … Service … Service … Service … Service … Service … Adding is easy! You can „buy“ a shorter initial time-to-value by choreography. It yields in technical debt. @berndruecker
- 35. Order Decide about responsibility Checkout Payment Inventory Shipment Payment received Order placed Retrieve payment @berndruecker This is choreography This is orchestration
- 36. Order Decide about responsibility Checkout Payment Inventory Shipment Order placed Retrieve payment It can still be messaging! @berndruecker
- 37. Order Checkout Payment Inventory Shipment Stateful orchestration This orchestration requires state @berndruecker
- 39. Glue code (e.g. Java) https://github.com/berndruecker/flowing- retail/blob/master/kafka/java/order- zeebe/src/main/java/io/flowing/retail/kafka/or der/flow/FetchGoodsAdapter.java
- 40. Now it is easy to change the process flow @berndruecker
- 42. Order Checkout Payment Inventory Shipment @berndruecker The workflow is domain logic as part of the service @berndruecker
- 43. Summary • Use a workflow engine to automate long running processes • Visibility is vital for business agility • Event-driven choreography is not a magic cure • Orchestration • Helps to avoid chaos • Can be handled within one service (decentral) • Does not mean synchronous communication
- 44. 62 Recap: Workflow engines orchestrate anything Straight through processing Human Task Management Orchestrate … Microservices RPA bots Functions, Code, … Decisions (e.g. DMN Tables)
- 45. 63 Recap: The challenge are long running tasks = waiting
- 46. 64 Recap: A workflow engine persists state 23 79 51
- 47. 65 Recap: A workflow engine can be operated decentralized
- 48. 66 Many use cases (real-life examples) Industry Pilot Lighthouse Broadscale Sample Clients Cross-Industry Approvals Employee Onboarding Credit Check Microservices Orchestration Centralized Workflow Platform High Throughput E2E Process Automation Legacy BPM Replacement Cloud Native Applications IT Service Orchestration Banking & Finance Asset Management Customer Onboarding Loan Origination & Decisioning ATM Provisioning Enterprise Platform Development Trading Risk Management / Fraud Detection Insurance Audit Customer Onboarding Claim Service & Settlement Policy Underwriting & Contracts Integrated KYC Core System Automation Risk Management / Fraud Detection Telecommunication Online Systems Integration New Product Delivery Cell Tower Provisioning Order Management Contracting, Upgrading, Termination OSS & BSS Open Network Automation Platform (ONAP) Media & Entertainment Website Content Delivery System Process Fallout Subscription Management Licensing Content Distribution Digital Supply Chain New Product Development Fraud Management Manufacturing & High Tech Application prototype Order Execution Payments Servicing Research & Development Embedded Workflow Automation Customer Service Supply Chain