Refactoring Into Microservices. Chicago Coders Conference 2017-06-26
0 likes289 views
Derek C. Ashmore discusses the challenges of refactoring monolithic applications into microservices, highlighting symptoms of monoliths such as long QA cycles and difficulties in maintenance. He advocates for iterative refactoring while maintaining the same technical stack, emphasizing that full rewrites can pose significant business risks and delays. Strategies presented include breaking down applications by functionality, addressing data store issues, and managing dependencies between services to facilitate a smoother transition to microservices.
Refactoring Into Microservices. Chicago Coders Conference 2017-06-26
- 1. Refactoring Into Microservices Breaking the Monolith Given by Derek C. Ashmore June 26, 2017 ©2017 Derek C. Ashmore, All Rights Reserved 1
- 2. Who am I? • Professional Geek since 1987 • Java/J2EE/Java EE since 1999 • AWS since 2010 • Specialties • Refactoring • Performance Tuning • Yes – I still code! ©2017 Derek C. Ashmore, All Rights Reserved 2
- 3. Discussion Resources • This slide deck – http://www.slideshare.net/derekashmore • This resource page – http://breakthemonolith.guru/resources.html • Slide deck has hyper-links! – Don’t bother writing down URLs ©2017 Derek C. Ashmore, All Rights Reserved 3
- 4. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2017 Derek C. Ashmore, All Rights Reserved 4
- 5. What is a Monolith? • Monolith – Single deployable application that’s gotten too large and complex to maintain • Symptoms – Hard to change • QA test cycles are long • Change causes unintended consequences – Hard to onboard new developers – Married to your technical stack – Harder to diagnose bottlenecks and memory issues ©2017 Derek C. Ashmore, All Rights Reserved 5
- 6. What’s the cure? • Smaller services rather than one very large application • Refactoring easier said than done – Hard to do safely – It’s a hard design problem • Existing team members often too close to the problem – Developers can’t resist common business binaries • Reduces code duplication Increases coupling • Coupling is a greater evil than code duplication! • Why not a complete rewrite? – No benefit until the end – Nobody knows *all* the business requirements! – Risk of negative impact to the business during the transition is high • Microservices are the other side of the spectrum ©2017 Derek C. Ashmore, All Rights Reserved 6
- 7. Why Break a Monolith? • Increases Speed to Market – Monolith is slow to change • Long QA cycles – Quicker response to business competition • New features added quickly • Makes continuous delivery possible – Microservices + Continuous Delivery • Lowers Business Risk – Reduces the defect rate – Reduces Downtime – Fixes are faster too 7 • Amazon deploys once a second • Netflix deploys over 5,500 times a day Source: newrelic.com
- 8. What are Microservices? • No concrete definition • Common microservice traits – Single functional purpose • Most/all changes only impact one service • Not dependent on execution context – “loosely coupled” – Independent process/jvm – Stateless – Standard Interface (typically Web Service/REST) – Analogy: Stereo system, Linux utilities ©2017 Derek C. Ashmore, All Rights Reserved 8
- 9. Microservices Application Architecture • Separate Databases • Eventual Consistency • More network activity ©2017 Derek C. Ashmore, All Rights Reserved 9
- 10. Microservice Reuse ©2017 Derek C. Ashmore, All Rights Reserved 10
- 11. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2017 Derek C. Ashmore, All Rights Reserved 11
- 12. Microservices / Monolith Mixture • 80 / 20 Rule Applies • Smaller, less complex, services are easier to support • Few companies are 100% Microservices • Traditional architectures do have a place • They are easier to create and manage while they are small • Some never grow large enough to refactor ©2017 Derek C. Ashmore, All Rights Reserved 12
- 13. Do NOT attempt a complete rewrite • It’s too big and complex – Usually no documentation of all implemented business rules • Not written down • Nobody has it in their head – Will negatively impact your business • The new won’t ‘identically’ match behavior – Business won’t benefit until the very end • It’s a large project Compelte rewrite will take a long time to realize – Often has no safeguards beyond manual testing – It’s a *very* big problem to handle all at once. • Refactor iteratively – Yes; this will take a longer time – It’s safer – It gives staff a chance to learn how to manage microservice libraries before they have too many to manage manually – It gives staff a chance to address non-functionals while not also being burdened with a complete redesign/rewrite ©2017 Derek C. Ashmore, All Rights Reserved 13
- 14. Stick with the same technical stack for the refactoring • Corollary: Do not rewrite and refactor at the same time. –Rewrite after automated contract testing is in place • New technical stacks have learning curves –Refactoring is hard enough ©2017 Derek C. Ashmore, All Rights Reserved 14
- 15. Preperation Steps • Establish a base automated test harness for the application –At an integration level, not unit test level –We want a chance of catching defects refactoring introduces –This will *not* be 100% comprehensive • The monolith is too complex for that • 80 / 20 rule applies • Establish Continuous Integration if it doesn’t exist –Jenkins CI Builds and Tests –Run Frequently (at least nightly) –Fix it immediately when somebody breaks the build ©2017 Derek C. Ashmore, All Rights Reserved 15
- 16. Types of Monoliths • Monoliths are not created equal • Types of Monoliths – Feature-Bloated Web Application – Data Store Strangle – Over-engineered Tarball – Distributed Monolith • Tactics differ per Monolith type ©2017 Derek C. Ashmore, All Rights Reserved 16
- 17. Feature-Bloated App • Feature set to large to manage – Usually a web app, but could be “Thick” or “Batch” as well • Symptoms – Unintended consequence problem – Long regression test cycles • Causes – New Feature Additions • Add complexity • Often not envisioned in original design – Tacked on – Developer assumptions • Streamlines code initially • Are not documented • Create landmines for new feature additions • Removing unused features – Never happens – Benefit not clear to the business – No stats on used vs unused ©2017 Derek C. Ashmore, All Rights Reserved 17
- 18. Tactics: Feature-Bloated Web App • Chop off it’s head! – Separate the User Interface • Separation of concerns • Leaves consolidated UI • Forces Monolith to have REST interface – Easier to test more completely • Makes further refactoring safer • Forces you to identify REST resources within the monolith ©2017 Derek C. Ashmore, All Rights Reserved 18
- 19. Separated UI • Monolith now has a REST resource interface – Easier for automated testing • Safety in further refactoring – Functionality categorized into logical units • Easier to identify sections to separate out • Cosmetic changes occur on their own schedule ©2017 Derek C. Ashmore, All Rights Reserved 19
- 20. Factor out additional services • Factor out by related REST resources – Separate underlying database too • Prioritize resources by – Code Change Velocity – Code Complexity – Enhancement needs ©2017 Derek C. Ashmore, All Rights Reserved 20
- 21. Data Store Strangle • Data Storage is the Monolith’s Achilles Heel • Symptoms – No flexibility in data access • Larger memory footprint – Lots of replicated data – Complex data access code – Storage design hardest part of adding new features • Causes – Bad Data Storage Design • Usually process oriented – Replicated Data – Coupled with business process • Causes code complexity – Amplifies technical debt • Storage refactoring difficult – Never happens – Benefits not clear to the business – Carries more risk than refactoring code ©2017 Derek C. Ashmore, All Rights Reserved 21
- 22. Tactics: Data Store Strangle • Tourniquet Strategy – Separate storage reads/writes into separate service • Separate into service • All access uses central service • Spin off a read-only copy asynchronously in legacy format for reporting if needed – Wrap automated testing around the tourniquet service – Refactor the underlying database incrementally • Refactoring databases is a difficult topic – Book: Refactoring Databases: Evolutionary Database Design • Introduce new API to simplify access – Reduces caller complexity – More targeted and flexible access – Usually possible only after refactoring • Deprecate Tourniquet API ©2017 Derek C. Ashmore, All Rights Reserved 22
- 23. Over-engineered Tarball • Internal component obstacle – Component with unnecessary complexity • Symptoms – Nobody understands it – Everyone fears changing it – Common impediment for new feature enhancements • Causes – Bored, talented developers • Good intentioned • Manufacturing interesting puzzles • Real problem when the developer leaves – Nobody else can change it safely – Creates complexity in code surrounding it ©2017 Derek C. Ashmore, All Rights Reserved 23
- 24. Tactics: Over-engineered Tarball • Tourniquet Strategy – Separate tarball into separate service • Separate into service • All access uses central service • Also separate out data storage – Wrap automated testing around the tourniquet service – Refactor the tarball incrementally or completely replace • Automated tests are safety net ©2017 Derek C. Ashmore, All Rights Reserved 24
- 25. Distributed Monolith • Related services that are tightly coupled / bound together • Symptoms – Always deploy a group of services together • Independent deployments rarely happen – New features require changes to the same service group • Causes – High coupling between services • Short-term developer convenience • Inappropriate Delegation – Micro-management for indirect service calls – Pass-through arguments to one service to indirectly control behavior of a subsequent service • Shared Business Code Binaries – Changes in binaries force redeployments • Higher management overhead without benefits from smaller independent services ©2017 Derek C. Ashmore, All Rights Reserved 25
- 26. Tactics: Distributed Monolith • Solving shared business code problem – Factor into separate service(s) • Deployed once and commonly called • Don’t deploy as library – Reduces forced deployments – Performance mitigations • Expiring Cache • Make asynchronous – Coupling worse evil than Code Duplication ©2017 Derek C. Ashmore, All Rights Reserved 26
- 27. Tactics: Distributed Monolith (con’t) • Solving the Inappropriate Delegation problem – Eliminate the middle- man ©2017 Derek C. Ashmore, All Rights Reserved 27
- 28. Agenda Microservices and Monoliths Tactics to Refactor Common Problems Summary / Q&A ©2017 Derek C. Ashmore, All Rights Reserved 28
- 29. Common code between services? • Yes, but…. – Version it • Services make decision as to when to upgrade – Changes to common code can’t require the deployment of multiple services ©2017 Derek C. Ashmore, All Rights Reserved 29
- 30. Real-World Problems • Corporate-Level Reporting –Reporting for data across the enterprise –Many read underlying databases for applications that house the data –Refactoring will impact this type of reporting • Database Refactoring –Much more expensive –Can’t simply rollback –Book Recommendation • Database Factoring: Evolutionary Database Design ©2017 Derek C. Ashmore, All Rights Reserved 30
- 31. When is your refactoring complete? • You can stop at any point –If what remains of your monolith is supportable, you can stop –80 / 20 rule applies • That’s why services are prioritized by volatility • You can resume at any point –Guided by code volatility and defect rates –Change produces refactoring needs ©2017 Derek C. Ashmore, All Rights Reserved 31
- 32. Further Reading • This slide deck – http://www.slideshare.net/derekashmore • This resource page – http://breakthemonolith.guru/resources.html ©2017 Derek C. Ashmore, All Rights Reserved 32
- 33. Questions? • Derek Ashmore: – Blog: www.derekashmore.com – LinkedIn: www.linkedin.com/in/derekashmore • Connect Invites from attendees welcome – Twitter: https://twitter.com/Derek_Ashmore – GitHub: https://github.com/Derek-Ashmore – Book: http://dvtpress.com/ ©2017 Derek C. Ashmore, All Rights Reserved 33