AWS Community Day: From Monolith to Microservices - What Could Go Wrong?
- 1. MELBOURNE
- 2. From monolith to microservices What could go wrong? BY MAI AND ALICIA | 2019
- 3. About us Phuong Mai Nguyen Consultant Developer at ThoughtWorks Code Like a Girl Member @mai_isthebest Alicia Bong Consultant Developer at ThoughtWorks
- 4. Welcome to the microservices team retro! In today’s retro, we will be addressing the following: 1. What are microservices? 2. How did some of the migration from monolith to microservices go? 3. What went wrong during the migration? 4. How to improve upon them? 5. Retro action points
- 5. ● A BIG STURDY ROCK! Monolith
- 6. ● Sets of loosely coupled services which work together to create a system ● Enables "incremental tech refresh” ● Scalable cost savings ● Flexible: can have THE right tool for the job ● Reliable: faster and independent deployments ● Velocity: enable innovation quickly Microservices
- 7. Migrating from monolith to microservices “You cannot learn without making mistakes. Thankfully, there are others who have made mistakes and we can learn from them.” Cool, so microservices are awesome, let’s all go today and change our app to microservice, shall we? It’s not as easy as it sounds.
- 8. Background ● A global mail logistics operation presenting connections to over 200 countries ● It offers cross-border solutions for international mail shipments – whether a few grams or large parcels. Freipost
- 9. They had: ● Large codebase built with .NET ● Six different projects ● Strongly coupled ● Ownership of projects Why microservices for Freipost? They wanted: ● To lower the dependencies between each projects ● More testability ● Faster deployments
- 10. ● Single team was tasked with dealing with both BAU and extracting monolith to microservices. ● BAU is a never-ending rabbit hole ● Will you concentrate on BAU or try to finish extracting your microservices? What went wrong 1. BAU as a distractor
- 11. Separate responsibilities ● Have different teams and resources that don't overlap with BAU and developing new features. How to improve Don’t add new features while extracting ● Again, one thing at a time.
- 12. ● Would you take the hardest, complex part and convert it to microservice? ● Would you take the least impactful part and convert? ● Would you take the most important part and convert first? What went wrong 2. Trying to be cool and take out the most complicated first!
- 13. ● “Do something you believe you can do in 3 months.” (Evan Bottcher) ● By starting small, your team will start shaking off patterns and potential issues in the new architecture. Start defining a dev process, CI/CD and release cadence. ● Use Domain-Driven Design to help separate monolithic applications into multiple standalone service applications or develop them separately from the beginning with the help of bounded contexts. How to improve Start with a simple capability with fewer dependencies
- 14. Microservice abstractions at Freipost
- 15. Background ● Australia’s number one online classifieds network for cars, bikes, boats, industry machinery and equipment. Anyone here been to our website before? ● Around 200 developers, QA engineers, DevOps engineers in total. ● Three main platforms - Desktop, Mobi and App Carsales
- 16. Tradernet
- 17. ● Developers unhappy with the sheer size of code ● Release process took weeks to plan and test ● Deployments around 2am to get it on prod by 6am ● Rollbacks were similar too! ● Want a new feature? It may take some time (like a few months…) Why microservices for Carsales?
- 18. ● How do you know if your deployment was a success? ● How do you tell if something goes wrong among your few hundred microservices? ● How do you debug the nature of issues with microservices? What went wrong 3. Lack of observability
- 20. ● Setup monitoring for each microservice and give each team ownership of their apps ● Setup alerts and keep refining the metrics as the applications mature ● Log! Log! Log! Without logging, good luck fixing issues on cloud! How to improve Monitoring and logging before everything else
- 22. Microservice abstractions at Carsales
- 23. Background ● One of the biggest banks in Australia ● 30,000 employees ● 2500 developers ● Over 9 million customers around the world BankX
- 24. ● The monolithic system has a lot of system dependencies. Whenever there is a new release, the whole system will be tested. This requires a huge operational cost. ● Hard to onboard new people ● Need to speed up the delivery time of features and leverage cloud technologies. Why microservices for Bank X?
- 25. ● Chinese whispers equivalent of teams sharing what their idea of microservices are. ● Misconceptions ● Buzzwords or real use-case? What went wrong 4. Semantic diffusion causes distributed monoliths
- 26. ● Understand not just what microservices are, but is it really for you? ● Do you have the capacity for it? ● Can your teams and management be re-arranged around new technology and methodologies? ● Are you actually ready for it? How to improve Understand before implementing
- 27. Let's talk about the two most enticing parts of microservices architecture: ● Ability to follow agile methodology and release at fast pace ● Fail fast, fix fast mantra What went wrong 5. Not completely agile, not completely waterfall
- 28. Now let's address two of the most important requirements for banks: ● Security ● Stability What went wrong 5. Not completely agile, not completely waterfall
- 29. Aren’t those points conflicting with each other? ● You bet! What went wrong 5. Not completely agile, not completely waterfall
- 30. Internet Banking Department Cards Team Control & Governance Security TeamRelease Team Solution Design Team Cards Delivery & Support Team Integration & Performance Team Support TeamDevOps TeamCards Team Production Team Integration Test Squad Back-End SquadFront-End SquadDelivery Squad Performance Squad Release Schedule Squad
- 31. ● Conway's Law asserts that organizations are constrained to produce application designs which are copies of their communication structures. This often leads to unintended friction points. ● The Inverse Conway Maneuver recommends evolving your team and organizational structure to promote your desired architecture. How to improve Evolve your team to suit your architecture
- 32. Security Team Release Team Solution Design Team DevOps Team Integration Test Squad Back-End Squad Front-End Squad Delivery Squad Performance Squad Release Schedule Squad Front-End & Back-End Developers QAs (testing, integration, performance tests) Security controls & governance people Solution Architects Release to Prod people Product Team
- 33. Microservice abstractions at Bank X
- 34. 1. Separate responsibilities: have different teams and resources that don't overlap with BAU and developing new features 2. Do not add new features while extracting microservices 3. Warm up with a simple capability that has fewer dependencies 4. Get your monitoring and logging in place first 5. Build a shared understanding before you implement 6. Evolve your team and organizational structure to promote your desired architecture Retro action points Evolve your team to suit your architecture
- 35. 1. Identify your expectations with microservices 2. Make sure you are competent enough for microservices Two more points before you start!
- 36. Freipost, Carsales & Bank X 2010 Carsales: Monolith Carpoint acquisition (inherited the old codebase) 2012 Carsales: Small chunks of monolith Carsales, Bikesales were separated Started migrating to AWS 2014 Carsales: Infrastructure change Migrated to AWS and started adopting CI/CD 2016 Freipost: Monolith 2017 Carsales: Microservices Using dockers and separating microservices based on features. Freipost: Half Done Microservices 2019 Bank X: Monolith Started out as Distributed Monoliths and on the journey...
- 37. ● Going microservices is an EPIC journey ● You need stamina, perseverance, and grit! ● Be prepared for lots of trials and fails that could take years! ● You need to keep asking yourself: “WHY MICROSERVICES? Is it really for me?” Conclusion