Finding Patterns in the Clouds - Cloud Design Patterns
5 likes449 views
The document discusses the evolution of web application architectures from single server deployments to modern cloud-native microservice architectures. It covers how early web apps faced challenges as demand grew that led to adopting caching, load balancing, and the cloud. It then discusses how further scaling issues were addressed through containers, microservices, and patterns like API gateways and circuit breakers. Key lessons are around adopting cloud-native patterns to improve performance, scalability, resilience and developer productivity for applications.
![Circuit Breaker States
@ardalis | Finding Patterns in the Clouds
Closed
Open
When failCount > threshold, Open
Half-Open
After [timeout], go to Half-Open
Still not responding…
Reset failCount to zero and Close
Retry on failure; increment failCount](https://image.slidesharecdn.com/clouddesignpatterns-200111014501/85/Finding-Patterns-in-the-Clouds-Cloud-Design-Patterns-73-320.jpg)
Finding Patterns in the Clouds - Cloud Design Patterns
- 1. Finding Patterns in the Clouds Steve “ardalis” Smith @ardalis | [email protected] ardalis.com | weeklydevtips.com Design Patterns for Cloud- Native Applications
- 2. Please Rate in the App • AttendeeHub @ardalis | Finding Patterns in the Clouds
- 3. More Resources • Podcast WeeklyDevTips.com • Group Mentoring Program DevBetter.com • Free Microsoft eBooks ardalis.com/architecture-ebook ardalis.com/cloud-native-book @ardalis | Finding Patterns in the Clouds
- 4. Let’s take a trip back to the beginning of today’s web… @ardalis | Finding Patterns in the Clouds
- 7. A Simpler Time “Just” ~20 years ago… Compaq AlphaServer DS20 circa 1999 @ardalis | Finding Patterns in the Clouds
- 8. A Simpler Time “Just” 20 years ago…
- 9. One Web Server To Rule Them All Client Machine Server Request Response App Data (and one Webmaster to run it all – the original Full Stack Developer™) @ardalis | Finding Patterns in the Clouds NCSA Mosaic
- 10. One Web Server To Rule Them All Client Machine Server Request Response
- 11. Report Card: One Server To Rule Them All @ardalis | Finding Patterns in the Clouds Availability Data Management Design and Implementation Messaging Security Management and Monitoring Resiliency Performance and Scalability
- 12. Web App Considerations and Challenges @ardalis | Finding Patterns in the Clouds
- 13. Cloud-Hosted Web App Considerations and Challenges @ardalis | Finding Patterns in the Clouds
- 14. Availability How often is the system or service up? Often expressed as a percentage. 99.99% uptime = 1 minute of downtime per week 99.999% uptime = 26 seconds of downtime per month @ardalis | Finding Patterns in the Clouds
- 15. Data Management Many more options than in traditional-hosted single-database apps Distributed data Consistency Synchronization @ardalis | Finding Patterns in the Clouds
- 16. Design and Implementation Consistency is key Consider factors like • Maintenance ease • Administration • Development • Diagnostics • Cost @ardalis | Finding Patterns in the Clouds
- 17. Messaging How do subsystems communicate? Direct, synchronous calls? Asynchronous messaging? Each option presents challenges. @ardalis | Finding Patterns in the Clouds
- 18. Management and Monitoring No direct server access to PaaS resources means other tools are critical. Cloud resources are more like cattle herds than pets. @ardalis | Finding Patterns in the Clouds
- 19. Performance and Scalability How responsive is the system to requests? How does this responsiveness change with increased load? Scaling up Scaling out @ardalis | Finding Patterns in the Clouds
- 20. Resiliency Can the system gracefully (and automatically) recover from errors or failures? Detect failures and replace resources automatically @ardalis | Finding Patterns in the Clouds
- 21. Security Protect from attacks Guard sensitive data Restrict access to approved users @ardalis | Finding Patterns in the Clouds
- 22. One Machine To Rule Them All Client Machine Server Request Response App Data @ardalis | Finding Patterns in the Clouds
- 23. Report Card: One Server To Rule Them All 🙁 🙂 🙂 🙂 🙂 🙂 but also 🙁 😐 😐 @ardalis | Finding Patterns in the Clouds
- 24. Demand Grows
- 25. 🙁 Vertical Scaling is Maxed Out 🙁 Performance is suffering at times Remedy: Move Database to a separate server 🙂 Performance improves Current Assessment: @ardalis | Finding Patterns in the Clouds
- 26. 1 Web, 1 DB Server Client Machine Web Server Request Response App Data DB Server @ardalis | Finding Patterns in the Clouds
- 27. Parts of the App are SLOW
- 28. 🙁 Vertical Scaling is Maxed Out (both servers) – or at least there’s no budget for more right now 🙂 Data has been optimized with indexes, etc. No more gains to be had here. 🙁 Some queries just hammer the database, take time, and impact other queries. Current Assessment: @ardalis | Finding Patterns in the Clouds
- 29. Cache Aside Pattern @ardalis | Finding Patterns in the Clouds
- 30. Read-Through Strategy @ardalis | Finding Patterns in the Clouds
- 31. Write-Through Strategy Data 1. Update the data store 2. Invalidate (or update) its cache entry $123 @ardalis | Finding Patterns in the Clouds Another option is to use a very short cache duration, something I refer to as micro-caching
- 32. Add Simple Memory Caching Client Machine Web Server Request Response App Data DB Server Cache @ardalis | Finding Patterns in the Clouds
- 33. There are only 2 hard things in Computer Science 0. Cache Invalidation 1. Naming Things 2. Off-by-One Errors @ardalis | Finding Patterns in the Clouds
- 34. Speaking of Naming Things… • Use a standard way to generate a cache keys for given scenarios • Avoid hard-coding keys, especially as local method literals • Many caching patterns require access to keys from different parts of your applications (including read vs. write operations) @ardalis | Finding Patterns in the Clouds
- 35. 🙂 Performance is usually (much) better 🙁 Some users still complain of delays due to cache misses 🙁 Keeping Cache up to date is a new challenge 🙁 Customers may now see stale data New Problems… 🙁 New monitoring required for cache 🙁 New tools to clear or update cache required Hmm, that’s a lot of 🙁
- 36. Demand Grows
- 37. Time to Scale Out!
- 38. Simple Web Farm Client Machine Web Server Request Response App Data DB Server Cache Load Balancer Web Server App Cache @ardalis | Finding Patterns in the Clouds
- 39. 🙂Performance and Scalability improved 🙁 Some users still complain due to cache misses 😧 Keeping Multiple Caches up to date is a big challenge (in this model) New Behavior… 🙁New monitoring and tools required for load balancer 🙁 More servers to manage 🙂 Web servers can be updated without taking down the whole system @ardalis | Finding Patterns in the Clouds
- 40. Embrace the Cloud! @ardalis | Finding Patterns in the Clouds
- 41. The Cloud (xkcd.com/908) @ardalis | Finding Patterns in the Clouds
- 42. Simple Cloud Architecture Client Machine Request Response Load Balancer App Service Instances Azure Cache / Redis Instance Azure SQL Database @ardalis | Finding Patterns in the Clouds
- 43. 🙂 Scalability improved 🙁 Some users still complain due to cache misses (consider priming the cache) 🙂 Cache synchronization easier New Behavior… 🙂 No more servers to manage 🙂 Monitoring tools built-in to platform 🙂 Web instances easily managed without downtime That ratio of 🙂 to 🙁 is a lot better…@ardalis | Finding Patterns in the Clouds
- 44. “Let’s build more of these apps” @ardalis | Finding Patterns in the Clouds
- 45. More Apps Client Machine Request Response Load Balancer App Service Instances Azure Cache / Redis Instance Azure SQL Database @ardalis | Finding Patterns in the Clouds
- 46. More Apps App Service Instances Azure Cache / Redis Instance Azure SQL Database App A App Service Instances App B App Service Instances App C Shared Resources
- 47. Everything’s Great! Except… Vendor lock-in Shared database hurts Shared resources (e.g. data schema) limit app developer agility We’ll address these in a moment but first… @ardalis | Finding Patterns in the Clouds
- 48. “Don’t forget auth!” @ardalis | Finding Patterns in the Clouds
- 49. Authentication and Identity These apps require: • Single Sign-on • Security – protection from unauthorized use • This was surely built into the apps before this point, but the pain becomes apparent now @ardalis | Finding Patterns in the Clouds
- 50. Simple Database Managed Identity App Service Instances Azure Cache / Redis Instance Azure SQL Database App A App Service Instances App B App Service Instances App C Identity Tables/Data Username Password Login Username Password Login Username Password Login YOU get a login screen, and YOU get a login screen
- 51. Federated Identity Pattern @ardalis | Finding Patterns in the Clouds
- 53. Redis Cache Azure SQL App A Redis cache Azure SQL App B Redis cache Azure SQL App C Identity Microservice Leveraging Containers and Federated Identity Azure SQL Client Machine 1. Authenticate 2. Get Secure Token 3. Present Token Note: Vendor lock-in mitigated by containers
- 54. “What about data sync?” @ardalis | Finding Patterns in the Clouds
- 55. Redis Cache Azure SQL App A Redis cache Azure SQL App B Redis cache Azure SQL App C Identity Microservice Implementing a Message/Event Bus Azure SQL @ardalis | Finding Patterns in the Clouds
- 56. Moving from Apps to Microservices • Apps are very coarse-grained to deploy, scale • Common functionality duplicated between apps • Stable parts of apps disrupted by deployment of unstable bits • Decompose apps into small, independent, cohesive microservices @ardalis | Finding Patterns in the Clouds
- 57. App A – An eCommerce Site Redis Cache Azure SQL App A Client apps Mobile app Web app @ardalis | Finding Patterns in the Clouds
- 58. Microservice 2 Microservice 1 container container Web API Web API Microservice 3 container Web API Client apps Microservices Split into Microservices Call each as appropriate from clients Mobile app Web app
- 59. Security Concerns • Client apps may not need every microservice feature • Microservices may have multiple clients; shouldn’t need to know security rules of every one • Should limit feature surface area specific to client needs @ardalis | Finding Patterns in the Clouds
- 60. API Gateway Pattern @ardalis | Finding Patterns in the Clouds
- 61. Using a custom API Gateway Service Microservice 2 Microservice 1 Client WebApp MVC container container Web API Web API ASP.NET Core MVC container Microservice 3 container Web API Client SPA Web app JavaScript Client mobile app API Gateway ASP.NET Core Web API container Back end Traditional Web app Browser HTML HTML JSON JSON
- 62. API Gateway with Azure API Management Architecture Client WebApp MVC ASP.NET Core MVC container Client SPA Web app JavaScript Client mobile app Developer portal API Gateway Publisher portal Azure API Management Microservice 2 Microservice 1 container container Web API Web API Microservice 3 container Web API Back end
- 63. Accessing Secure Files @ardalis | Finding Patterns in the Clouds
- 64. New Problem – Secure File Access • Apps control access to media files based on authorized user • Simple approach of a dumb CDN doesn’t protect actual media URLs from being accessed by anyone • Current solution: Web App authenticates user, accesses the file, and streams it to the end user @ardalis | Finding Patterns in the Clouds
- 65. Secure Media File Access Client Machine Request Response Web App Request Response File/BLOB Store (not publicly accessible) @ardalis | Finding Patterns in the Clouds
- 66. Concerns 🙁 Load on web app higher than necessary 🙁 Cost! 💰 May be paying extra to move files in/out of web app 🙁 Greater chance of downtime with web app and file store both required to stream file @ardalis | Finding Patterns in the Clouds
- 67. Valet Key • Provide direct access to media files using an access token • Azure supports Shared Access Signatures (SAS) for this purpose • File transfers occur directly between file store and client @ardalis | Finding Patterns in the Clouds
- 68. Remote Resources @ardalis | Finding Patterns in the Clouds
- 69. “I want fast, reliable, always-on services!” @ardalis | Finding Patterns in the Clouds
- 70. Retry Pattern First attempt failed • Is it likely to be a transient problem? If not, give up. • Immediately try again (maybe it was just a fluke) • (wait) • Try again • (wait longer) • Try again • Give up. @ardalis | Finding Patterns in the Clouds
- 71. Retries can overload downstream service • Imagine typical load is 10 requests per second. • With a typical “try 3 times then fail” strategy, when the service comes up it’s immediately seeing 30 requests per second of load! • Can result in longer time to respond and creation of more resources than necessary (more hosting $$$) • Don’t DOS (denial of service) yourself! @ardalis | Finding Patterns in the Clouds
- 72. Circuit-Breaker Pattern 3 States • Closed (working) • Open (not working) • Half-Open (throttled)
- 73. Circuit Breaker States @ardalis | Finding Patterns in the Clouds Closed Open When failCount > threshold, Open Half-Open After [timeout], go to Half-Open Still not responding… Reset failCount to zero and Close Retry on failure; increment failCount
- 74. Consider a tool like Polly @ardalis | Finding Patterns in the Clouds
- 75. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 1 • Microservices access reporting data directly • Introduces coupling • Reduces microservice independence • Bypasses microservice logic Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
- 76. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 2 • Reporting app hits microservices directly • Poor performance • Data may be too large for HTTP • Difficult to perform complex queries Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
- 77. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 3 • Batch job updates reporting db from microservice dbs • Same coupling as approach 1. Changes to microservice db schemas break batch data job. Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
- 78. Bonus Microservice Anti-Pattern: Reach-In Reporting Solution • Async event publication • Encapsulation and independence of microservices is preserved • Performance is usually acceptable Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
- 79. Key Takeaways • Cloud architecture abstracts away servers • Cache Aside pattern is great for performance improvements • Containers offer improved deployment and scaling options with less vendor lock-in • Microservices offer finer-grained control over app functionality • Federated Identity improves security and user experience • API Gateways help secure collections of services • Valet key provides cheaper, faster access to secure media • Consider Retry but you may need a Circuit-Breaker • Avoid the Reach In Reporting anti-pattern for your microservices @ardalis | Finding Patterns in the Clouds
- 80. More Cloud Design Patterns https://bit.ly/1T8q2w8 @ardalis | Finding Patterns in the Clouds
- 81. Thank You! • Contact me! twitter.com/ardalis [email protected] • Podcast WeeklyDevTips.com • Group Mentoring Program DevBetter.com • Free Microsoft eBooks ardalis.com/architecture-ebook ardalis.com/cloud-native-book