AWS Re:Invent - Optimizing Costs with AWS
Download as PPTX, PDF8 likes5,171 views
The document discusses cost optimization strategies using AWS at Netflix, focusing on high-level methodologies and resource-specific optimizations. It emphasizes the importance of improving availability, adjusting capacity for workloads, and utilizing a range of specialized tools and frameworks to enhance efficiency. Additionally, it covers Netflix’s cultural approach to engineering and highlights various topics presented at the re:Invent conference.
AWS Re:Invent - Optimizing Costs with AWS
- 1. Optimizing Costs with AWS
- 3. • Rationale and High-level Methodology • AWS resource-specific optimizations • Performance Testing • Results • Q&A
- 5. Rationale • Applications operate at massive scale • Across three regions and multiple zones per region • Service oriented architecture • Many moving parts (teams) • Unconstrained deployment capabilities • “Freedom and Responsibility” culture
- 6. Rationale, cont. • Improve availability • Avoid saturation of key resources • Dynamically adjust capacity to meet workload demands • Plan for increasing workloads • Less focus on reducing current demand • Maximize efficiency • Balance OLTP and batch demands • “That which is measured improves”
- 7. • Asgard framework enables turnkey deployment (Netflix open-sourced) • All engineers have full access • Real-time reservation capacity • Unconstrained ASG size limits
- 9. • Birds-eye view of usage • Near real-time data • Open sourcing plans for tool • Decomposes by application
- 15. Unused Reservation Instance Hours * 2,000 1,500 Need to use this 1,000 capacity 500 0 Mon Tue Wed Thu Fri Sat * - fictitious volumes
- 19. Healthy Thrashing Double-Jump Y-axis = number of instances in ASG
- 22. Adopted batch delete Requests/day Started batch send adoption Batch capabilities Adoption complete Time
- 28. Legend Github / Techblog Priam Exhibitor Servo and Autoscaling Scripts Apache Cassandra as a Service Zookeeper as a Service Contributions Astyanax Curator Honu Techblog Post Only Cassandra client for Java Zookeeper Patterns Log4j streaming to Hadoop Coming Soon CassJMeter EVCache Circuit Breaker - Hystrix Cassandra test suite Memcached as a Service Robust service pattern Cassandra Multi-region EC2 Eureka / Discovery Asgard - AutoScaleGroup datastore support Service Directory based AWS console Aegisthus Archaius Chaos Monkey Hadoop ETL for Cassandra Dynamics Properties Service Robustness verification Edda Explorers Latency Monkey Queryable config history Governator - Library lifecycle Server-side latency/error Janitor Monkey and dependency injection injection Odin REST Client + mid-tier LB Bakeries and AMI Workflow orchestration Blitz4j - Async logging Configuration REST endpoints Build dynaslaves
- 29. Netflix at 2012 re:Invent Date/Time Presenter Topic Wed 8:30-10:00 Reed Hastings Keynote with Andy Jassy Wed 1:00-1:45 Coburn Watson Optimizing Costs with AWS Wed 2:05-2:55 Kevin McEntee Netflix’s Transcoding Transformation Wed 3:25-4:15 Neil Hunt / Yury I. Netflix: Embracing the Cloud Wed 4:30-5:20 Adrian Cockcroft High Availability Architecture at Netflix Thu 10:30-11:20 Jeremy Edberg Rainmakers – Operating Clouds Thu 11:35-12:25 Kurt Brown Data Science with Elastic Map Reduce (EMR) Thu 11:35-12:25 Jason Chan Security Panel: Learn from CISOs working with AWS Thu 3:00-3:50 Adrian Cockcroft Compute & Networking Masters Customer Panel Thu 3:00-3:50 Ruslan M./Gregg U. Optimizing Your Cassandra Database on AWS Thu 4:05-4:55 Ariel Tseitlin Intro to Chaos Monkey and the Simian Army
- 30. We are sincerely eager to hear your FEEDBACK on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance. Contact: [email protected]