Capacity Planning
Download as PPTX, PDF15 likes4,674 views
The document discusses the importance of capacity planning for MongoDB deployments, highlighting key factors such as availability, throughput, and responsiveness. It outlines the necessary resources, methods for estimation, and monitoring practices to prevent downtime and ensure scalable performance. Additionally, it emphasizes the iterative nature of capacity planning and offers insights into tools and best practices for effective management.
Capacity Planning
- 2. MongoDB Capacity Planning Jay Runkel Principal Solution Architect [email protected] @jayrunkel
- 3. Capacity Planning • What is capacity planning? • Why is it important? • Which resources are affected? • How to do it?
- 5. Fine Art of … Requirements
- 6. Fine Art of … Requirements Resources
- 7. Preparing for Launch • Developers are about to finish final Sprint • Code is good (so they say ) • You feeling comfortable to launch soon • How to deploy?
- 8. Requirements • Availability – Uptime requirements: RPO and RTO • Throughput – Average read/writes/users – Peek throughput – Operations per second ? per day? per month? • Responsiveness – What's the acceptable latency? • Higher during peek time? RTO=recovery time objective RPO=recovery point objective
- 9. Resources
- 10. Resources • CPU • Storage • Memory • Network
- 12. Resource Usage • Storage • IOPS • Size • Data & Loading Patterns • CPU • Speed • Cores • Memory • Working Set • Network • Latency • Throughput
- 13. Why is that Important?
- 14. Why? • Once we launch, we don't want to have avoidable down time due to poorly selected HW • As our success grows we want to stay in front of the demand curve • We want to meet business and users expectations • We want to keep our jobs! • Don't be the "goat"
- 15. Under allocation
- 16. Over Capacity
- 17. Over spending
- 18. Important Aspects • Capacity – Under – Over – Just Right? • Prediction Models – User/Load – OPS/Request – System Behavior (stress testing anyone?) • Change Velocity – Data / Resource-Allocation / Provisioning – Minimum Viable Product? – Future Releases / Roadmap
- 19. Important Aspects • When? – Not too early – Before is too late! – Iterative Process Launch Version 2
- 20. Important Aspects • When? – Not too early – Before is too late! – Iterative Process Launch Version 2
- 21. Important Aspects • When? – Not too early – Before is too late! – Iterative Process Launch Version 2
- 23. CPU • Compression/Decompression • Encryption/Decryption • Non-indexed Data • Sorting • Aggregation – Map/Reduce – Aggregation Framework • Data – Fields – Nesting – Arrays/Embedded-Docs
- 24. Network • Latency – WriteConcern – ReadPreference – Batching • Throughput – Update/Write Patterns – Reads/Queries
- 25. Network • Latency – W:? – Nearest – Bulk Write Operations • Throughput – Use $set operator – Use field projection on queries
- 26. Storage • Active • Archival • Loading Patterns • Integration (BI/DW)
- 27. Storage Capability Type IOPS 7200 rpm SATA ~ 75 – 100 15000 rpm SAS ~ 175 – 210 http://en.wikipedia.org/wiki/IOPS
- 28. Storage Capability Type IOPS 7200 rpm SATA ~ 75 – 100 15000 rpm SAS ~ 175 – 210 SSD Intel X25-E (SLC) ~ 5000 SSD Intel X25-M G2 (MLC) ~ 8000 http://en.wikipedia.org/wiki/IOPS
- 29. Storage Capability Type IOPS 7200 rpm SATA ~ 75 – 100 15000 rpm SAS ~ 175 – 210 SSD Intel X25-E (SLC) ~ 5000 SSD Intel X25-M G2 (MLC) ~ 8000 Amazon EBS ~ 100 Amazon EBS Provisioned Up to ~10,000 Amazon EBS Provisioned IOPS (SSD) Up to ~20,000 http://en.wikipedia.org/wiki/IOPS
- 30. Storage Capability Type IOPS 7200 rpm SATA ~ 75 – 100 15000 rpm SAS ~ 175 – 210 SSD Intel X25-E (SLC) ~ 5000 SSD Intel X25-M G2 (MLC) ~ 8000 Amazon EBS ~ 100 Amazon EBS Provisioned Up to ~10,000 Amazon EBS Provisioned IOPS (SSD) Up to ~20,000 FusionIO ~135,000 Violin Memory 6000 ~ 1,000,000 http://en.wikipedia.org/wiki/IOPS Higher IOPS higher the Cost!!!
- 31. Storage Considerations • Work out how much data you need to write per unit of time! • Databases will use storage to persist data – More data = Bigger indexes = More Storage • MongoDB Stores Information into Documents • BSON Format – http://bsonspec.org/
- 32. Memory • Working Set – Active Data in Memory – Measured Over Periods • And other operations – Sorting – Aggregation – Connections • WiredTiger Storage Engine Cache
- 33. Caching in WiredTiger RAM DISK (compressed) File System Cache (compressed) WT Cache (uncompressed) MongoD (WT) 50% of RAM
- 35. Basic Rules • Determine data size, working set, query throughput requirements • Use good measuring and monitoring practices • Plan ahead but be flexible! • Iterate – Review Requirements – Review Capacity
- 36. MongoDB cluster sizing at 30,000 ft • Disk Space • RAM • Query Throughput
- 37. • Generate a sample document set – Write some code? • Use db.stats() to measure – Disk space – Compression • Do the math – Estimate production disk requirements • Sum of disk space across shards > greater than required storage size Disk Space
- 38. • Sum of disk space across shards > greater than required storage size Disk Space: How Many Shards Do I Need? Example Data Size = 9 TB WiredTiger Compression Ratio: .33 Storage size = 3 TB Server disk capacity = 2 TB 2 Shards Required
- 39. RAM Requirements • Working Set < RAM • WorkSet = Indexes plus the set of documents accessed frequently • WorkSet in RAM – Shorter latency – Higher Throughput
- 40. Estimating Working Set • Using your sample data set – Create required indexes based upon queries – Use db.coll.stats() to get index size • Do the math to get production index size • Estimate the working set – Given the queries – What are the frequently accessed docs? – Examples: • Last x days of data • Most queried devices
- 41. RAM: How Many Shards Do I Need? Example Working Set = 428 GB Server RAM = 128 GB 428/128 = 3.34 4 Shards Required
- 42. • Measure max sustained query rate of a single server (with replication) – Use prototype/development version – Use application queries and data – Measure max sustained performance • Assume sharding overhead of 20-30% Query Rate
- 43. • Measure max sustained query rate of a single server (with replication) – build a prototype and measure • Assume sharding overhead of 20-30% Query Rate: How Many Shards Do I Need? Example Require: 50K ops/sec Prototype performance: 20 ops/sec (1 replica set) 4 Shards Required: 80 ops/sec * .7 = 56K ops/sec
- 44. Measuring & Monitoring • What to measure – IOPS – Page Faults – Resident Memory (Working Set) – Connections – Lock % • How to measure and monitor Ops/Cloud Manager • Command Line Tools – iostat – vmstat – mongostat
- 45. The Best Way to Run MongoDB Ops Manager allows you leverage and automate the best practices we’ve learned from thousands of deployments in a comprehensive application that helps you run MongoDB safely and reliably. Benefits include: 10x-20x more efficient operations Complete performance visibility Protection from data loss Assisted performance optimization
- 46. How It Works Ops Manager mongod mongodmongod Agent Agent Agent
- 47. Monitoring and Alerting Over 100+ database metrics Dozens of optimized charts Custom alerts so incidents don’t become emergencies
- 48. APM lntegration Monitor MongoDB alongside the rest of your app infrastructure using our RESTful API Leverage packaged integrations with leading APM platforms
- 49. Visual Query Profiler Identify your slow-running queries with the click of a button
- 50. Index Suggestions Index recommendations to improve your deployment
- 51. Automated Index Builds Automate rolling index builds to reduce operational overhead and the risk of failovers
- 52. Database Automation Automate tasks that you would have otherwise performed manually, such as… • Deploying a new cluster • Upgrades • Adding capacity • Database restores
- 53. Backup with Point-in-time Recovery Restore to precisely the moment you need, quickly and safely. Ops Manager is the only MongoDB backup solution that offers point-in-time backups of replica sets and cluster- wide snapshots of sharded clusters.
- 55. Capacity Planning is … • Needed – Involves resource allocation – Hardware specification and sizing – Cost! • Vital – Translate Requirements and Expectations into Experience and Functionality • And meeting those • Requires understanding your application – Measuring resource needs – Monitoring – Iterating – Repeating process
- 56. For More Information Resource Location Case Studies mongodb.com/customers Presentations mongodb.com/presentations Free Online Training education.mongodb.com Webinars and Events mongodb.com/events Documentation docs.mongodb.org MongoDB Downloads mongodb.com/download Additional Info [email protected]
Editor's Notes
- #6: Fine art of translating Requirements of the application into resources needed to support the application that implements those requirements
- #7: Fine art of translating Requirements of the application into resources needed to support the application that implements those requirements
- #12: Understanding the requirements and resources implication
- #24: Collection scans
- #28: These numbers are already old and not relevant anymore.
- #29: These numbers are already old and not relevant anymore.
- #30: These numbers are already old and not relevant anymore.
- #31: These numbers are already old and not relevant anymore.
- #46: MongoDB Ops Manager can do a lot for [ops teams]. Best Practices, Automated. Ops Manager takes best practices for running MongoDB and automates them. So you run ops the way MongoDB engineers would do it. This not only makes it more fool-proof, but it also helps you… Cut Management Overhead. No custom scripting or special setup needed. You can spend less time running and managing manual tasks because Ops Manager takes care of a lot of the work for you, letting you focus on other tasks. Meet SLAs. Automating critical management tasks makes it easier to meet uptime SLAs. This includes managing failover as well as doing rolling upgrades with no downtime. Scale Easily. Provision new nodes and systems with a single click.
- #47: Ops Manager agents are installed on servers (where MongoDB will be deployed), either through configuration tools such as Chef or Puppet, or by an administrator. The administrator creates a new design goal for the system, either as a modification to an existing deployment (e.g., upgrade, oplog resize, new shard), or as a new system. The agents periodically check in with the Ops Manager central server and receive the new design instructions. Agents create and follow a plan for implementing the design. Using a sophisticated rules engine, agents continuously adjust their individual plans as conditions change. In the face of many failure scenarios – such as server failures and network partitions – agents will revise their plans to reach a safe state. Minutes later, the system is deployed – safely and reliably.
- #48: Ops Manager monitors 100+ metrics that could impact the performance of your database Track key performance indicators across dozens of optimized charts Build customized alerts that trigger when metrics are out of range; have them delivered how you want
- #49: RESTful API to monitor MongoDB alongside the rest of your application infrastructure, from a single “pane of glass” Packaged integrations with leading APM platforms such as New Relic now available with MongoDB Cloud Manager
- #50: Quickly identify your slow-running queries. Part of MongoDB Ops Manager, the Visual Query Profiler displays how query and write latency vary over time With the click of a button, the Visual Query Profiler consolidates and displays metrics from all your nodes on a single screen
- #51: The Visual Query Profiler analyzes the data it displays and presents recommendations for new indexes that can be created to improve the performance of your deployment.
- #52: The best practice for adding new indexes to your deployment is a rolling index build – starting with each of the secondaries and finally applying changes to the original primary, after swapping its role with one of the secondaries. Ops Manager can automate this process across your replica sets, reducing your operational overhead and the risk of failovers caused by incorrectly sequencing management processes.
- #53: Ops Manager coordinates and orchestrates critical operational tasks across the servers in a MongoDB system. Tasks that you would have otherwise performed manually, such as… Deploying a new cluster Upgrades Adding capacity Database restores Use the Ops Manager UI directly, or invoke the Ops Manager RESTful API from your existing enterprise orchestration frameworks, such as Chef, Puppet, etc.
- #54: Restore to precisely the moment you need, quickly and safely. Ops Manager is the only MongoDB backup solution that offers point-in-time backups of replica sets and cluster-wide snapshots of sharded clusters. Integrates with standard network-mountable file systems Backups can be configured against specific collections, rather than the entire database