mongoDB Performance
7 likes3,958 views
The document discusses MongoDB performance optimization strategies presented by Moshe Kaplan at a VP R&D Open Seminar. It covers topics like sharding, in-memory databases, MapReduce, profiling, indexes, server stats, schema design, and locking in MongoDB. Slides include information on tuning configuration parameters, analyzing profiling results, explain plans, index management, and database stats.
![http://blogs.microsoft.co.il/blogs/vprnd
http://top-performance.blogspot.com
Profiling Configuration
Enable:
• mongod --profile=1 --slowms=15
• db.setProfilingLevel([level] , [time])
How much:
• 0 (none) 1 (slow queries only) 2 (all)
• 100ms: default
Where:
• system.profile collection
17](https://image.slidesharecdn.com/20130520-mongodbperformance-130520145908-phpapp01/85/mongoDB-Performance-17-320.jpg)
mongoDB Performance
- 1. The VP R&D Open Seminar MongoDB Performance Moshe Kaplan [email protected] http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
- 5. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com5 Strategy B - In Memory Databases
- 6. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com6 700 Inserts/Sec In Memory Engine 3000 Inserts/Sec InnoDB Engine 700 Inserts/Sec Amazon AWS Standard Large Instance
- 7. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com7 Strategy C – MapReduce http://blogs.microsoft.co.il/blogs/vprnd
- 8. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Strategy D - NoSQL insert get multiget remove truncate 8 <Key, Value> http://wiki.apache.org/cassandra/API
- 10. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar MONGODB TUNING 10
- 11. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com journalCommitInterval = 300: Write to disk: 2ms <= t <= 300ms Default 100ms, increase to 300ms to save resources Disk The Journal 11 Memory Journal Data 1 2
- 12. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com mongod.conf Tuning maxConns: # of connection (<20,000) objcheck: BSON validation. false to save overhead. nohttpinterface = true: disables http interface noprealloc = false: preallocates space @ startup noscripting = true: disables scripting engine profile = 0: disable profiling rest = false: disable REST service 12
- 13. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar SERVER STATS 13
- 14. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com What we are facing of? > db.serverStatus().mem; > db.serverStatus().extra_info; > db.serverStatus().globalLock; > db.serverStatus().backgroundFlushing; > db.serverStatus().connections; > db.serverStatus().network; 14 RAM Page Faults Lock Ratio I/O Speed Not Too Many Bandwidth
- 15. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com RAM Optimization dataSize + indexSize < RAM 15 OS Data Index Journal
- 16. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar PROFILING AND SLOW LOG 16
- 17. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Profiling Configuration Enable: • mongod --profile=1 --slowms=15 • db.setProfilingLevel([level] , [time]) How much: • 0 (none) 1 (slow queries only) 2 (all) • 100ms: default Where: • system.profile collection 17
- 18. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Profiling Results Analysis Last 5 >1ms: show profile w/o commands: db.system.profile.find( { op: { $ne : 'command' } } ).pretty() Specific database: db.system.profile.find( { ns : 'mydb.test' } ).pretty() Slower than: db.system.profile.find( { millis : { $gt : 5 } } ).pretty() Between dates: db.system.profile.find({ts : { $gt : new ISODate("2012-12-09T03:00:00Z") , $lt : new ISODate("2012-12-09T03:40:00Z") }}).pretty() 18
- 19. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Explain > db.courses.find().explain(); { "cursor" : "BasicCursor", "isMultiKey" : false, "n" : 11, “nscannedObjects" : 11, "nscanned" : 11, "nscannedObjectsAllPlans" : 11, "nscannedAllPlans" : 11, "scanAndOrder" : false, "indexOnly" : false, "nYields" : 0, "nChunkSkips" : 0, "millis" : 0, "indexBounds" : {}, "server" : "primary.domain.com:27017" } 19
- 20. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar INDEXES 20
- 21. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Index Management Regular Index db.users.ensureIndex( { user_id: 1 } ) Multiple + DESC Index db.users.ensureIndex( { user_id: 1, age: -1 } ) Sub Document Index db.users.ensureIndex( { address.zipcode: 1 } ) List Indexes db.users.getIndexes() Drop Indexes db.users.dropIndex(“indexName”) 21
- 22. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Known Index Issues Bound filter should be the last (in the index as well). 22
- 23. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar STATS & SCHEMA DESIGN 23
- 24. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Sparse Matrix? I don’t Think so mongostat > db.stats(); > db.collectionname.stats(); Fragmentation if storageSize/size > 2 db.collectionanme.runCommand(“compact”) Padding (wrong design) if paddingFactor > 2 24
- 25. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com The VP R&D Open Seminar THE LOCKING DILEMA 25
- 26. http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com Locking on Write (CRUD) 2.0 Global Lock 2.2 Database Lock 2.X … What to do till then? 26