1404 app dev series - session 8 - monitoring & performance tuning
Download as PPTX, PDF1 like852 views
This document discusses MongoDB monitoring tools and key metrics. It provides an overview of tools like mongostat, the MongoDB shell, MMS, and mtools for monitoring operations per second, memory usage, page faults, and other metrics. It also discusses using logs to analyze query performance and disk saturation. The importance of monitoring queued readers/writers, page faults, background flush processes, memory usage, locks, and other core metrics is highlighted.
![27
• Increase the log level
– See further insight to operation performance
– Index efficiency
– Document moves
– Operation time
Set log level
//Increase log level verbosity from the shell
> db.adminCommand( { setParameter:1, logLevel:1 } )
{ "was" : 0, "ok" : 1 }
>
-v [ --verbose ] be more verbose (include multiple times for more
verbosity e.g. -vvvvv)](https://image.slidesharecdn.com/1404-appdevseries-session8-monitoringperformancetuning-140516052049-phpapp02/85/1404-app-dev-series-session-8-monitoring-performance-tuning-27-320.jpg)
![28
• Pinpoint problems
– In this example we can see that there has been a document move.
Log output example
2014-05-02T13:55:02.047+0100 [conn7] update cms.articles query: {
_id: ObjectId('532198379fb5ba99a6bd4063') } update: { $inc: {
comment_count: 1 }, $push: { comments: { $each: [ { date: new
Date(1399035302013), text: "Data locality provides an amazing
performance boost over relational" } ], $slice: -10, $sort: { date: 1 } } } }
nscanned:1 nscannedObjects:1 nmoved:1 nMatched:1 nModified:1
keyUpdates:0 numYields:0 locks(micros) w:33529 33ms](https://image.slidesharecdn.com/1404-appdevseries-session8-monitoringperformancetuning-140516052049-phpapp02/85/1404-app-dev-series-session-8-monitoring-performance-tuning-28-320.jpg)
1404 app dev series - session 8 - monitoring & performance tuning
- 1. Application Development Series Back to Basics Monitoring & Performance Tuning Daniel Roberts @dmroberts #MongoDBBasics
- 2. 2 • Recap from last session • Monitoring Tools – MongoDB command line and shell • Key Metrics in MongoDB • Logs – Log levels – mtools • Disk saturation Agenda
- 3. 3 • Virtual Genius Bar – Use the chat to post questions – EMEA Solution Architecture / Support team are on hand – Make use of them during the sessions!!! Q & A
- 4. Recap from last time….
- 5. 5 • mongodump & mongorestore • File system copy • Files system snapshot • Don’t use mongoimport & mongoexport! Backup Tools & Approaches //mongodump Example server> mongodump -h myhost -d cms -c articles
- 6. 6 • Creates a .bson file – (and a json metadata file) – Over network or direct from file system. Mongodump >mongodump –h myhost -d cms -c articles connected to: myhost 2014-04-16T12:54:56.758+0100 DATABASE: cms to dump/cms 2014-04-16T12:54:56.759+0100 cms.articles to dump/cms/articles.bson 2014-04-16T12:54:56.816+0100 7 documents 2014-04-16T12:54:56.817+0100 Metadata for cms.articles to dump/cms/articles.metadata.json
- 7. 7 • Use secondary for backup – Or hidden secondary • Fsync+Lock • Balance off – In sharded cluster Backup Architecture mongodump
- 9. 9 • Key MongoDB tools – Mongostat – Mongo shell – Mongo Management Service mms • Mtools – Logs • OS – iostat Tools
- 11. 11 MMS Alerts
- 12. 12 Metrics Collection and Reporting
- 13. 13 Logs and Profile data
- 14. 14 • Access to key statistics – Connect to mongod or mongos for sharded cluster stats – Shows, operations per second, memory usage, page faults, queues and more Mongostat >mongostat -h localhost --port 27017 connected to: localhost:27017 insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:24 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:25 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:26 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:27 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:28 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:29 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:30 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:31 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.1% 0 0|0 0|0 62b 5k 2 09:08:32 *0 *0 *0 *0 0 1|0 0 1.67g 5.78g 29m 0 .:0.0% 0 0|0 0|0 62b 5k 2 09:08:33
- 15. 15 • Commands for access metrics – db.serverStatus() – workingSet analyzer Mongo shell //to access a subset of serverStatus output specify the sub document >db.serverStatus().mem { "bits" : 64, "resident" : 266, "virtual" : 5920, "supported" : true, "mapped" : 1712, "mappedWithJournal" : 3424 }
- 16. 16 • Estimates the ‘working set’ of the database – Number of ‘pagesInMemory’ accessed in RAM over ‘overSeconds’ number of seconds. – Default Page is 4k – overSecond – delta between oldest and newest WorkingSet analyzer >db.serverStatus({ workingSet : 1 }) { "note" : "thisIsAnEstimate", "pagesInMemory" : 723, "computationTimeMicros" : 4601, "overSeconds" : 2311 } overSeconds decreasing or small, workingSet could be larger than RAM
- 17. MongoDB Metrics
- 18. 18 • Queued readers | writers • Page faults • OpCounters • Background flush process • Memory usage • Lock % • Btree misses • connections Key Metrics Available from : • MMS • db.serverStatus() • mongostat
- 19. 19 Queued Reader | Writers • Number of operations waiting for: • Read Lock • Write Lock
- 20. 20 Page Faults • How often we are going to disk to retrieve data? • Can be a problem if disk IO is saturated • Combine with iostat
- 21. 21 Background Flush Process • Length of time to flush data back to disk • Again can point to disk saturation.
- 22. 22 Memory • Virtual • Mapped • Resident
- 23. 23 Lock % • Percentage of time in the global lock
- 24. 24 btree • Accesses • Hits • Misses • Bad! • Points to Indexes not in RAM
- 25. 25 connections • Connections to the database • Growing over time? • Check pool logic in your application code.
- 26. Logs
- 27. 27 • Increase the log level – See further insight to operation performance – Index efficiency – Document moves – Operation time Set log level //Increase log level verbosity from the shell > db.adminCommand( { setParameter:1, logLevel:1 } ) { "was" : 0, "ok" : 1 } > -v [ --verbose ] be more verbose (include multiple times for more verbosity e.g. -vvvvv)
- 28. 28 • Pinpoint problems – In this example we can see that there has been a document move. Log output example 2014-05-02T13:55:02.047+0100 [conn7] update cms.articles query: { _id: ObjectId('532198379fb5ba99a6bd4063') } update: { $inc: { comment_count: 1 }, $push: { comments: { $each: [ { date: new Date(1399035302013), text: "Data locality provides an amazing performance boost over relational" } ], $slice: -10, $sort: { date: 1 } } } } nscanned:1 nscannedObjects:1 nmoved:1 nMatched:1 nModified:1 keyUpdates:0 numYields:0 locks(micros) w:33529 33ms
- 29. 29 • Log analysis (example syntax) – Show me queries that took more than 1000 ms from 6 am to 6 pm: • Now, graph those queries: mtools prompt> mlogfilter mongodb.log --from 06:00 --to 18:00 -- slow 1000 > mongodb-filtered.log prompt> mplotqueries --logscale mongodb-filtered.log
- 30. 30 mtools output
- 31. Disk saturation
- 32. 32 • iostat – Disk IO usage – Use to review % utilisation of disk – High % • If it’s sustained, you may need to increase disk IO. – Sharding – RAID 1+0 – Partitioning on different disk – Provisioned IOPS OS Tools and Paramenters prompt> iostat –xmt 1
- 33. Summary
- 34. 34 • Monitor regularly – Set alerts for changes • Quick check with command line and shell • View trends with MMS graphs over time • High levels of utilisation – Increase RAM, disk IO – Scale out – (Having first checked Indexes etc!) • Database profiler (covered in Indexing session) Summary