Check Please!
0 likes639 views
This document discusses the importance of monitoring a Postgres database to prevent issues and ensure high performance. It provides examples of key things to monitor such as connections, disk space, transactions, queries, and settings. Regular monitoring helps identify problems early before they seriously impact the database or lead to data loss or downtime. Tools can help support monitoring but experience is also important.
Check Please!
- 1. Check Please! What your Postgres database wishes you would monitor / Presentation Friday, May 21, 2010
- 2. Who am I? • Lead Database Operations at OmniTI • Database Consulting / Management • Postgres? • TB+ OLAP/DSS • multiple 1000+ tps OLTP • custom built, private label • long time user (6.5-9.x) • community member • major contributor Friday, May 21, 2010
- 3. Check Yourself! • Basic Tuning Is Job #1 • shared buffers, effective cache size, checkpoints • Resources • http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server • http://www.slideshare.net/xzilla/the-essential-postgresqlconf-presentation Friday, May 21, 2010
- 4. Before You Wreck Yourself! Friday, May 21, 2010
- 5. Before You Wreck Yourself! • Monitoring • If a server crashes in the woods • Pain is a great motivator Friday, May 21, 2010
- 6. Before You Wreck Yourself! Friday, May 21, 2010
- 7. Before You Wreck Yourself! • Trending • Knowing what things look like when they’re good helps determine when things are bad • You can often tell where you’re going by looking at where you came from Friday, May 21, 2010
- 8. Tools? • Tools cannot replace experience and discipline • But they can help you maintain that discipline • Popular tools • nagios / munin • cacti / mrtg • circonus.com / reconnoiter • { check_postgres } Friday, May 21, 2010
- 9. Connections • Hard limit on allowed connections • Game Over • Large numbers of concurrent users • Internet facing systems - Paul RJ Muller, beached whale Friday, May 21, 2010
- 10. Disk Space • Data, clog, xlogs, log files • Game Over • (planning) • Everybody - Squiggle, Overloaded? Friday, May 21, 2010
- 11. WAL Files • pg_xlog directory, transaction logs • maintain database consistency • excessive disk space • excessive recovery time • heavy write transactions • pg start/stop backup (buggy systems) - Jazzmasterson, Workspace 3.0 - Noguchi File Friday, May 21, 2010
- 12. Size Matters • more data == more disk space • only grow if you should • large tables, i/o issues • unbounded growth? • fast paced development • everyone else (eventually) - elmada, Size Matters Friday, May 21, 2010
- 13. Bloating • mvcc leaves dead rows • unused space in tables, indexes • i/o issues • disk space (eventually) • heavy updates, data churn • untuned systems - Joe Alterio, burpalurpa Friday, May 21, 2010
- 14. Transactions • every statement is in a transaction • select/insert/update/deletes • underlying effects • load spikes • OLTP Systems • Logging/Internet Facing Systems - laurieofindy , Walmart on Black Friday 2009 Friday, May 21, 2010
- 15. All Stats • pg_stat tables • tables, indexes... scans, tuples • underlying effects • load spikes • OLTP Systems • Logging/Internet Facing Systems - Inju, Statistics for the Utterly Confused Friday, May 21, 2010
- 16. Free Space Map • Tracks unused space • Keeps vacuum effective • Table / Index Bloat • Medium to Large Systems • High Update / Data Churn DB - SkyTruth, Deepwater Horizon Oil Spill - RADARSAT-2, May 8, 2010 Friday, May 21, 2010
- 17. Autovacuum Max Freeze Age • Ensures all tables get vacuumed • Prevents XID wrap-around • Heavy I/O • Locking Issues • High TPS / OLTP • pg_dump - Joe Marinaro, Good Morning! Friday, May 21, 2010
- 18. Long Running Queries • postgres can’t freeze query plans • pg_stat_activity • Can cause issues for vacuum • Uh, response time obligations • active data collection • developers write queries ;-) - _Tawcan, Spiral Out..keep going Friday, May 21, 2010
- 19. Idle Transactions • BEGIN; zzz... • pg_stat_activity • Can cause issues for vacuum • Connections holding memory • pretty much everyone - psd, Canadian Cashpoints, Bah! Friday, May 21, 2010
- 20. Sequence limits • sequences limited to 2 billion • non-transactional • Can break inserts • Heavy insert (update?) systems - gavinzac, Rise and Fall in Donegal Friday, May 21, 2010
- 21. Wrap-around • Postgres must vacuum every table within 2 Billion transactions • Catastrophic data loss • pretty much everyone - Jurvetson, Wrapped Around the Axle Friday, May 21, 2010
- 22. Settings • postgresql.conf doesn’t always reflect reality • temporary changes can lead to long term trouble • pretty much everyone - denovich, P9220453.jpg Friday, May 21, 2010
- 23. Thanks! • PGCon • PGCommunity • OmniTI • Want more? • [email protected], http://www.xzilla.net • @robtreat2 • [email protected], http://www.omniti.com/is/hiring Friday, May 21, 2010