Top 10 Tips for an Effective Postgres Deployment
6 likes947 views
The document provides a comprehensive guide for effective PostgreSQL deployment, covering essential areas such as preparation, planning, questions to ask during deployment, uptime requirements, backups, and ongoing evaluation. It includes insights on hardware requirements, replication strategies, upgrade policies, and recovery options, emphasizing the importance of communication among teams and continuous adjustments. EDB's expertise and resources are highlighted, along with customer satisfaction regarding their PostgreSQL solutions.
Top 10 Tips for an Effective Postgres Deployment
- 1. © 2015 EDB All rights reserved. 1 10 Tips for an Effective Postgres Deployment
- 2. © 2015 EnterpriseDB Corporation. All rights reserved. 2 • EnterpriseDB Overview • Preparation, Planning, and Postgres • Top 10 Questions to Ask • Ongoing Questions and Dealing with Change • Summary and Resources • Q&A Agenda We will take 2 quick-polls during the webinar to gather group information
- 3. © 2015 EnterpriseDB Corporation. All rights reserved. 3 Brief EDB Overview
- 4. © 2015 EnterpriseDB Corporation. All rights reserved. 4 EDB is a Recognized Leader Magic Quadrant for Operational DBMS, Q4 ‘15 • Gartner Comments − “EnterpriseDB is responsible for many features of PostgreSQL, contributing to JSON, materialized views and partitioning. Increasing numbers of users of Gartner’s inquiry services recognize the EnterpriseDB brand.” − “Clients report that the functionality of EnterpriseDB's Postgres Plus Oracle compatibility feature is more than sufficient to run both mission- critical and non-mission critical applications.” − “Reference customers rated EnterpriseDB highly for the stability of its DBMS, and were satisfied with its HAS/DR features.”
- 5. © 2015 EnterpriseDB Corporation. All rights reserved. 5 POSTGRES innovation ENTERPRISE reliability 24/7 support Services & training Enterprise-class features & tools Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Enabling commercial adoption of Postgres
- 6. © 2015 EnterpriseDB Corporation. All rights reserved. 6 Postgres Plus Advanced Server Postgres Plus Cloud Database High Availability PerformanceManagement REMOTE DBA 24x7 SUPPORT PROFESSIONAL SERVICES TRAINING EDB Serves All Your Postgres Needs PostgreSQL Security
- 7. © 2015 EnterpriseDB Corporation. All rights reserved. 7 EDB Customers EDB currently has over 3,000 total customers including 63 of the Fortune 500 and 137 of the Forbes Global 2000
- 8. © 2015 EnterpriseDB Corporation. All rights reserved. 8 Preparation, Planning, and Postgres
- 9. © 2015 EnterpriseDB Corporation. All rights reserved. 9 Recovery (Objectives and Continuity) Upgrades Availability Requirements Application (Usage Pattern, Response Expectations) Hardware (Acquire and Configure) Deployment Planning ‘Timeline’
- 10. © 2015 EnterpriseDB Corporation. All rights reserved. 10 • Why ask these questions? − Keep everyone on the same page − Not everyone knows what you know − Your answers might mean something unforeseen by others • When to ask these questions? − Early − Often Why? When? © 2011 EnterpriseDB . All rights reserved.
- 11. © 2015 EnterpriseDB Corporation. All rights reserved. 11 Quick Poll Postgres Database Maturity Stage
- 12. © 2015 EnterpriseDB Corporation. All rights reserved. 12 • Question: What stage are you currently in with your Postgres project? • Answer choices: a. Evaluation or Proof of Concept b. Development – active development c. Deployment – planning for production or recently launched d. Maintenance Audience Quick Poll #1
- 13. © 2015 EnterpriseDB Corporation. All rights reserved. 13 The Top 10 List
- 14. © 2015 EnterpriseDB Corporation. All rights reserved. 14 1. Where should I focus my money? 2. How should I partition the I/O? 3. What does my app want? 4. Can my app handle what the DB offers? 5. Highly available or highly recoverable? 6. Is ASYNC enough or do I need SYNC? 7. Cascaded replication or fanned? 8. How do I upgrade? 9. Why upgrade? 10. How much data can I afford to lose? 10 Questions to Ask Yourself and Others
- 15. © 2015 EnterpriseDB Corporation. All rights reserved. 15 Hardware
- 16. © 2015 EnterpriseDB Corporation. All rights reserved. 16 1. Acquiring Hardware – Where should I focus my money? • Where should I focus my money? − RAM − Max it out − ECC − I/O − Low latency − Battery backed − Local, not network − CPU − More cache == more better − Less CPUs, more cores − One 12-core CPU > Two 6-core CPU
- 17. © 2015 EnterpriseDB Corporation. All rights reserved. 17 © 2011 EnterpriseDB . All rights reserved. 2. How Should I Partition the I/O ? http://www.enterprisedb.com/ ahc • $PGDATA/pg_xlog − SSDs − ext2 • $PGDATA/pg_log − syslog • $PGDATA/pg_stat_tmp − tmpfs • $PGDATA/base − zfs, ext4, xfs - YES − btrfs - not yet − ext3 - NO
- 18. © 2015 EnterpriseDB Corporation. All rights reserved. 18 I/O Partitioning (cont.) • Tablespaces (you should be using them) − Group like objects into a tablespace of their own − Indexes − Reporting/historical tables − zfs, ext4, xfs − Indexes on SSD • Things to avoid − Hardware deduplication (just don't) − Hybrid SSD/HDD (use tablespaces instead) − NFS, GlusterFS, FUSE, remote* storage
- 19. © 2015 EnterpriseDB Corporation. All rights reserved. 19 File System and Storage Subsystem • Types of Storage Systems − Direct Attached Storage (DAS) − Storage Area Network (SAN) w. Fiber Channel − Storage Area Network (SAN) w. iSCSI − Network File System (NFS) • DAS: Fast and low latency • SAN: Fast, expensive (w. Fiber Channel), scalable, can include redundancy and smart file system operations • NFS: Not an optimal solution for Postgres
- 20. © 2015 EnterpriseDB Corporation. All rights reserved. 20 Your Application
- 21. © 2015 EnterpriseDB Corporation. All rights reserved. 21 • A few long-running connections crunching lots of data? − large work_mem − Emphasize effective_cache over shared_buffers • Lots of 'quick hit' connections? − connection pooling − larger shared_buffers • More read? Or more write? − autovacuum aggressiveness − Checkpoint spreading 3. What Does My App Want? © 2011 EnterpriseDB . All rights reserved.
- 22. © 2015 EnterpriseDB Corporation. All rights reserved. 22 • Can the app recover from a failed transaction? − During failover, for example • Can the app re-request if the answer isn't up-to-date? − Read-only slave, slightly behind the master • Can the app handle 'eventual consistency'? − Multi-master replication 4. Can My App Handle what the DB Offers? © 2011 EnterpriseDB . All rights reserved.
- 23. © 2015 EnterpriseDB Corporation. All rights reserved. 23 Availability
- 24. © 2015 EnterpriseDB Corporation. All rights reserved. 24 • Highly available − Enterprise Failover Manager (EFM) − xDB • Highly recoverable − Warm standby − WAL archiving − PITR • Geographically durable − xDB − Cascaded Replication 5. High Availability or High Recoverability? © 2011 EnterpriseDB . All rights reserved.
- 25. © 2015 EnterpriseDB Corporation. All rights reserved. 25 • ASYNC Replication − Default − Eventually consistent − May be 1+ transactions behind − Unlimited* number of ASYNC slaves • SYNC Replication − Atypical − Immediately consistent − Performance implications on the master − Only ever one SYNC slave 6. Is ASYNC Enough or Do I Need SYNC? © 2011 EnterpriseDB . All rights reserved.
- 26. © 2015 EnterpriseDB Corporation. All rights reserved. 26 • Cascaded − Can survive death of the master − May introduce replication lag • Fanned − Does not survive death of the master − No additional replication lag • Both − No reason you can't mix-n-match 7. Cascaded or Fanned Replication? © 2011 EnterpriseDB . All rights reserved. OR
- 27. © 2015 EnterpriseDB Corporation. All rights reserved. 27 Upgrades
- 28. © 2015 EnterpriseDB Corporation. All rights reserved. 28 Quick Poll Upgrade Policies
- 29. © 2015 EnterpriseDB Corporation. All rights reserved. 29 • Question: What is your current upgrade policy? • Answer choices: a. Fully defined patch mgmt. policy w/ standing maintenance windows b. We apply security fixes only as needed c. We apply patches/upgrades whenever the mood strikes us d. We apply patches/upgrades when the vendor forces it e. Upgrades? Audience Quick Poll #2
- 30. © 2015 EnterpriseDB Corporation. All rights reserved. 30 8. How Do I Upgrade? • Postgres version number: xx.yy.zz − xx.yy = major version (8.4, 9.2, 9.3) − zz = minor version (9.1.14, 9.3.5) • Current version: − 9.3.x → 9.3.5 − 9.2.x → 9.2.9 • Major version changes − Not backwards compatible for stored data − Dump/load − pg_upgrade • Minor version changes − Replace the binaries, restart
- 31. © 2015 EnterpriseDB Corporation. All rights reserved. 31 • Use replication ! • Upgrade the slave first − If a minor release upgrade, streaming replication − If a major release upgrade, logical replication • Failover to the upgraded slave • Upgrade the old master • Slave the old master to the new master • Fail back (optional) How Do I Upgrade (cont.)?
- 32. © 2015 EnterpriseDB Corporation. All rights reserved. 32 9. Why Upgrade? • Postgres minor releases fix only: − Frequently-encountered bugs − Security bugs − Data corruption bugs • As such, you are more at risk by not upgrading • Major versions are supported for 5 years which lessens the upgrade burden significantly
- 33. © 2015 EnterpriseDB Corporation. All rights reserved. 33 Recovery
- 34. © 2015 EnterpriseDB Corporation. All rights reserved. 34 • What’s the loss to the business per minute of the app being down? − High cost == design for high availability − Think multi-master replication − Think offsite replication − Think automatic failover • What’s the loss to the business for every transaction you can’t recover? − High cost == design for high recoverability − Think SYNC replication − Think continuous WAL archiving − Think outside the db (zfs snapshots, RHCS, Veritas clustering) 10. How Much Data Can I Afford to Lose?
- 35. © 2015 EnterpriseDB Corporation. All rights reserved. 35 • Backup and Recovery Strategies protect you in case of: − Catastrophic device failure − Site failure − Maintenance − Operator error − Compliance − Data corruption • Consider − Allowable PITR timeframe − Data retention policy − Test, test, test − Periodic backup validation Why Do I Need Backup and Recovery?
- 36. © 2015 EnterpriseDB Corporation. All rights reserved. 36 • Various recovery options are available: − PITR − BART / Barman − pg_basebackup − pg_dump • How long does restoration take? • Do I know how to restore using my chosen method? What Are My Recovery Options?
- 37. © 2015 EnterpriseDB Corporation. All rights reserved. 37 Dealing with Change – Ongoing Questions
- 38. © 2015 EnterpriseDB Corporation. All rights reserved. 38 • When things change − Database load − Hardware − Software versions − Business needs − User base grows And • At minimum, annually => Reconsider all questions Moving Forward
- 39. © 2015 EnterpriseDB Corporation. All rights reserved. 39 Summary and Useful Resources
- 40. © 2015 EnterpriseDB Corporation. All rights reserved. 40 • Preparation, Planning, and Postgres − Most problems occur after moving to production − Many times these problems were foreshadowed − Try to think of “what's next” as much as “what's now” • Communicate − Nothing exists in a vacuum − Seek input from other teams − Proactively “push” info to others • Evaluate, Adjust, and Repeat − Monitor all the things − Define your 'line in the sand' Summary
- 41. © 2015 EnterpriseDB Corporation. All rights reserved. 41 • Get the team ready for Postgres - Developer, DBA, Infrastructure • EDB Knowledge Base • Get Certified - http://www.enterprisedb.com/training • Follow the conversation: − #postgres on Twitter − @enterprisedb on Twitter − Planet PostgreSQL, @planetpostgres , http://planet.postgresql.org • User groups and Meetups – almost all major cities − http://www.postgresql.org/community/user-groups/ − http://postgresql.meetup.com/ What You Can Do
- 42. © 2015 EnterpriseDB Corporation. All rights reserved. 42 • Architectural Health Check − Review of database, OS, env. settings − Guidance on tuning, backup, high availability strategies • Kick Start − Training − Flex consulting − Developer subscription How We Can Help Whitepapers and Recorded Webinars http://www.enterprisedb.com/resources-community • Training and Certification − Flexible formats: on-demand, live virtual, on-site − Continuing education and Postgres certification • Remote DBA Services − 24 x 7 monitoring − Supplemental staff − Architectural health check − Access to Postgres expertise