Scaling with Postgres (Highload++ 2010)
0 likes1,376 views
The document discusses strategies for scaling PostgreSQL databases. It recommends using PostgreSQL version 8.3 or higher, implementing connection pooling, and setting up replication between a master and slave database. Monitoring and gaining visibility into database metrics and queries is also emphasized as important for capacity planning and performance tuning during scaling. The presenter advocates for a culture where application developers work closely with database administrators on schema design and queries.
1 of 40
Ad
Recommended
DownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
DownTheRabbitHole.js – How to Stay Sane in an Insane EcosystemFITC This document provides a history of JavaScript development from 1995 to the present. It describes how JavaScript evolved from a scripting language created in 10 days for Netscape (Mocha/LiveScript) to an industry standard (ECMAScript). It outlines major developments like Node.js, npm, and the rise of JavaScript modules/tooling. It recommends choosing technologies based on your specific needs rather than trends, investing in great tooling, and continuing to learn as the ecosystem rapidly changes.
Nuxeo World 2013] HOW TO CONTRIBUTE TO THE COMMUNITY - LAURENT DOGUIN
Nuxeo World 2013] HOW TO CONTRIBUTE TO THE COMMUNITY - LAURENT DOGUINNuxeo The document outlines how individuals can contribute to the Nuxeo community through various means such as feedback, bug fixing, translations, and open sourcing projects. It encourages participation and provides resources for getting involved, including contact information and links to relevant platforms. The overall message emphasizes the collaborative spirit of the community and the appreciation for contributors.
Less Alarming Alerts - SRECon 2016
Less Alarming Alerts - SRECon 2016 Robert Treat The document discusses improving alert systems by reducing unnecessary alerts. It suggests focusing on alerts that require human action, identifying the business impact of alerts, and documenting how each alert is addressed. Non-actionable alerts should be converted to notices or removed. Organizations should design resilience into their systems to minimize reliance on human intervention during incidents.
GeoServer on Steroids
GeoServer on Steroids GeoSolutions The document discusses various strategies to optimize the performance of GeoServer, covering topics such as preparing raster and vector data, optimizing styling, and configuring resource limits. It emphasizes the importance of data formats like GeoTIFF for raster data and advises on tuning for server performance, including JVM settings and connection pooling. Additionally, it highlights the benefits of tiling, caching, and utilizing vector tiles to enhance map rendering speed and efficiency.
Postgres Scaling Opportunities and Options
Postgres Scaling Opportunities and OptionsEDB The document discusses various scaling opportunities for PostgreSQL, highlighting methods for improving database performance under heavy workloads. It explores options such as vertical and horizontal scaling, multi-session and single-session scaling techniques, and the use of connection poolers and sharding. The presentation emphasizes the challenges of scaling while providing strategies for effective resource utilization and configuration adjustments.
What Ops Can Learn From Design
What Ops Can Learn From DesignRobert Treat The document discusses the principles of design and its impact on usability, emphasizing the importance of understanding natural mapping, perceived affordances, and feedback mechanisms in user experience. It argues that many incidents attributed to human error are often rooted in poor design, suggesting that a design-centered approach can enhance system functionality and user interaction. The speaker, Robert Treat, combines insights from web development and design to illustrate how effective design can lead to clearer and more efficient processes.
Beyond Commodities - Gulf investors and the new Africa
Beyond Commodities - Gulf investors and the new AfricaJoannes Mongardini The document examines Sub-Saharan Africa's growth outside of natural resources and commodities, and the role of Gulf investors. It finds that Africa has proven more resilient to global headwinds than expected, supported by demographic trends, a growing middle class, and economic reforms. East Africa is emerging as the most appealing region for non-commodity Gulf investment, particularly in manufacturing, retail, tourism, and education. Gulf investors have potential options like co-investing with private equity or direct acquisitions. Retail, tourism, and improving logistics networks represent opportunities, but challenges remain in some sectors and countries.
Reactive Navigation of Autonomous Mobile Robot Using Neuro-Fuzzy System
Reactive Navigation of Autonomous Mobile Robot Using Neuro-Fuzzy SystemWaqas Tariq The document presents a neuro-fuzzy system for the reactive navigation of autonomous mobile robots, emphasizing the ability to navigate in unknown environments using human-like expertise. The proposed algorithm integrates neural networks for environmental sensing and fuzzy logic for behavioral control, demonstrating effectiveness in various obstacle scenarios through simulation results. The paper discusses the advantages of this approach over traditional neural and fuzzy methods, addressing challenges in robot navigation such as sensor data integration and obstacle avoidance.
Ping pong & tennis.
Ping pong & tennis.BentonGeometry This document compares the dimensions of a ping pong table and tennis court. A ping pong table measures 5 feet by 9 feet while a tennis court is 27 feet by 78 feet. These dimensions are not proportional as dividing the tennis court measurements by the ping pong table measurements results in ratios of 1:5.4 for width and 1:8.6 for length.
Annual_Report_2015_04_Final_Single-Page
Annual_Report_2015_04_Final_Single-PageHeather Thomas The document provides information on the 2014 Public Health Advisory Council and Board of Health for Snohomish County, Washington. It lists the members of each group and notes that the Snohomish Health District works with these partners to promote public health in the community. The document then summarizes the Health District's strategic plan update and priorities for evolving its programs and services. It also provides some statistics on the services provided and funding challenges faced by the Health District.
Getting your name out there
Getting your name out thereAlex Cato The document discusses the importance of building a professional brand, emphasizing personal authenticity and continuous learning. It outlines key considerations for online presence, including attractiveness, desired messaging, and connectivity. Additionally, it encourages face-to-face networking and provides social media examples for building connections.
Dr ravi r kasliwal cardiologist medanta medicity
Dr ravi r kasliwal cardiologist medanta medicityDr R R Kasliwal Dr. R. R. Kasliwal is a leading cardiologist with over 30 years of experience, recognized for his contributions to cardiovascular sciences and community outreach programs in India and beyond. He holds multiple qualifications, has received numerous awards, and has published extensively on cardiology topics while actively teaching and nurturing future cardiologists. His work emphasizes the use of non-invasive techniques for early disease detection and he has been instrumental in expanding cardiac care to the underprivileged.
Less Alarming Alerts!
Less Alarming Alerts!Robert Treat The document discusses the importance of effective alert systems in managing operational infrastructure, emphasizing the need to focus on actionable alerts with business impact rather than overwhelming users with numerous false alarms. It encourages a shift towards understanding alerts' remediation and promotes knowledge sharing within teams to identify gaps and improve response strategies. Ultimately, it advocates for designing resilient systems that minimize reliance on human intervention during off-hours.
Managing Databases In A DevOps Environment 2016
Managing Databases In A DevOps Environment 2016Robert Treat The document discusses managing databases in a DevOps environment, emphasizing the importance of configuration management, monitoring, and deployment challenges specific to databases. It highlights the need for effective communication between developers and DBAs to facilitate changes while minimizing risks. Additionally, the author advocates for incremental changes and collaboration as key strategies for successful database management within a DevOps culture.
Postgres 9.4 First Look
Postgres 9.4 First LookRobert Treat The document discusses upcoming features in Postgres 9.4 including:
- Performance improvements such as reduced WAL size for updates, improved GIN index performance, and ability to separate query planning and execution times.
- Administration enhancements like ALTER SYSTEM for configuration changes, improved EXPLAIN output, and a new pg_stat_archiver view.
- New SQL functions like MAKE_TIMESTAMP and ordered-set aggregates.
- Backend changes like PL/pgSQL stack traces, concurrent REFRESH for materialized views, and the new JSONB binary format.
- Future directions include additional performance work and SQL features.
Managing Databases In A DevOps Environment
Managing Databases In A DevOps EnvironmentRobert Treat The document discusses the integration of database management within a DevOps environment, emphasizing the importance of configuration management and monitoring to manage significant data and transactional operations. It highlights the challenges faced by DBAs, including managing changes and ensuring data integrity while collaborating with developers. The author advocates for a new collaborative approach to database administration, positioning DBAs as essential partners in the DevOps pipeline.
Modern Monitoring - devopsdays Cuba
Modern Monitoring - devopsdays Cubabridgetkromhout This document discusses monitoring in the context of containers and microservices. It provides an overview of the history and evolution of containers from chroot and jails to Docker. It outlines different approaches to monitoring from manual to reactive to proactive. It discusses challenges of monitoring ephemeral infrastructure and the need for automation. It also covers architectural considerations for monitoring containers and microservices as well as specific techniques for collecting events, logs and metrics from containerized applications.
Scaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset The document discusses the use of Celery, an asynchronous task queue, for managing task processing in Python applications. It outlines the benefits of using task queues, provides code examples for implementing tasks, and shares ten key lessons learned in scaling and optimizing Celery. Additionally, it covers best practices for task management, monitoring, and system architecture considerations.
Salam_rahal_CV
Salam_rahal_CVSalam Rahal Salam Z. Rahal is seeking an administrative assistant position utilizing over 15 years of experience. She has held roles with Seedbound, USAgriseeds Co., United Towers Holding Company, KIPCO Asset Management Company, Kuwait Real Estate Investment Consortium, and Kuwaiti Interest For Financial Investment. Rahal has a technical diploma in computer programming and is proficient in Microsoft Office, with strengths in working under pressure and as part of a team.
rough draft sketches
rough draft sketchesbano2630 This magazine document provides information on health, fitness, weight loss, and beauty topics. It promotes detox boot camps, strategies for getting fit on a budget, and ways to lose 10 pounds in 4 weeks through diet and exercise. Additionally, it shares instant beauty tricks and tips for eating and drinking while still losing weight through motivation and lifestyle changes rather than dieting.
Scaling with Postgres (Robert Treat)
Scaling with Postgres (Robert Treat)Ontico Robert Treat presented on scaling Postgres databases. He emphasized establishing a culture where developers own database schemas and queries. It is important to gain visibility into the system through monitoring, alerts, trending and performance tuning. Using the latest Postgres version, replication, and connection pooling can help with scaling. Developers, database administrators, and operations teams should collaborate closely.
Check Please!
Check Please!Robert Treat This document discusses the importance of monitoring a Postgres database to prevent issues and ensure high performance. It provides examples of key things to monitor such as connections, disk space, transactions, queries, and settings. Regular monitoring helps identify problems early before they seriously impact the database or lead to data loss or downtime. Tools can help support monitoring but experience is also important.
Top 10 Tips for an Effective Postgres Deployment
Top 10 Tips for an Effective Postgres DeploymentEDB The document provides a comprehensive guide for effective PostgreSQL deployment, covering essential areas such as preparation, planning, questions to ask during deployment, uptime requirements, backups, and ongoing evaluation. It includes insights on hardware requirements, replication strategies, upgrade policies, and recovery options, emphasizing the importance of communication among teams and continuous adjustments. EDB's expertise and resources are highlighted, along with customer satisfaction regarding their PostgreSQL solutions.
Top10 list planningpostgresdeployment.2014
Top10 list planningpostgresdeployment.2014EDB The document outlines key considerations and questions for effectively planning a PostgreSQL deployment, focusing on preparation, planning, and adaptation to changes post-deployment. It emphasizes the importance of infrastructure, application needs, and ongoing evaluation to ensure reliability and performance. Additionally, it highlights resources for training and ongoing support from EnterpriseDB.
Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA EDB The document outlines best practices for PostgreSQL Database Administrators (DBAs), focusing on monitoring, maintenance, capacity planning, and configuration tuning. It emphasizes the importance of architectural health checks, performance tuning, and effective backup strategies to ensure database reliability and efficiency. Additionally, it shares insights from customer case studies and offers recommendations for addressing common issues such as performance problems and replication failures.
Postgres in Production - Best Practices 2014
Postgres in Production - Best Practices 2014EDB The document discusses best practices in PostgreSQL usage, focusing on user challenges in adoption, development, deployment, and maintenance, while providing insights into tuning parameters and backup strategies. It emphasizes the necessity for PostgreSQL users to enhance their product knowledge throughout the database lifecycle and offers recommendations for performance and scalability improvements. Additionally, it highlights the importance of effective backup and recovery strategies to safeguard against data loss and operational issues.
Postgres in production.2014
Postgres in production.2014EDB The document outlines best practices for deploying and maintaining PostgreSQL in enterprise environments, highlighting key challenges, tuning parameters, and maintenance strategies. It emphasizes the importance of backup and recovery plans, as well as the need for PostgreSQL users to enhance their understanding of the database lifecycle. The document also provides insights from support ticket analyses to improve user success with PostgreSQL.
Pro PostgreSQL, OSCon 2008
Pro PostgreSQL, OSCon 2008Robert Treat This document provides an overview and introduction to PostgreSQL for new users. It covers getting started with PostgreSQL, including installing it, configuring authentication and logging, upgrading to new versions, routine maintenance tasks, hardware recommendations, availability and scalability options, and query tuning and optimization. The document is presented as a slide deck with different sections labeled by letters (e.g. K-0, S-0, U-0).
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan PachenkoEqunix Business Solutions Ivan Panchenko discusses the evolution and challenges of PostgreSQL in Russia, highlighting its prominence as the leading database since 1999. The document outlines the growth of the PostgreSQL community, key advancements, and ongoing issues such as horizontal scalability and performance optimization. It emphasizes the need for addressing challenges to ensure PostgreSQL remains robust in industrial applications and developer support.
More Related Content
Viewers also liked (13)
Ping pong & tennis.
Ping pong & tennis.BentonGeometry This document compares the dimensions of a ping pong table and tennis court. A ping pong table measures 5 feet by 9 feet while a tennis court is 27 feet by 78 feet. These dimensions are not proportional as dividing the tennis court measurements by the ping pong table measurements results in ratios of 1:5.4 for width and 1:8.6 for length.
Annual_Report_2015_04_Final_Single-Page
Annual_Report_2015_04_Final_Single-PageHeather Thomas The document provides information on the 2014 Public Health Advisory Council and Board of Health for Snohomish County, Washington. It lists the members of each group and notes that the Snohomish Health District works with these partners to promote public health in the community. The document then summarizes the Health District's strategic plan update and priorities for evolving its programs and services. It also provides some statistics on the services provided and funding challenges faced by the Health District.
Getting your name out there
Getting your name out thereAlex Cato The document discusses the importance of building a professional brand, emphasizing personal authenticity and continuous learning. It outlines key considerations for online presence, including attractiveness, desired messaging, and connectivity. Additionally, it encourages face-to-face networking and provides social media examples for building connections.
Dr ravi r kasliwal cardiologist medanta medicity
Dr ravi r kasliwal cardiologist medanta medicityDr R R Kasliwal Dr. R. R. Kasliwal is a leading cardiologist with over 30 years of experience, recognized for his contributions to cardiovascular sciences and community outreach programs in India and beyond. He holds multiple qualifications, has received numerous awards, and has published extensively on cardiology topics while actively teaching and nurturing future cardiologists. His work emphasizes the use of non-invasive techniques for early disease detection and he has been instrumental in expanding cardiac care to the underprivileged.
Less Alarming Alerts!
Less Alarming Alerts!Robert Treat The document discusses the importance of effective alert systems in managing operational infrastructure, emphasizing the need to focus on actionable alerts with business impact rather than overwhelming users with numerous false alarms. It encourages a shift towards understanding alerts' remediation and promotes knowledge sharing within teams to identify gaps and improve response strategies. Ultimately, it advocates for designing resilient systems that minimize reliance on human intervention during off-hours.
Managing Databases In A DevOps Environment 2016
Managing Databases In A DevOps Environment 2016Robert Treat The document discusses managing databases in a DevOps environment, emphasizing the importance of configuration management, monitoring, and deployment challenges specific to databases. It highlights the need for effective communication between developers and DBAs to facilitate changes while minimizing risks. Additionally, the author advocates for incremental changes and collaboration as key strategies for successful database management within a DevOps culture.
Postgres 9.4 First Look
Postgres 9.4 First LookRobert Treat The document discusses upcoming features in Postgres 9.4 including:
- Performance improvements such as reduced WAL size for updates, improved GIN index performance, and ability to separate query planning and execution times.
- Administration enhancements like ALTER SYSTEM for configuration changes, improved EXPLAIN output, and a new pg_stat_archiver view.
- New SQL functions like MAKE_TIMESTAMP and ordered-set aggregates.
- Backend changes like PL/pgSQL stack traces, concurrent REFRESH for materialized views, and the new JSONB binary format.
- Future directions include additional performance work and SQL features.
Managing Databases In A DevOps Environment
Managing Databases In A DevOps EnvironmentRobert Treat The document discusses the integration of database management within a DevOps environment, emphasizing the importance of configuration management and monitoring to manage significant data and transactional operations. It highlights the challenges faced by DBAs, including managing changes and ensuring data integrity while collaborating with developers. The author advocates for a new collaborative approach to database administration, positioning DBAs as essential partners in the DevOps pipeline.
Modern Monitoring - devopsdays Cuba
Modern Monitoring - devopsdays Cubabridgetkromhout This document discusses monitoring in the context of containers and microservices. It provides an overview of the history and evolution of containers from chroot and jails to Docker. It outlines different approaches to monitoring from manual to reactive to proactive. It discusses challenges of monitoring ephemeral infrastructure and the need for automation. It also covers architectural considerations for monitoring containers and microservices as well as specific techniques for collecting events, logs and metrics from containerized applications.
Scaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset The document discusses the use of Celery, an asynchronous task queue, for managing task processing in Python applications. It outlines the benefits of using task queues, provides code examples for implementing tasks, and shares ten key lessons learned in scaling and optimizing Celery. Additionally, it covers best practices for task management, monitoring, and system architecture considerations.
Salam_rahal_CV
Salam_rahal_CVSalam Rahal Salam Z. Rahal is seeking an administrative assistant position utilizing over 15 years of experience. She has held roles with Seedbound, USAgriseeds Co., United Towers Holding Company, KIPCO Asset Management Company, Kuwait Real Estate Investment Consortium, and Kuwaiti Interest For Financial Investment. Rahal has a technical diploma in computer programming and is proficient in Microsoft Office, with strengths in working under pressure and as part of a team.
rough draft sketches
rough draft sketchesbano2630 This magazine document provides information on health, fitness, weight loss, and beauty topics. It promotes detox boot camps, strategies for getting fit on a budget, and ways to lose 10 pounds in 4 weeks through diet and exercise. Additionally, it shares instant beauty tricks and tips for eating and drinking while still losing weight through motivation and lifestyle changes rather than dieting.
Similar to Scaling with Postgres (Highload++ 2010) (20)
Scaling with Postgres (Robert Treat)
Scaling with Postgres (Robert Treat)Ontico Robert Treat presented on scaling Postgres databases. He emphasized establishing a culture where developers own database schemas and queries. It is important to gain visibility into the system through monitoring, alerts, trending and performance tuning. Using the latest Postgres version, replication, and connection pooling can help with scaling. Developers, database administrators, and operations teams should collaborate closely.
Check Please!
Check Please!Robert Treat This document discusses the importance of monitoring a Postgres database to prevent issues and ensure high performance. It provides examples of key things to monitor such as connections, disk space, transactions, queries, and settings. Regular monitoring helps identify problems early before they seriously impact the database or lead to data loss or downtime. Tools can help support monitoring but experience is also important.
Top 10 Tips for an Effective Postgres Deployment
Top 10 Tips for an Effective Postgres DeploymentEDB The document provides a comprehensive guide for effective PostgreSQL deployment, covering essential areas such as preparation, planning, questions to ask during deployment, uptime requirements, backups, and ongoing evaluation. It includes insights on hardware requirements, replication strategies, upgrade policies, and recovery options, emphasizing the importance of communication among teams and continuous adjustments. EDB's expertise and resources are highlighted, along with customer satisfaction regarding their PostgreSQL solutions.
Top10 list planningpostgresdeployment.2014
Top10 list planningpostgresdeployment.2014EDB The document outlines key considerations and questions for effectively planning a PostgreSQL deployment, focusing on preparation, planning, and adaptation to changes post-deployment. It emphasizes the importance of infrastructure, application needs, and ongoing evaluation to ensure reliability and performance. Additionally, it highlights resources for training and ongoing support from EnterpriseDB.
Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA EDB The document outlines best practices for PostgreSQL Database Administrators (DBAs), focusing on monitoring, maintenance, capacity planning, and configuration tuning. It emphasizes the importance of architectural health checks, performance tuning, and effective backup strategies to ensure database reliability and efficiency. Additionally, it shares insights from customer case studies and offers recommendations for addressing common issues such as performance problems and replication failures.
Postgres in Production - Best Practices 2014
Postgres in Production - Best Practices 2014EDB The document discusses best practices in PostgreSQL usage, focusing on user challenges in adoption, development, deployment, and maintenance, while providing insights into tuning parameters and backup strategies. It emphasizes the necessity for PostgreSQL users to enhance their product knowledge throughout the database lifecycle and offers recommendations for performance and scalability improvements. Additionally, it highlights the importance of effective backup and recovery strategies to safeguard against data loss and operational issues.
Postgres in production.2014
Postgres in production.2014EDB The document outlines best practices for deploying and maintaining PostgreSQL in enterprise environments, highlighting key challenges, tuning parameters, and maintenance strategies. It emphasizes the importance of backup and recovery plans, as well as the need for PostgreSQL users to enhance their understanding of the database lifecycle. The document also provides insights from support ticket analyses to improve user success with PostgreSQL.
Pro PostgreSQL, OSCon 2008
Pro PostgreSQL, OSCon 2008Robert Treat This document provides an overview and introduction to PostgreSQL for new users. It covers getting started with PostgreSQL, including installing it, configuring authentication and logging, upgrading to new versions, routine maintenance tasks, hardware recommendations, availability and scalability options, and query tuning and optimization. The document is presented as a slide deck with different sections labeled by letters (e.g. K-0, S-0, U-0).
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan PachenkoEqunix Business Solutions Ivan Panchenko discusses the evolution and challenges of PostgreSQL in Russia, highlighting its prominence as the leading database since 1999. The document outlines the growth of the PostgreSQL community, key advancements, and ongoing issues such as horizontal scalability and performance optimization. It emphasizes the need for addressing challenges to ensure PostgreSQL remains robust in industrial applications and developer support.
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch ElizabethGarrettChri The document outlines essential responsibilities and considerations for a PostgreSQL Database Administrator (DBA), including data availability, security, backup strategies, and performance tuning. It discusses various hosting options, resource sizing, and the importance of high availability, alongside practical advice for monitoring and troubleshooting PostgreSQL. Key topics include memory configuration, connection management, and the use of tools for query performance analysis.
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya KosmodemianskyPostgreSQL-Consulting The document presents a humorous take on PostgreSQL 'worst practices' as discussed by Ilya Kosmodemiansky during the FOSDEM PGDay 2017. It lists over 100 counterintuitive recommendations aimed at demonstrating how to effectively disrupt database management and performance, covering topics such as avoiding indexes, mishandling data replication, and disregarding schema design. The talk emphasizes the irony of these practices while encouraging engagement through questions and contributions from the audience.
PostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized WorldJignesh Shah This document discusses PostgreSQL high availability in a containerized environment. It begins with an overview of containers and their advantages like lower footprint and density. It then covers enterprise needs for high availability like recovery time objectives. Common approaches to PostgreSQL high availability are discussed like replication, shared storage, and using projects like Patroni and Stolon. Modern trends with containers are highlighted like separating data and binaries. Kubernetes is presented as a production-grade orchestrator that can provide horizontal scaling and self-healing capabilities. The discussion concludes with challenges of multi-region deployments and how service discovery with Consul can help address those challenges.
Best Practices & Lessons Learned from Deployment of PostgreSQL
Best Practices & Lessons Learned from Deployment of PostgreSQLEDB The document discusses best practices and lessons learned from the deployment of PostgreSQL, emphasizing the importance of understanding application landscapes and fostering collaboration for effective adoption. It highlights the increasing trend of organizations migrating to open source databases like PostgreSQL and provides recommendations for successful deployment, including automation and standardization. The document also outlines various deployment architectures and the significance of creating a supportive culture between developers and database administrators to enable effective use of PostgreSQL.
Webinar: ROI on Big Data - RDBMS, NoSQL or Both? A Simple Guide for Knowing H...
Webinar: ROI on Big Data - RDBMS, NoSQL or Both? A Simple Guide for Knowing H...DataStax The document discusses the advantages of NoSQL databases, particularly Apache Cassandra and Datastax Enterprise, over traditional RDBMS systems, emphasizing their suitability for modern big data applications. It highlights the need for flexibility, scalability, and continuous availability in today's data-driven environments, as well as the growing adoption of NoSQL solutions by major companies. The summary also compares the functionalities and use cases of RDBMS and NoSQL systems, providing insights into strategic decisions for businesses considering the transition.
No sql findings
No sql findingsChristian van der Leeden The document discusses evaluating different NoSQL databases to replace a PostgreSQL database that is expected to become a performance bottleneck due to high volume growth that is unpredictable and non-linear. It summarizes the criteria for the NoSQL solution, including the ability to scale horizontally, good performance for Ruby, and active development. Several NoSQL databases are evaluated, including MongoDB, Redis, Cassandra, HBase, and Membase. MongoDB, Redis, and Cassandra are discussed in more detail regarding experiences and performance.
EnterpriseDB's Best Practices for Postgres DBAs
EnterpriseDB's Best Practices for Postgres DBAsEDB This document provides an agenda and overview for a presentation on best practices for PostgreSQL database administrators (DBAs). The presentation covers EnterpriseDB's expertise in PostgreSQL, the key responsibilities of a PostgreSQL DBA including monitoring, maintenance, capacity planning and configuration tuning. It also discusses deployment planning, professional development resources, and takes questions. Examples from architectural health checks and remote DBA services illustrate common issues found like index bloat and lack of backups. The document recommends performance monitoring and security tools and techniques for PostgreSQL.
Tales from the Postgres Front - and What We Can Learn
Tales from the Postgres Front - and What We Can LearnEDB The document presents an overview of Enterprisedb's PostgreSQL offerings, focusing on migration experiences, customer support trends, and common issues facing users. It highlights the growing adoption of PostgreSQL in complex applications and identifies support ticket trends, emphasizing the need for targeted education and help for non-traditional PostgreSQL users. The findings aim to guide product development and enhance customer experience by addressing the most common user challenges.
The Accidental DBA
The Accidental DBAPostgreSQL Experts, Inc. This document provides a summary of a presentation on becoming an accidental PostgreSQL database administrator (DBA). It covers topics like installation, configuration, connections, backups, monitoring, slow queries, and getting help. The presentation aims to help those suddenly tasked with DBA responsibilities to not panic and provides practical advice on managing a PostgreSQL database.
5 Steps to PostgreSQL Performance
5 Steps to PostgreSQL PerformanceCommand Prompt., Inc This document provides an overview of five steps to improve PostgreSQL performance: 1) hardware optimization, 2) operating system and filesystem tuning, 3) configuration of postgresql.conf parameters, 4) application design considerations, and 5) query tuning. The document discusses various techniques for each step such as selecting appropriate hardware components, spreading database files across multiple disks or arrays, adjusting memory and disk configuration parameters, designing schemas and queries efficiently, and leveraging caching strategies.
Five steps perform_2009 (1)
Five steps perform_2009 (1)PostgreSQL Experts, Inc. This document provides an overview of five steps to improve PostgreSQL performance: 1) hardware optimization, 2) operating system and filesystem tuning, 3) configuration of postgresql.conf parameters, 4) application design considerations, and 5) query tuning. The document discusses various techniques for each step such as selecting appropriate hardware components, spreading database files across multiple disks or arrays, adjusting memory parameters, effective indexing, and caching queries and data.
Ad
More from Robert Treat (19)
Advanced Int->Bigint Conversions
Advanced Int->Bigint ConversionsRobert Treat The document discusses advanced integer-to-bigint conversions in PostgreSQL, highlighting issues related to integer overflow and the use of serial and bigserial data types for surrogate primary keys. It explains various strategies to manage and prevent overflow errors, including using bigint where necessary, the implications of foreign keys, and has practical examples involving SQL commands. Additionally, it emphasizes the need for careful monitoring and management in complex distributed systems to avoid production failures caused by integer-related issues.
Explaining Explain
Explaining ExplainRobert Treat The document explains the use of the EXPLAIN command in PostgreSQL for analyzing query execution plans, which includes estimated costs, row counts, and widths. It describes components such as sequential scans, cost estimation, and the importance of using EXPLAIN ANALYZE for actual performance metrics. Additionally, it covers cost calculations for various scenarios and provides examples of query plans and their implications on performance.
the-lost-art-of-plpgsql
the-lost-art-of-plpgsqlRobert Treat The document is a presentation by Robert Treat on PL/pgSQL, focusing on its usage for creating stored procedures and user-defined functions in PostgreSQL. It provides an overview of PL/pgSQL features, including examples of queries and functions, and discusses the differences between functions and stored procedures. The presentation emphasizes the utility of PL/pgSQL in server-side computation and its integration with PostgreSQL's features.
Managing Chaos In Production: Testing vs Monitoring
Managing Chaos In Production: Testing vs MonitoringRobert Treat This document discusses the trade-offs between testing and monitoring in a DevOps lifecycle. It argues that while testing is important and can find many issues, it is not enough on its own as systems are complex with many unknown unknowns. Monitoring is needed to catch issues that occur in production. The key aspects of monitoring discussed are focusing on monitoring metrics that affect business outcomes rather than just technical metrics, taking a top-down approach to understand the business and what to monitor, and increasing overall observability into systems. Both testing and monitoring are important, with their combined use providing better confidence in systems than either approach alone.
Past, Present, and Pachyderm - All Things Open - 2013
Past, Present, and Pachyderm - All Things Open - 2013Robert Treat The document outlines key developments in PostgreSQL from its early versions to the latest updates, including features and improvements introduced in version 9.3 such as materialized views, updatable views, and JSON support. It also touches on market trends in cloud computing, NoSQL, and big data, highlighting the operational challenges and innovations in managing large datasets. Additionally, there is a call for continued focus on fault tolerance and analytics in future PostgreSQL developments.
Big Bad "Upgraded" Postgres
Big Bad "Upgraded" PostgresRobert Treat The presentation by Robert Treat discusses the experiences and challenges faced during the upgrade of Postgres databases at Omniti, which involved significant trial and error, collaboration, and the need to address various issues such as disk failures and software bugs. Key takeaways include cost savings from migrating away from Oracle, the complexities of database upgrades, encountering bugs in tools like pg_upgrade, and ultimately the importance of testing and verification in database management. The journey showcases both the technical aspects and the operational considerations of managing large-scale database systems.
The Essential PostgreSQL.conf
The Essential PostgreSQL.confRobert Treat The document outlines essential parameters and settings for configuring PostgreSQL's postgresql.conf file, including types of settings, when they take effect, and common 'gotchas'. Key topics include memory sizing, logging, connection management, checkpoint and vacuum management, and WAL management for performance and replication. The presentation emphasizes best practices for achieving optimal database performance and provides additional resources for further learning.
Pro Postgres 9
Pro Postgres 9Robert Treat The document discusses PostgreSQL version 9 and provides information about upgrading PostgreSQL. It recommends making backups before upgrading, reading release notes and commit logs, and testing upgrades. It describes using pg_dump/pg_restore for simple upgrades but notes potential issues with time and disk usage. It also mentions using a replication-based "Slony method" which is more complex and has potential compatibility issues.
Advanced WAL File Management With OmniPITR
Advanced WAL File Management With OmniPITRRobert Treat This document provides an overview and examples of using OmniPITR, an open source tool for managing PostgreSQL WAL archiving and point-in-time recovery (PITR). It describes how OmniPITR consolidates and improves upon earlier scripts for archiving WAL files to multiple destinations, and allows for reusable, complete PITR solutions. Examples are given of using OmniPITR-Archive to configure archiving of WAL files with options like compression, custom logging and multiple archiving destinations. OmniPITR-Restore is also briefly mentioned as providing options for restoring from archived WAL files.
Intro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialRobert Treat The document discusses new features and improvements in PostgreSQL 9.0, including performance enhancements like improved VACUUM FULL and index usage, and new administration features like granting permissions on all tables in a schema at once and improved initialization process using pg_ctl initdb. It also covers areas like development, procedures, and replication.
Database Scalability Patterns
Database Scalability PatternsRobert Treat The document discusses various database scalability patterns, including vertical and horizontal scaling, read slaves, and multi-master configurations. It emphasizes the complexities and challenges of scaling, particularly focusing on write operations and the need for application changes. The significance of vertical scalability in both horizontally scaled systems and the costs associated with adding new nodes are also highlighted.
A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0Robert Treat The document is a detailed presentation on PostgreSQL 9.0, covering its performance enhancements, administrative changes, and developmental features. Key topics include improved indexing, vacuum operations, and new functionalities for managing database privileges and schemas. It also highlights the timeline of releases, including new feature patches and community contributions.
Scaling With Postgres
Scaling With PostgresRobert Treat This document discusses various strategies for scaling PostgreSQL databases, including implementation tuning, optimizing queries and schemas, caching, replication, and federation. The key points are that PostgreSQL scales well vertically by adding hardware resources like memory, CPUs and disks, and optimization efforts should focus on resolving underlying data and schema issues rather than being an end solution. Horizontal scaling using replication and federation can also help scale reads and writes.
Intro to Postgres 8.4 Tutorial
Intro to Postgres 8.4 TutorialRobert Treat This document provides a detailed overview of PostgreSQL 8.4, including installation instructions, performance improvements, and new administrative features. Key updates include enhancements to vacuum performance, column-level privileges, and new modules for tracking statistics and query planning. The document serves as a guide for users to leverage the capabilities of PostgreSQL 8.4 effectively.
Intro to pl/PHP Oscon2007
Intro to pl/PHP Oscon2007Robert Treat The document provides an introduction to pl/php, a procedural language based on PHP for database programming within PostgreSQL. It describes its installation prerequisites, usage examples, and the benefits of integrating closely with PostgreSQL, such as reducing network traffic. Additionally, the document outlines the installation process and some basic functions, highlighting the distinction between trusted and untrusted versions of the language.
The Essential postgresql.conf
The Essential postgresql.confRobert Treat This document discusses the essential parameters of postgresql.conf and their configurations for optimizing PostgreSQL performance. Key topics include types of settings, when they take effect, and specific configurations for connection management, logging, memory sizing, Write Ahead Logging (WAL) management, checkpoint management, and vacuum management. Additional details are provided on best practices, potential pitfalls, and external resources for PostgreSQL tuning.
PostgreSQL Partitioning, PGCon 2007
PostgreSQL Partitioning, PGCon 2007Robert Treat The document discusses PostgreSQL table partitioning, including what it is, how it is implemented using rules, constraints and inheritance, when to use triggers for partitioning, and automated partition maintenance. It notes that partitioning provides performance benefits as queries on large tables slow down, manageability benefits as only relevant partitions need to be queried, and scalability benefits. The key topics covered are performance, scalability, and manageability advantages of PostgreSQL partitioning.
Database Anti Patterns
Database Anti PatternsRobert Treat This document provides an overview of database anti-patterns including:
- The entity-attribute-value pattern, which stores data in a type-value format and cannot efficiently model constraints. Other options include seeking proper relational models or a "poor man's EAV" approach.
- Materialized path trees, which reference the full parent path for each child and violate normalization rules but allow retrieving the entire tree or subtrees with a single query. Moving subtrees only requires updating the path column.
- Surrogate keys, which are artificially added unique identifiers that can make queries harder to read but help ensure primary keys remain immutable.
Pro PostgreSQL
Pro PostgreSQLRobert Treat This document provides an overview of PostgreSQL topics including:
- Installation and configuration best practices such as using package management and configuring logging
- Routine maintenance activities like vacuuming and backups
- Upgrades and the differences between major, minor, and bugfix versions
- Advanced SQL topics like window functions, common table expressions, and querying slow queries
Ad
Recently uploaded (20)
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025OpenACC The OpenACC organization focuses on enhancing parallel computing skills and advancing interoperability in scientific applications through hackathons and training. The upcoming 2025 Open Accelerated Computing Summit (OACS) aims to explore the convergence of AI and HPC in scientific computing and foster knowledge sharing. This year's OACS welcomes talk submissions from a variety of topics, from Using Standard Language Parallelism to Computer Vision Applications. The document also highlights several open hackathons, a call to apply for NVIDIA Academic Grant Program and resources for optimizing scientific applications using OpenACC directives.
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdfEarley Information Science As AI reshapes expectations in retail and B2B commerce, organizations are recognizing a critical reality: meaningful AI outcomes depend on well-structured, adaptable infrastructure. In this session, Seth Earley is joined by Phil Ryan - AI strategist, search technologist, and founder of Glass Leopard Technologies - for a candid conversation on what it truly means to engineer systems for scale, agility, and intelligence.
Phil draws on more than two decades of experience leading search and AI initiatives for enterprise organizations. Together, he and Seth explore the challenges businesses face when legacy architectures limit personalization, agility, and real-time decisioning - and what needs to change to support agentic technologies and next-best-action capabilities.
Key themes from the webinar include:
Composability as a prerequisite for AI - Why modular, loosely coupled systems are essential for adapting to rapid innovation and evolving business needs
Search and relevance as foundational to AI - How techniques honed-in enterprise search have laid the groundwork for more responsive and intelligent customer experiences
From MDM and CDP to agentic systems - How data platforms are evolving to support richer customer context and dynamic orchestration
Engineering for business alignment - Why successful AI programs require architectural decisions grounded in measurable outcomes
The conversation is practical and forward-looking, connecting deep technical understanding with real-world business needs. Whether you’re modernizing your commerce stack or exploring how AI can enhance product discovery, personalization, or customer journeys, this session provides a clear-eyed view of the capabilities, constraints, and priorities that matter most.
Mastering AI Workflows with FME by Mark Döring
Mastering AI Workflows with FME by Mark DöringSafe Software Harness the full potential of AI with FME: From creating high-quality training data to optimizing models and utilizing results, FME supports every step of your AI workflow. Seamlessly integrate a wide range of models, including those for data enhancement, forecasting, image and object recognition, and large language models. Customize AI models to meet your exact needs with FME’s powerful tools for training, optimization, and seamless integration
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's SeatSafe Software Finding a specific car online can be a time-consuming task, especially when checking multiple dealer websites. A few years ago, I faced this exact problem while searching for a particular vehicle in New Zealand. The local classified platform, Trade Me (similar to eBay), wasn’t yielding any results, so I expanded my search to second-hand dealer sites—only to realise that periodically checking each one was going to be tedious. That’s when I noticed something interesting: many of these websites used the same platform to manage their inventories. Recognising this, I reverse-engineered the platform’s structure and built an FME workspace that automated the search process for me. By integrating API calls and setting up periodic checks, I received real-time email alerts when matching cars were listed. In this presentation, I’ll walk through how I used FME to save hours of manual searching by creating a custom car-finding automation system. While FME can’t buy a car for you—yet—it can certainly help you find the one you’re after!
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdfNilesh Gule Slide deck related to the GitHub Copilot Bootcamp in Melbourne on 17 June 2025
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core InnovationsIBM penPOWER offers a fully open, royalty-free CPU architecture for custom chip design.
It enables both lightweight FPGA cores (like Microwatt) and high-performance processors (like POWER10).
Developers have full access to source code, specs, and tools for end-to-end chip creation.
It supports AI, HPC, cloud, and embedded workloads with proven performance.
Backed by a global community, it fosters innovation, education, and collaboration.
10 Key Challenges for AI within the EU Data Protection Framework.pdf
10 Key Challenges for AI within the EU Data Protection Framework.pdfPriyanka Aash 10 Key Challenges for AI within the EU Data Protection Framework
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...Fwdays What can go wrong if you allow each service to access the database directly? In a startup, this seems like a quick and easy solution, but as the system scales, problems appear that no one could have guessed.
In my talk, I'll share Solidgate's experience in transforming its architecture: from the chaos of direct connections to a service-based data access model. I will talk about the transition stages, bottlenecks, and how isolation affected infrastructure support. I will honestly show what worked and what didn't. In short, we will analyze the controversy of this talk.
Connecting Data and Intelligence: The Role of FME in Machine Learning
Connecting Data and Intelligence: The Role of FME in Machine LearningSafe Software In this presentation, we want to explore powerful data integration and preparation for Machine Learning. FME is known for its ability to manipulate and transform geospatial data, connecting diverse data sources into efficient and automated workflows. By integrating FME with Machine Learning techniques, it is possible to transform raw data into valuable insights faster and more accurately, enabling intelligent analysis and data-driven decision making.
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdfPriyanka Aash Techniques for Automatic Device Identification and Network Assignment
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Salesforce Summer '25 Release Frenchgathering.pptx.pdfyosra Saidani Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Curietech AI in action - Accelerate MuleSoft development
Curietech AI in action - Accelerate MuleSoft developmentshyamraj55 CurieTech AI in Action – Accelerate MuleSoft Development
Overview:
This presentation demonstrates how CurieTech AI’s purpose-built agents empower MuleSoft developers to create integration workflows faster, more accurately, and with less manual effort
linkedin.com
+12
curietech.ai
+12
meetups.mulesoft.com
+12
.
Key Highlights:
Dedicated AI agents for every stage: Coding, Testing (MUnit), Documentation, Code Review, and Migration
curietech.ai
+7
curietech.ai
+7
medium.com
+7
DataWeave automation: Generate mappings from tables or samples—95%+ complete within minutes
linkedin.com
+7
curietech.ai
+7
medium.com
+7
Integration flow generation: Auto-create Mule flows based on specifications—speeds up boilerplate development
curietech.ai
+1
medium.com
+1
Efficient code reviews: Gain intelligent feedback on flows, patterns, and error handling
youtube.com
+8
curietech.ai
+8
curietech.ai
+8
Test & documentation automation: Auto-generate MUnit test cases, sample data, and detailed docs from code
curietech.ai
+5
curietech.ai
+5
medium.com
+5
Why Now?
Achieve 10× productivity gains, slashing development time from hours to minutes
curietech.ai
+3
curietech.ai
+3
medium.com
+3
Maintain high accuracy with code quality matching or exceeding manual efforts
curietech.ai
+2
curietech.ai
+2
curietech.ai
+2
Ideal for developers, architects, and teams wanting to scale MuleSoft projects with AI efficiency
Conclusion:
CurieTech AI transforms MuleSoft development into an AI-accelerated workflow—letting you focus on innovation, not repetition.
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdfPriyanka Aash A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my billstijn40 VMware’s pricing changes have forced organizations to rethink their datacenter cost management strategies. While FinOps is commonly associated with cloud environments, the FinOps Foundation has recently expanded its framework to include Scopes—and Datacenter is now officially part of the equation. In this session, we’ll map the FinOps Framework to a VMware-based datacenter, focusing on cost visibility, optimization, and automation. You’ll learn how to track costs more effectively, rightsize workloads, optimize licensing, and drive efficiency—all without migrating to the cloud. We’ll also explore how to align IT teams, finance, and leadership around cost-aware decision-making for on-prem environments. If your VMware bill keeps increasing and you need a new approach to cost management, this session is for you!
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...janeliewang985 the slides of the MP selection approach CMPS
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...Fwdays We will explore how e-commerce projects prepare for the busiest time of the year, which key aspects to focus on, and what to expect. We’ll share our experience in setting up auto-scaling, load balancing, and discuss the loads that Silpo handles, as well as the solutions that help us navigate this season without failures.
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdfcaoyixuan2019 A presentation at Internetware 2025.
Scaling with Postgres (Highload++ 2010)
- 1. Highload++ 2010 Scaling with Postgres Robert Treat Monday, October 25, 2010
- 2. Who Am I? ✤ Robert Treat ✤ OmniTI ✤ Design, Development, Database, Ops Monday, October 25, 2010
- 3. Who Am I? ✤ Robert Treat ✤ OmniTI ✤ Design, Development, DATABASE, Ops Monday, October 25, 2010
- 4. Who Am I? ✤ Robert Treat ✤ OmniTI ✤ Design, Development, DATABASE, Ops ✤ Etsy, Allisports, National Geographic, Gilt, etc... Monday, October 25, 2010
- 5. Who Am I? ✤ Robert Treat ✤ Postgres ✤ Web, Advocacy, phpPgAdmin ✤ Major Contributor Monday, October 25, 2010
- 6. Who Am I? ✤ Postgres 6.5 -> 9.1alpha1 ✤ Terabytes of data ✤ Millions of transactions per day ✤ OLTP, ODS, DW ✤ Perl, PHP, Java, Ruby, C# Monday, October 25, 2010
- 7. Who Am I? OBSERVATION == LEARNING (hopefully) Monday, October 25, 2010
- 8. Scalability It is the ability of a computer application or product (hardware or software) to continue to function well when it (or its context) is changed in size or volume in order to meet a user need. Monday, October 25, 2010
- 9. Scalability Given ever increasing load Monday, October 25, 2010
- 10. Scalability NEVER GO DOWN ALWAYS PERFORM WELL Given ever increasing load Monday, October 25, 2010
- 11. Scalability NEVER GO DOWN ALWAYS PERFORM WELL Given ever increasing load impossible goal, but we’ll try Monday, October 25, 2010
- 12. Scalability NEVER GO DOWN ALWAYS PERFORM WELL Given ever increasing load NOTE! data loss is not a goal, but ideally we won’t lose it :-) Monday, October 25, 2010
- 13. It starts with culture... Monday, October 25, 2010
- 14. ✤ Get over schema purity ✤ add column default not null Monday, October 25, 2010
- 15. ✤ Get over schema purity ✤ add column default not null Good performance comes from good schema design, HOWEVER, perfect relational modeling is NOT THE GOAL Monday, October 25, 2010
- 16. ✤ Devs must own schema and queries ✤ they design, you refine Monday, October 25, 2010
- 17. ✤ Devs must own schema and queries ✤ they design, you refine Performance and scalability cannot be managed solely within the database; both require application level knowledge. To achieve this, application developers need to have visibility of the resources they work on Monday, October 25, 2010
- 18. GainVisibility Monday, October 25, 2010
- 19. GainVisibility ✤ Monitoring ✤ Alerts ✤ Trending ✤ Capacity Planning ✤ Performance Tuning Monday, October 25, 2010
- 20. GainVisibility ✤ Alerts ✤ server: out of disk space, high load, etc... ✤ database: connections, sequences, etc... ✤ business: registrations, revenue, etc... ✤ etc... check_postgres.pl Monday, October 25, 2010
- 21. GainVisibility ✤ Trending ✤ server: disk usage, load, etc... ✤ database: connections, sequences, etc... ✤ business: registrations, revenue, etc... ✤ etc... cacti, mrtg, circonus Monday, October 25, 2010
- 22. GainVisibility ✤ Capacity Planning ✤ disks, cpu, memory ✤ connections, vacuum, bloat simple projections, done regularly, are good enough Monday, October 25, 2010
- 23. GainVisibility ✤ Performance tuning ✤ how long do queries take? ✤ how often do they run? pgfouine Monday, October 25, 2010
- 25. GainVisibility ALL alerts, graphs, query reports, etc... MUST be available to EVERYONE on the team AT ALL TIMES Monday, October 25, 2010
- 26. Hands on You can’t succeed without first putting the right culture in place. Once you are on the right path, make sure you have the right technology Monday, October 25, 2010
- 27. PostgresVersions ✤ MINIMUM: 8.3 ✤ removes xid for read only queries, significant reduction in vacuum activity Monday, October 25, 2010
- 28. PostgresVersions ✤ MINIMUM: 8.3 ✤ removes xid for read only queries, significant reduction in vacuum activity seriously! Monday, October 25, 2010
- 29. PostgresVersions ✤ MINIMUM: 8.3 ✤ removes xid for read only queries, significant reduction in vacuum activity ✤ BETTER: 8.4 ✤ revised free space map management leads to more efficient vacuuming Monday, October 25, 2010
- 30. PostgresVersions ✤ MINIMUM: 8.3 ✤ removes xid for read only queries, significant reduction in vacuum activity ✤ BETTER: 8.4 ✤ revised free space map management leads to more efficient vacuuming ✤ WHY NOT? 9.0 ✤ Hot standby / streaming replication couldn’t hurt Monday, October 25, 2010
- 31. Speaking of replication ✤ Common practice for scaling websites ✤ Good for READ based loads ✤ We have used many: ✤ slony, rubyrep, bucardo, 9.0 built-in, mammoth, wrote-our-own Monday, October 25, 2010
- 32. Speaking of replication Monday, October 25, 2010
- 33. Speaking of replication ✤ No favorite system for this, evaluate based on: ✤ avoid solutions that duplicate writes at sql level (imho) ✤ how comfortable am I debugging the system? ✤ do you need automated schema changes? ✤ how much redundancy / complexity do you need? ✤ how does the system handle node failure for N nodes? Monday, October 25, 2010
- 34. So what would you use? (tm) ✤ 2 Nodes, master + standby: Postgres 9.0 ✤ Master + multiple slaves: Slony ✤ Master-Master: Bucardo All choices subject to change!! Monday, October 25, 2010
- 35. A word about “Sharding” ✤ Distributed computing is hard(er) ✤ we think of things in a singular global state ✤ the more we can work in that model, the better ✤ RDBM offer poor solutions for multiple masters ✤ you must manage that complexity on your own Monday, October 25, 2010
- 36. A word about “Sharding” ✤ Splitting systems by service: ✤ separate db for login, forums, sales, etc... ✤ allows for growth ✤ provides simple interface Monday, October 25, 2010
- 37. Pooling ✤ Postgres connections are expensive! ✤ fork new process per connection ✤ keep 1 process open per connection ✤ 1000+ processes you will notice trouble Monday, October 25, 2010
- 38. Pooling ✤ Postgres connections are expensive! ✤ fork new process per connection ✤ keep 1 process open per connection ✤ 1000+ processes you will notice trouble ✤ POOLING ✤ JDBC, mod-perl ✤ pgbouncer ftw! Monday, October 25, 2010
- 39. Summary ✤ Schema / Queries should be shared between dev, dba teams! ✤ Monitoring + Visibility! ✤ >= 8.3 Required! ✤ Replication, jump in it! ✤ Use connection pooling! Monday, October 25, 2010
- 40. Thanks! more: @robtreat2 www.xzilla.net Oleg & Crew Highload++ OmniTI Postgres Community! You! Monday, October 25, 2010