SlideShare a Scribd company logo

PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky

PostgreSQL-Consulting, profile picture
PostgreSQL-Consulting

The document discusses various 'worst practices' in PostgreSQL databases, humorously suggesting that following these can lead to significant issues. Key points include avoiding indexes, using ORM, not enforcing foreign keys, and keeping time series data. The talk emphasizes that traditional best practices are often tedious and that these controversial approaches serve to highlight potential pitfalls in database management.

Engineering
1 of 58
Downloaded 56 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
PostgreSQL worst practices Ilya Kosmodemiansky ik@dataegret.com
Best practices are just boring • Never follow them, try worst practices • Only worst practices can really help you screw things up in a most effective way • PostgreSQL consultants are nice people - keep them happy! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
How it works? • I have a list, a little bit more than 100 worst practices • I do not make this stuff up, all of them are real-life examples • I reshuffle my list every time before presenting and pick a few examples • Well, there are some things, which I like more or less, so it is not a very honest shuffle dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
0. Do not use indexes (a test one!) • Basically, there is no difference between full table scan and index scan • You can check that. Just insert 10 rows into a test table on your test server and compare. • Nobody deals with more than 10 row tables in production! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
1. Use as many count(*) as you can • Figure 301083021830123921 is very informative for the end user • If it changes in a second to 30108302894839434020, it is still informative • select count(*) from sometable is a quite light-weighted query • Tuple estimation from pg_catalog can never be precise enough to you dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
2. Use ORM • All databases share the same syntax • You must write database-independent code • Are there any benefits, which are based on database specific features? • It always good to learn a new complicated technology dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
3. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
3. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level • Now you only need to implement nested loop join, hash join and merge join as well as query optimizer and page cache dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
4. Be in trend, be schema-less • You do not need to design the schema • You only need one table, two columns: id bigserial and extra jsonb • JSONB datatype is pretty effective in PostgreSQL, you can query it just like a well-structured table • Even if you put a 100M of JSON in it • Even if you have 1000+ tps dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
5. Be agile, use EAV • You only need 3 tables: entity, attribute, value dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • When it starts slowing down, just call those four tables The Core and add 1000+ tables with denormalized data dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • When it starts slowing down, just call those four tables The Core and add 1000+ tables with denormalized data • If it is not enough, you can always add value_version dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
6. Try to create as many indexes as you can • Indexes consume no disk space • Indexes consume no shared_bufers • There is no overhead on DML if one and every column in a table covered with bunch of indexes • Optimizer will definitely choose your index once you created it • Keep calm and create more indexes dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data • Solve the problem using partitioning... one partition for an hour or for a minute dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
8. Turn autovacuum off • It is quite auxiliary process, you can easily stop it • There is no problem at all to have 100Gb data in a database which is 1Tb in size • 2-3Tb RAM servers are cheap, IO is a fastest thing in modern computing • Besides, everyone likes BigData dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
9. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
9. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch • That allows you to run into all sorts of problems PostgreSQL had since introducing Slony dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
10. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
10. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover • To make things even worse, you can change only slave-related parameters at slave, leaving defaults for shared_buffers etc. dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
11. Put a synchronous replica to remote DC • Indeed! That will maximize availability! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
11. Put a synchronous replica to remote DC • Indeed! That will maximize availability! • Especially, if you put the replica to another continent dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
12. Never use Foreign Keys • Consistency control at application level always works as expected • You will never get data inconsistency without constraints • Even if you already have a bullet proof framework to maintain consistency, could it be good enough reason to use it? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
13. Always use text type for all columns • It is always fun to reimplement date or ip validation in your code • You will never mistakenly convert ”12-31-2015 03:01AM” to ”15:01 12 of undef 2015” using text fields dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
14. Always use improved ”PostgreSQL” • Postgres is not a perfect database and you are smart • All that annoying MVCC staff, 32 bit xid and autovacuum nightmares look the way they do because hackers are oldschool and lazy • Hack it in a hard way, do not bother submitting your patch to the community, just put it into production • It is easy to maintain such production and keep it compatible with ”not perfect” PostgreSQL upcoming versions dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout • Anyway, you can just start transaction and go away for a weekend dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
16. Never read your code, write it! genre_id IN ( SELECT id FROM genres WHERE genres.id IN (SELECT * FROM unnest(array[155])) ) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Have problems with you PostgreSQL installation? • Move those problems to the container! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Have problems with you PostgreSQL installation? • Move those problems to the container! • It is always good to have something very stable inside something very unstable! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Have problems with you PostgreSQL installation? • Move those problems to the container! • It is always good to have something very stable inside something very unstable! • Now your problems are both inside and outside! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! • Need a message queue? Write it! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! • Need a message queue? Write it! • Won’t use ORM? Write your own in plpgsql dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
18. And never, never use exceptions • Documentation says they are slow dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
18. And never, never use exceptions • Documentation says they are slow • Raise notice on errors - everyone reads logs constantly! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
18. And never, never use exceptions • Documentation says they are slow • Raise notice on errors - everyone reads logs constantly! • Who cares about errors? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
19. Application runs out of connections? • Set max_connections to 1000 dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now • Who said PostgreSQL workers have some overhead? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now • Who said PostgreSQL workers have some overhead? • And never ever use pgbouncer! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
20. Use pgpool-II instead • Pooling connections with pgpool is easy... dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... • Simple config, useful features dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... • Simple config, useful features • Consulters are happy! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf • Forget about those shared_buffers and checkpoints! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf • Forget about those shared_buffers and checkpoints! • geqo options are a good candidate to be a silver bullet! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
22. Have heard about a cool new feature? • Use it in production immediately! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) • Learn about xmin and xmax dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) • Learn about xmin and xmax • Use it in your applications logic! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
Do not forget That was WORST practices talk dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
Send me your favourite! ik@dataegret.com dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com

More Related Content

PDF
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
PostgreSQL-Consulting
 
PDF
PostgreSQL HA
haroonm
 
PDF
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
Alkin Tezuysal
 
PPTX
Automating with Ansible
Ricardo Schmidt
 
PDF
Oracle Enterprise Manager Cloud Control 13c for DBAs
Gokhan Atil
 
PDF
Ansible - Introduction
Stephane Manciot
 
PDF
MySQL Enterprise Backup (MEB)
Mydbops
 
PDF
How to monitor your micro-service with Prometheus?
Wojciech Barczyński
 
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
PostgreSQL-Consulting
 
PostgreSQL HA
haroonm
 
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
Alkin Tezuysal
 
Automating with Ansible
Ricardo Schmidt
 
Oracle Enterprise Manager Cloud Control 13c for DBAs
Gokhan Atil
 
Ansible - Introduction
Stephane Manciot
 
MySQL Enterprise Backup (MEB)
Mydbops
 
How to monitor your micro-service with Prometheus?
Wojciech Barczyński
 

What's hot (20)

PDF
Bulk Loading Data into Cassandra
DataStax
 
PDF
Patroni - HA PostgreSQL made easy
Alexander Kukushkin
 
PDF
How to find what is making your Oracle database slow
SolarWinds
 
PDF
PostgreSQL and RAM usage
Alexey Bashtanov
 
PPTX
Hadoop technology
tipanagiriharika
 
PDF
Backup and recovery in oracle
sadegh salehi
 
PPTX
Prometheus design and philosophy
Docker, Inc.
 
PDF
A Technical Introduction to WiredTiger
MongoDB
 
PPT
Galera Cluster Best Practices for DBA's and DevOps Part 1
Codership Oy - Creators of Galera Cluster
 
PDF
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
PPTX
Nginx
Dhrubaji Mandal ♛
 
PPTX
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations
Grant McAlister
 
PDF
New Generation Oracle RAC Performance
Anil Nair
 
PDF
Getting started with Ansible
Ivan Serdyuk
 
PDF
SeaweedFS introduction
chrislusf
 
PDF
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)
Brian Brazil
 
PPTX
MariaDB Galera Cluster
Abdul Manaf
 
PDF
MariaDB Galera Cluster - Simple, Transparent, Highly Available
MariaDB Corporation
 
PDF
stupid-simple-kubernetes-final.pdf
DaniloQueirozMota
 
PPTX
Windows Azure Blob Storage
ylew15
 
Bulk Loading Data into Cassandra
DataStax
 
Patroni - HA PostgreSQL made easy
Alexander Kukushkin
 
How to find what is making your Oracle database slow
SolarWinds
 
PostgreSQL and RAM usage
Alexey Bashtanov
 
Hadoop technology
tipanagiriharika
 
Backup and recovery in oracle
sadegh salehi
 
Prometheus design and philosophy
Docker, Inc.
 
A Technical Introduction to WiredTiger
MongoDB
 
Galera Cluster Best Practices for DBA's and DevOps Part 1
Codership Oy - Creators of Galera Cluster
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
Nginx
Dhrubaji Mandal ♛
 
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations
Grant McAlister
 
New Generation Oracle RAC Performance
Anil Nair
 
Getting started with Ansible
Ivan Serdyuk
 
SeaweedFS introduction
chrislusf
 
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)
Brian Brazil
 
MariaDB Galera Cluster
Abdul Manaf
 
MariaDB Galera Cluster - Simple, Transparent, Highly Available
MariaDB Corporation
 
stupid-simple-kubernetes-final.pdf
DaniloQueirozMota
 
Windows Azure Blob Storage
ylew15
 
Ad

Similar to PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky (20)

PDF
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
PostgreSQL-Consulting
 
PDF
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky
PostgreSQL-Consulting
 
PDF
The 5 Minute MySQL DBA
Irawan Soetomo
 
PDF
Neo4j Training Cypher
Max De Marzi
 
PPTX
Austin Python Learners Meetup - Everything you need to know about programming...
Danny Mulligan
 
PDF
Storage Systems For Scalable systems
elliando dias
 
PDF
Yapc10 Cdt World Domination
cPanel
 
PDF
Unbreaking Your Django Application
OSCON Byrum
 
PPTX
1. SQL Server forSharePoint geeksA gentle introductionThomas Vochten • Septem...
BIWUG
 
PDF
Lessons PostgreSQL learned from commercial databases, and didn’t
PGConf APAC
 
PDF
Running MySQL on Linux
Great Wide Open
 
PDF
How to Build a Compute Cluster
Ramsay Key
 
PPS
MySQL Optimization from a Developer's point of view
Sachin Khosla
 
PDF
Know thy cost (or where performance problems lurk)
Oren Eini
 
PPTX
Scaling a High Traffic Web Application: Our Journey from Java to PHP
120bi
 
PPTX
Scaling High Traffic Web Applications
Achievers Tech
 
PPTX
Introduction to Redis
Ofer Zelig
 
PDF
Linux internals for Database administrators at Linux Piter 2016
PostgreSQL-Consulting
 
PPT
Technical track-afterimaging Progress Database
Vinh Nguyen
 
PPTX
Redis: An introduction
Đặng Thảo
 
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
PostgreSQL-Consulting
 
PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky
PostgreSQL-Consulting
 
The 5 Minute MySQL DBA
Irawan Soetomo
 
Neo4j Training Cypher
Max De Marzi
 
Austin Python Learners Meetup - Everything you need to know about programming...
Danny Mulligan
 
Storage Systems For Scalable systems
elliando dias
 
Yapc10 Cdt World Domination
cPanel
 
Unbreaking Your Django Application
OSCON Byrum
 
1. SQL Server forSharePoint geeksA gentle introductionThomas Vochten • Septem...
BIWUG
 
Lessons PostgreSQL learned from commercial databases, and didn’t
PGConf APAC
 
Running MySQL on Linux
Great Wide Open
 
How to Build a Compute Cluster
Ramsay Key
 
MySQL Optimization from a Developer's point of view
Sachin Khosla
 
Know thy cost (or where performance problems lurk)
Oren Eini
 
Scaling a High Traffic Web Application: Our Journey from Java to PHP
120bi
 
Scaling High Traffic Web Applications
Achievers Tech
 
Introduction to Redis
Ofer Zelig
 
Linux internals for Database administrators at Linux Piter 2016
PostgreSQL-Consulting
 
Technical track-afterimaging Progress Database
Vinh Nguyen
 
Redis: An introduction
Đặng Thảo
 
Ad

More from PostgreSQL-Consulting (11)

PDF
10 things, an Oracle DBA should care about when moving to PostgreSQL
PostgreSQL-Consulting
 
PDF
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
PostgreSQL-Consulting
 
PDF
Linux tuning to improve PostgreSQL performance
PostgreSQL-Consulting
 
PDF
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL-Consulting
 
PDF
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
PostgreSQL-Consulting
 
PDF
Pgconfru 2015 kosmodemiansky
PostgreSQL-Consulting
 
PDF
Как PostgreSQL работает с диском
PostgreSQL-Consulting
 
PDF
Kosmodemiansky wr 2013
PostgreSQL-Consulting
 
PDF
Максим Богук. Postgres-XC
PostgreSQL-Consulting
 
PDF
Иван Фролков. Tricky SQL
PostgreSQL-Consulting
 
PDF
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
PostgreSQL-Consulting
 
10 things, an Oracle DBA should care about when moving to PostgreSQL
PostgreSQL-Consulting
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
PostgreSQL-Consulting
 
Linux tuning to improve PostgreSQL performance
PostgreSQL-Consulting
 
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL-Consulting
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
PostgreSQL-Consulting
 
Pgconfru 2015 kosmodemiansky
PostgreSQL-Consulting
 
Как PostgreSQL работает с диском
PostgreSQL-Consulting
 
Kosmodemiansky wr 2013
PostgreSQL-Consulting
 
Максим Богук. Postgres-XC
PostgreSQL-Consulting
 
Иван Фролков. Tricky SQL
PostgreSQL-Consulting
 
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
PostgreSQL-Consulting
 

Recently uploaded (20)

PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
Educational Group Mohammad Farshadian
 
PDF
composite construction of structures.pdf
AinieButt1
 
PDF
Model Code of Practice - Construction Work - 21102022 .pdf
AinieButt1
 
PPTX
Lecture Notes Electrical Wiring System Components
eedgecbhojpur
 
PPTX
MCN 401 KTU-2019-PPE KITS-MODULE 2.pptx
VinayB68
 
PPTX
bas. eng. economics group 4 presentation 1.pptx
kiribakimoses1993
 
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
pcmth867
 
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
smceramu
 
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Chandra Kumar S
 
PDF
PPT on Performance Review to get promotions
prashant593122
 
PPTX
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
ssusera400e8
 
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
Yu Huang
 
PPT
Project quality management in manufacturing
BarMusaTetik
 
PDF
R24 SURVEYING LAB MANUAL for civil enggi
MNANDITHACIVILSTAFF
 
DOCX
573137875-Attendance-Management-System-original
AYUSHMANAV
 
PPTX
web development for engineering and engineering
keshriji700
 
PPTX
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
Alberto González Trastoy
 
PPTX
Internet of Things (IOT) - A guide to understanding
Davies Chacko
 
PDF
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PRIZ Guru
 
PPTX
additive manufacturing of ss316l using mig welding
premcdr7799
 
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
Educational Group Mohammad Farshadian
 
composite construction of structures.pdf
AinieButt1
 
Model Code of Practice - Construction Work - 21102022 .pdf
AinieButt1
 
Lecture Notes Electrical Wiring System Components
eedgecbhojpur
 
MCN 401 KTU-2019-PPE KITS-MODULE 2.pptx
VinayB68
 
bas. eng. economics group 4 presentation 1.pptx
kiribakimoses1993
 
Automation-in-Manufacturing-Chapter-Introduction.pdf
pcmth867
 
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
smceramu
 
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Chandra Kumar S
 
PPT on Performance Review to get promotions
prashant593122
 
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
ssusera400e8
 
Embodied AI: Ushering in the Next Era of Intelligent Systems
Yu Huang
 
Project quality management in manufacturing
BarMusaTetik
 
R24 SURVEYING LAB MANUAL for civil enggi
MNANDITHACIVILSTAFF
 
573137875-Attendance-Management-System-original
AYUSHMANAV
 
web development for engineering and engineering
keshriji700
 
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
Alberto González Trastoy
 
Internet of Things (IOT) - A guide to understanding
Davies Chacko
 
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PRIZ Guru
 
additive manufacturing of ss316l using mig welding
premcdr7799
 

PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky

  • 2. Best practices are just boring • Never follow them, try worst practices • Only worst practices can really help you screw things up in a most effective way • PostgreSQL consultants are nice people - keep them happy! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 3. How it works? • I have a list, a little bit more than 100 worst practices • I do not make this stuff up, all of them are real-life examples • I reshuffle my list every time before presenting and pick a few examples • Well, there are some things, which I like more or less, so it is not a very honest shuffle dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 4. 0. Do not use indexes (a test one!) • Basically, there is no difference between full table scan and index scan • You can check that. Just insert 10 rows into a test table on your test server and compare. • Nobody deals with more than 10 row tables in production! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 5. 1. Use as many count(*) as you can • Figure 301083021830123921 is very informative for the end user • If it changes in a second to 30108302894839434020, it is still informative • select count(*) from sometable is a quite light-weighted query • Tuple estimation from pg_catalog can never be precise enough to you dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 6. 2. Use ORM • All databases share the same syntax • You must write database-independent code • Are there any benefits, which are based on database specific features? • It always good to learn a new complicated technology dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 7. 3. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 8. 3. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level • Now you only need to implement nested loop join, hash join and merge join as well as query optimizer and page cache dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 9. 4. Be in trend, be schema-less • You do not need to design the schema • You only need one table, two columns: id bigserial and extra jsonb • JSONB datatype is pretty effective in PostgreSQL, you can query it just like a well-structured table • Even if you put a 100M of JSON in it • Even if you have 1000+ tps dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 10. 5. Be agile, use EAV • You only need 3 tables: entity, attribute, value dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 11. 5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 12. 5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • When it starts slowing down, just call those four tables The Core and add 1000+ tables with denormalized data dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 13. 5. Be agile, use EAV • You only need 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • When it starts slowing down, just call those four tables The Core and add 1000+ tables with denormalized data • If it is not enough, you can always add value_version dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 14. 6. Try to create as many indexes as you can • Indexes consume no disk space • Indexes consume no shared_bufers • There is no overhead on DML if one and every column in a table covered with bunch of indexes • Optimizer will definitely choose your index once you created it • Keep calm and create more indexes dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 15. 7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 16. 7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 17. 7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 18. 7. Always keep all your time series data • Time series data like tables with logs or session history should never be deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data • Solve the problem using partitioning... one partition for an hour or for a minute dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 19. 8. Turn autovacuum off • It is quite auxiliary process, you can easily stop it • There is no problem at all to have 100Gb data in a database which is 1Tb in size • 2-3Tb RAM servers are cheap, IO is a fastest thing in modern computing • Besides, everyone likes BigData dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 20. 9. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 21. 9. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch • That allows you to run into all sorts of problems PostgreSQL had since introducing Slony dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 22. 10. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 23. 10. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover • To make things even worse, you can change only slave-related parameters at slave, leaving defaults for shared_buffers etc. dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 24. 11. Put a synchronous replica to remote DC • Indeed! That will maximize availability! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 25. 11. Put a synchronous replica to remote DC • Indeed! That will maximize availability! • Especially, if you put the replica to another continent dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 26. 12. Never use Foreign Keys • Consistency control at application level always works as expected • You will never get data inconsistency without constraints • Even if you already have a bullet proof framework to maintain consistency, could it be good enough reason to use it? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 27. 13. Always use text type for all columns • It is always fun to reimplement date or ip validation in your code • You will never mistakenly convert ”12-31-2015 03:01AM” to ”15:01 12 of undef 2015” using text fields dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 28. 14. Always use improved ”PostgreSQL” • Postgres is not a perfect database and you are smart • All that annoying MVCC staff, 32 bit xid and autovacuum nightmares look the way they do because hackers are oldschool and lazy • Hack it in a hard way, do not bother submitting your patch to the community, just put it into production • It is easy to maintain such production and keep it compatible with ”not perfect” PostgreSQL upcoming versions dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 29. 15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 30. 15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 31. 15. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout • Anyway, you can just start transaction and go away for a weekend dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 32. 16. Never read your code, write it! genre_id IN ( SELECT id FROM genres WHERE genres.id IN (SELECT * FROM unnest(array[155])) ) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 33. 17. Have problems with you PostgreSQL installation? • Move those problems to the container! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 34. 17. Have problems with you PostgreSQL installation? • Move those problems to the container! • It is always good to have something very stable inside something very unstable! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 35. 17. Have problems with you PostgreSQL installation? • Move those problems to the container! • It is always good to have something very stable inside something very unstable! • Now your problems are both inside and outside! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 36. 17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 37. 17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! • Need a message queue? Write it! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 38. 17. Not only Slony should be reinvented! • Need to convert timestamp? Stored procedure in C will help! • Need a message queue? Write it! • Won’t use ORM? Write your own in plpgsql dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 39. 18. And never, never use exceptions • Documentation says they are slow dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 40. 18. And never, never use exceptions • Documentation says they are slow • Raise notice on errors - everyone reads logs constantly! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 41. 18. And never, never use exceptions • Documentation says they are slow • Raise notice on errors - everyone reads logs constantly! • Who cares about errors? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 42. 19. Application runs out of connections? • Set max_connections to 1000 dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 43. 19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 44. 19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now • Who said PostgreSQL workers have some overhead? dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 45. 19. Application runs out of connections? • Set max_connections to 1000 • Common, servers with 1000 CPUs are cheap now • Who said PostgreSQL workers have some overhead? • And never ever use pgbouncer! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 46. 20. Use pgpool-II instead • Pooling connections with pgpool is easy... dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 47. 20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 48. 20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... • Simple config, useful features dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 49. 20. Use pgpool-II instead • Pooling connections with pgpool is easy... • Just like writing a code in that Emacs OS... • Simple config, useful features • Consulters are happy! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 50. 21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 51. 21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf • Forget about those shared_buffers and checkpoints! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 52. 21. Always start tuning PostgreSQL... • From optimizer options in postgresql.conf • Forget about those shared_buffers and checkpoints! • geqo options are a good candidate to be a silver bullet! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 53. 22. Have heard about a cool new feature? • Use it in production immediately! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 54. 22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 55. 22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) • Learn about xmin and xmax dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 56. 22. Have heard about a cool new feature? • Use it in production immediately! • Attend MVCC Unmasked by Bruce Momjian (Today, 15:50, Liberty I) • Learn about xmin and xmax • Use it in your applications logic! dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 57. Do not forget That was WORST practices talk dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com
  • 58. Send me your favourite! [email protected] dataegret.com Why this talk • Linux is a most common OS for databases • DBAs often run into IO problems • Most of the information on topic is written by kerneldevelopers (for kernel developers) or is checklist-style • Checklists are useful, but up to certain workload dataegret.com