Practical Memory Tuning for PostgreSQL
- 1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. PRACTICAL MEMORY TUNING FOR POSTGRESQL Grant McAlister Senior Principal Engineer
- 2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why you should care 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 5 105 205 305 405 505 605 705 805 905 1005 1105 1205 1305 1405 1505 1605 1705 Transactions Per Second (TPS) Seconds sysbench read only point selects Normal Tuning Less than Optimal Tuning 11X buffers not in memory swap thrashing
- 3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. OOM & Swap 3
- 4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is the Out of Memory (OOM) Killer postgres 1 postgres 2 postgres 3 postgres 4 badness_for_task = total_vm_for_task / (sqrt(cpu_time_in_seconds) * sqrt(sqrt(cpu_time_in_minutes)))
- 5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is the Out of Memory (OOM) Killer postgres 1 postgres 2 postgres 3 postgres 4 badness_for_task = total_vm_for_task / (sqrt(cpu_time_in_seconds) * sqrt(sqrt(cpu_time_in_minutes)))
- 6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why swap? RAM request 8k
- 7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why swap? RAM request 16k FULL
- 8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why swap? RAM SWAP request 16k
- 9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why swap? RAM SWAP request 16k swap out swap in THRASHING
- 10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Overview 6
- 11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Memory Overview Operating System shared_buffers postgres processes pagecache 25% Operating System shared_buffers postgres processes pagecache 75%
- 12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pagecache File System Cache Storage shared_buffers wal archiving misc temp_buffers
- 13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pagecache File System Cache Storage shared_buffers wal archiving misc temp_buffers
- 14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pagecache File System Cache Storage shared_buffers wal archiving misc shared_buffers wal archiving misc temp_buffers temp_buffers
- 15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pagecache File System Cache Storage shared_buffers wal archiving misc shared_buffers wal archiving misc temp_buffers temp_buffers shared_buffers misc temp_buffers
- 16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size - Heat Orders_2023 Orders_2022 Orders_2021 Orders_2020 Orders_2019 Orders_2018 Orders_2017 Orders_2016 800GB 80-100GB hot working set
- 17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Inventory_SEA Inventory_LAX Inventory_SFO Inventory_EWR Inventory_SYD Inventory_YYC Inventory_BIO Inventory_SBP Working Set Size - Heat Orders_2023 Orders_2022 Orders_2021 Orders_2020 Orders_2019 Orders_2018 Orders_2017 Orders_2016 800GB 80-100GB hot working set Inventory_SEA Inventory_LAX Inventory_SFO Inventory_EWR Inventory_SYD Inventory_YYC Inventory_BIO Inventory_SBP 800GB 200-400GB hot working set
- 18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data Random heap index
- 19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data Random heap index
- 20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data Random heap index
- 21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data Random heap index
- 22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data right leaning heap index
- 23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data right leaning heap index
- 24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data right leaning heap index
- 25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Working Set Size – Indexes and Data right leaning heap index
- 26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Shared Buffers 12
- 27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Small Cache File System Cache Storage shared_buffers
- 28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. shared_buffers comparison 14 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 900,000 25% of host memory - 80 tables 25% of host memory - 125 tables 25% of host memory - 250 tables 12.5% of host memory - 250 tables 5% of host memory - 250 tables 1% of host memory - 250 tables Percent of object type in memory Transactions Per Second sysbench - read only point selects TPS Index Tables 60%
- 29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. shared_buffers comparison – pg_buffercache Type Usage count 25% of host memory 1% of host memory Index 5 855806 3630 Index 4 707472 2624 Index 3 124292 23 Index 2 22019 1754 Index 1 3799 73303 Index 0 558 71569 Table 5 316 1 Table 4 2764 3 Table 3 33572 105 Table 2 359638 180 Table 1 2933760 76505 Table 0 2606002 76303 15
- 30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. shared_buffers comparison – pg_buffercache Type Usage count 25% of host memory 1% of host memory Index 5 855806 3630 Index 4 707472 2624 Index 3 124292 23 Index 2 22019 1754 Index 1 3799 73303 Index 0 558 71569 Table 5 316 1 Table 4 2764 3 Table 3 33572 105 Table 2 359638 180 Table 1 2933760 76505 Table 0 2606002 76303 15
- 31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Big Cache File System Cache Storage shared_buffers
- 32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Big Cache File System Cache Storage shared_buffers
- 33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Big Cache File System Cache Storage
- 34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Big Cache File System Cache Storage shared_buffers
- 35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Big Cache File System Cache Storage shared_buffers
- 36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. HugePages 17
- 37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why HugePages – page mapping 18 PageTables shared_buffers pg 1 pg 2 pg 3 4k 4k size of page table = # of PostgreSQL process X amount of shared buffers accessed 2M
- 38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 5 30 55 80 105 130 155 180 205 230 255 280 305 330 355 380 405 430 455 480 505 530 555 580 605 630 655 680 705 730 755 780 805 830 855 880 905 930 955 980 1005 1030 1055 1080 1105 1130 1155 1180 1205 1230 1255 1280 1305 1330 1355 1380 1405 1430 1455 1480 1505 1530 1555 1580 1605 1630 1655 1680 1705 1730 1755 1780 Transactions Per Second (TPS) Seconds Sysbench Read Only Point Selects – r6i.8xlarge – 250 tables x 2.5M rows – 160GB 250 Client Hupepage=on 250 Client Hupepage=off
- 39. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 5 30 55 80 105 130 155 180 205 230 255 280 305 330 355 380 405 430 455 480 505 530 555 580 605 630 655 680 705 730 755 780 805 830 855 880 905 930 955 980 1005 1030 1055 1080 1105 1130 1155 1180 1205 1230 1255 1280 1305 1330 1355 1380 1405 1430 1455 1480 1505 1530 1555 1580 1605 1630 1655 1680 1705 1730 1755 1780 Transactions Per Second (TPS) Seconds Sysbench Read Only Point Selects – r6i.8xlarge – 250 tables x 2.5M rows – 160GB 500 Client Hupepage=on 500 Client Hupepage=off
- 40. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 5 30 55 80 105 130 155 180 205 230 255 280 305 330 355 380 405 430 455 480 505 530 555 580 605 630 655 680 705 730 755 780 805 830 855 880 905 930 955 980 1005 1030 1055 1080 1105 1130 1155 1180 1205 1230 1255 1280 1305 1330 1355 1380 1405 1430 1455 1480 1505 1530 1555 1580 1605 1630 1655 1680 1705 1730 1755 1780 Transactions Per Second (TPS) Seconds Sysbench Read Only Point Selects – r6i.8xlarge – 250 tables x 2.5M rows – 160GB 1000 Client Hupepage=on 1000 Client Hupepage=off
- 41. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cost of not setting HugePages 22 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 0 20 40 60 80 100 120 140 250 500 1000 Percent of host memory GB of PageTable number of postgreSQL process sysbench read only - shard_buffer=25% of ram PageTables size (gb) % of host memory
- 42. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Per Session 23
- 43. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. temp_buffers shared_buffers temp_buffers File System Cache regular tables temp tables global session local
- 44. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. SQL Stream logical_decoding_work_mem OLTP DW SQL Stream logical decoding slot wal log DW2 logical decoding slot SQL Stream
- 45. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Per Session Operation 26
- 46. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. work_mem postgres=# set work_mem TO '1 GB’; postgres=# explain analyze select mykey::bigint, (random()*10000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,3800000) as mykey order by scratch; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------ - Sort (cost=7219224.79..7228724.79 rows=3800000 width=4116) (actual time=7083.506..8358.093 rows=3800000 loops=1) Sort Key: (((random() * '10000000000'::double precision))::bigint) Sort Method: external merge Disk: 3919000kB -> Function Scan on generate_series mykey (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=194.053..493.117 rows=3800000 loops=1) Planning Time: 0.049 ms Execution Time: 9117.344 ms (6 rows) ~3.8GB
- 47. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 28 0 1 2 3 4 5 6 7 8 9 10 1 6 11 16 21 26 31 36 41 46 51 56 GB RAM Seconds work_mem - sorting 1GB work_mem 3.8GB query
- 48. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. work_mem postgres=# set work_mem TO '4 GB’; postgres=# explain analyze select mykey::bigint, (random()*10000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,3800000) as mykey order by scratch; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------ - Sort (cost=7219224.79..7228724.79 rows=3800000 width=4116) (actual time=2766.682..3408.706 rows=3800000 loops=1) Sort Key: (((random() * '10000000000'::double precision))::bigint) Sort Method: quicksort Memory: 4128025kB -> Function Scan on generate_series mykey (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=194.365..506.965 rows=3800000 loops=1) Planning Time: 0.047 ms Execution Time: 3825.965 ms (6 rows) ~3.8GB
- 49. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 30 0 1 2 3 4 5 6 7 8 9 10 1 6 11 16 21 26 31 36 41 46 51 56 GB RAM Seconds work_mem - sorting 4GB work_mem - 3.8GB query 1GB work_mem 3.8GB query
- 50. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. work_mem postgres=# set work_mem TO '4 GB’; explain analyze select s1.filler, s2.filler,s1.mykey, s2.mykey, s1.scratch, s2.scratch from (select mykey::bigint, (random()*1000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,3800000) as mykey order by scratch) s1, (select mykey::bigint, (random()*10000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,3800000) as mykey order by scratch) s2 where s1.scratch=s2.scratch order by s1.mykey; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------ --------------------- Sort (cost=523467712452.33..523648212452.33 rows=72200000000 width=8232) (actual time=6851.977..6852.172 rows=1494 loops=1) Sort Key: ((mykey.mykey)::bigint) Sort Method: quicksort Memory: 3153kB -> Merge Join (cost=14438449.59..1097542949.59 rows=72200000000 width=8232) (actual time=5558.904..6850.847 rows=1494 loops=1) Merge Cond: ((((random() * '1000000000'::double precision))::bigint) = (((random() * '10000000000'::double precision))::bigint)) -> Sort (cost=7219224.79..7228724.79 rows=3800000 width=4116) (actual time=2785.745..3524.489 rows=3800000 loops=1) Sort Key: (((random() * '1000000000'::double precision))::bigint) Sort Method: quicksort Memory: 4128025kB -> Function Scan on generate_series mykey (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=194.222..515.986 rows=3800000 loops=1) -> Materialize (cost=7219224.79..7276224.79 rows=3800000 width=4116) (actual time=2772.383..2961.007 rows=380357 loops=1) -> Sort (cost=7219224.79..7228724.79 rows=3800000 width=4116) (actual time=2772.377..2867.394 rows=380353 loops=1) Sort Key: (((random() * '10000000000'::double precision))::bigint) Sort Method: quicksort Memory: 4128025kB -> Function Scan on generate_series mykey_1 (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=198.688..511.083 rows=3800000 loops=1) Planning Time: 0.098 ms Execution Time: 7511.377 ms ~3.8GB
- 51. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 32 0 1 2 3 4 5 6 7 8 9 10 1 6 11 16 21 26 31 36 41 46 51 56 GB RAM Seconds work_mem - sorting 4GB work_mem - 7.6GB+ query 4GB work_mem - 3.8GB query 1GB work_mem 3.8GB query
- 52. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. work_mem postgres=# set work_mem TO '4 GB’; postgres=# explain analyze select s1.filler, s2.filler,s1.mykey, s2.mykey, s1.scratch, s2.scratch from (select mykey::bigint, (random()*1000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,18800000) as mykey order by scratch) s1, (select mykey::bigint, (random()*10000000000)::bigint as scratch , repeat('X', 1024)::char(1024) filler from generate_series(1,3800000) as mykey order by scratch) s2 where s1.scratch=s2.scratch order by s1.mykey; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------ ------------------------- Sort (cost=3853846009559.71..3854739009559.71 rows=357200000000 width=8232) (actual time=52689.953..52690.952 rows=7184 loops=1) Sort Key: ((mykey_1.mykey)::bigint) Sort Method: quicksort Memory: 15122kB -> Merge Join (cost=43152211.85..5401444211.85 rows=357200000000 width=8232) (actual time=42991.198..52679.295 rows=7184 loops=1) Merge Cond: ((((random() * '10000000000'::double precision))::bigint) = (((random() * '1000000000'::double precision))::bigint)) -> Sort (cost=7219224.79..7228724.79 rows=3800000 width=4116) (actual time=2771.496..2877.777 rows=380382 loops=1) Sort Key: (((random() * '10000000000'::double precision))::bigint) Sort Method: quicksort Memory: 4128025kB -> Function Scan on generate_series mykey (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=194.404..516.400 rows=3800000 loops=1) -> Materialize (cost=35932987.06..36214987.06 rows=18800000 width=4116) (actual time=40218.833..48614.865 rows=18800001 loops=1) -> Sort (cost=35932987.06..35979987.06 rows=18800000 width=4116) (actual time=40218.824..46815.209 rows=18800000 loops=1) Sort Key: (((random() * '1000000000'::double precision))::bigint) Sort Method: external merge Disk: 19388680kB -> Function Scan on generate_series mykey_1 (cost=0.00..376000.00 rows=18800000 width=4116) (actual time=960.638..2593.178 rows=18800000 loops=1) Planning Time: 0.102 ms Execution Time: 55724.412 ms ~3.8GB ~18.8GB
- 53. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 34 0 1 2 3 4 5 6 7 8 9 10 1 6 11 16 21 26 31 36 41 46 51 56 GB RAM Seconds work_mem - sorting 4GB work_mem - 18.8GB+ query 4GB work_mem - 7.6GB+ query 4GB work_mem - 3.8GB query 1GB work_mem 3.8GB query
- 54. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. hash_mem_multiplier postgres=# show hash_mem_multiplier; hash_mem_multiplier --------------------- 2 postgres=# set work_mem=‘4 GB'; QUERY PLAN --------------------------------------------------------------------------------------------------------------------------------------------------------------- Hash Join (cost=2083766.00..5865195158.01 rows=167200000000 width=8232) (actual time=4360.794..16414.748 rows=3344 loops=1) Hash Cond: (s1.scratch = s2.scratch) -> Subquery Scan on s1 (cost=0.00..264000.00 rows=8800000 width=4116) (actual time=374.153..1692.160 rows=8800000 loops=1) -> Function Scan on generate_series mykey (cost=0.00..176000.00 rows=8800000 width=4116) (actual time=374.152..1161.469 rows=8800000 loops=1) -> Hash (cost=114000.00..114000.00 rows=3800000 width=4116) (actual time=3982.004..3982.006 rows=3800000 loops=1) Buckets: 2097152 Batches: 2 Memory Usage: 2020102kB -> Subquery Scan on s2 (cost=0.00..114000.00 rows=3800000 width=4116) (actual time=186.934..742.574 rows=3800000 loops=1) -> Function Scan on generate_series mykey_1 (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=186.932..515.290 rows=3800000 loops=1) Planning Time: 0.090 ms Execution Time: 16655.195 ms (10 rows) postgres=# set work_mem='1 GB'; SET QUERY PLAN --------------------------------------------------------------------------------------------------------------------------------------------------------------- Hash Join (cost=2083766.00..5865195158.01 rows=167200000000 width=8232) (actual time=5099.725..23063.116 rows=3575 loops=1) Hash Cond: (s1.scratch = s2.scratch) -> Subquery Scan on s1 (cost=0.00..264000.00 rows=8800000 width=4116) (actual time=376.939..1651.354 rows=8800000 loops=1) -> Function Scan on generate_series mykey (cost=0.00..176000.00 rows=8800000 width=4116) (actual time=376.937..1119.345 rows=8800000 loops=1) -> Hash (cost=114000.00..114000.00 rows=3800000 width=4116) (actual time=4714.890..4714.891 rows=3800000 loops=1) Buckets: 524288 Batches: 8 Memory Usage: 505086kB -> Subquery Scan on s2 (cost=0.00..114000.00 rows=3800000 width=4116) (actual time=181.422..740.776 rows=3800000 loops=1) -> Function Scan on generate_series mykey_1 (cost=0.00..76000.00 rows=3800000 width=4116) (actual time=181.420..512.871 rows=3800000 loops=1) Planning Time: 0.089 ms Execution Time: 23186.765 ms ~2.6GB – 16.6 seconds ~1.2GB – 23.1 seconds
- 55. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Per Session Item 36
- 56. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. prepared statements PREPARE sbtestplan (int) AS SELECT * FROM sbtest1 WHERE id=$1 ; EXECUTE sbtestplan(1); sbtest=# SELECT name, ident, level, total_bytes FROM pg_backend_memory_contexts where name ='CachedPlanSource'; name | ident | level | total_bytes ------------------+-----------------------------------------+-------+------------- CachedPlanSource | PREPARE sbtestplan3 (int) AS +| 2 | 4096 | SELECT * FROM sbtest3 WHERE id=$1 ; | | CachedPlanSource | PREPARE sbtestplan2 (int) AS +| 2 | 4096 | SELECT * FROM sbtest2 WHERE id=$1 ; | | CachedPlanSource | PREPARE sbtestplan (int) AS +| 2 | 4096 | SELECT * FROM sbtest1 WHERE id=$1 ; | |
- 57. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. prepared statements – 10 Million 38 0 20 40 60 80 100 1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97 103 109 115 121 127 133 139 145 151 157 163 169 175 181 187 193 199 205 211 217 223 229 235 241 247 253 259 265 271 277 283 289 295 301 307 313 319 325 331 337 343 349 355 361 GB Used Seconds 10 million prepared statements - 1 client
- 58. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Maintenance 39
- 59. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem – Index builds sbtest=# set maintenance_work_mem = '1 GB'; ^ sbtest=# create index foo on sbtest1 (id,c,pad,k desc) ; CREATE INDEX Time: 45803.577 ms (00:45.804) sbtest=# set maintenance_work_mem = '2 GB'; sbtest=# create index foo on sbtest1 (id,c,pad,k desc) ; CREATE INDEX Time: 45904.760 ms (00:45.905) sbtest=# set maintenance_work_mem = '4 GB'; sbtest=# create index foo on sbtest1 (id,c,pad,k desc) ; CREATE INDEX Time: 46081.414 ms (00:46.081) ~1GB ~2GB ~3GB
- 60. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem – how vacuum works heap index index vacuum scan heap
- 61. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem – how vacuum works heap index index vacuum maintenance_work_mem tid to clean scan heap
- 62. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem – how vacuum works heap index index vacuum maintenance_work_mem tid to clean scan heap scan indexes
- 63. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem - vacuum sbtest=# set maintenance_work_mem='100 MB'; sbtest=# vacuum (verbose) sbtest1; INFO: vacuuming "sbtest.public.sbtest1" INFO: launched 1 parallel vacuum worker for index vacuuming (planned: 1) INFO: launched 1 parallel vacuum worker for index vacuuming (planned: 1) INFO: finished vacuuming "sbtest.public.sbtest1": index scans: 2 pages: 0 removed, 13921343 remain, 13921343 scanned (100.00% of total) tuples: 11014287 removed, 500000000 remain, 0 are dead but not yet removable removable cutoff: 74587243, which was 0 XIDs old when operation ended index scan needed: 11236460 pages from table (80.71% of total) had 18915133 dead item identifiers removed index "sbtest1_pkey": pages: 1370933 in total, 0 newly deleted, 0 currently deleted, 0 reusable index "k_1": pages: 1202589 in total, 0 newly deleted, 0 currently deleted, 0 reusable avg read rate: 102.408 MB/s, avg write rate: 99.348 MB/s buffer usage: 16918231 hits, 24908382 misses, 24164093 dirtied WAL usage: 48871286 records, 26289264 full page images, 210588376068 bytes system usage: CPU: user: 184.21 s, system: 275.86 s, elapsed: 1900.21 s VACUUM
- 64. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem - vacuum sbtest=# set maintenance_work_mem='1 GB’; sbtest=# vacuum (verbose) sbtest1; INFO: vacuuming "sbtest.public.sbtest1" INFO: launched 1 parallel vacuum worker for index vacuuming (planned: 1) INFO: finished vacuuming "sbtest.public.sbtest1": index scans: 1 pages: 0 removed, 13921343 remain, 13921343 scanned (100.00% of total) tuples: 10990324 removed, 500000000 remain, 0 are dead but not yet removable removable cutoff: 103987274, which was 0 XIDs old when operation ended new relfrozenxid: 53987274, which is 11901571 XIDs ahead of previous value index scan needed: 10126301 pages from table (72.74% of total) had 16347414 dead item identifiers removed index "sbtest1_pkey": pages: 1370933 in total, 0 newly deleted, 0 currently deleted, 0 reusable index "k_1": pages: 1202589 in total, 0 newly deleted, 0 currently deleted, 0 reusable avg read rate: 90.361 MB/s, avg write rate: 101.013 MB/s buffer usage: 16700415 hits, 22645066 misses, 25314603 dirtied WAL usage: 63975507 records, 26568355 full page images, 219969619018 bytes system usage: CPU: user: 195.27 s, system: 265.41 s, elapsed: 1957.86 s VACUUM
- 65. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. maintenance_work_mem - vacuum parallel sbtest=# set maintenance_work_mem='1 GB’; sbtest=# vacuum (verbose) sbtest1; INFO: vacuuming "sbtest.public.sbtest1" INFO: launched 2 parallel vacuum workers for index vacuuming (planned: 2)
- 66. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. autovacuum_work_mem max mem = autovacuum_work_mem X autovacuum_max_workers
- 67. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why you should care 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 5 105 205 305 405 505 605 705 805 905 1005 1105 1205 1305 1405 1505 1605 1705 Transactions Per Second (TPS) Seconds sysbench read only point selects Normal Tuning Less than Optimal Tuning 11X buffers not in memory swap thrashing
- 68. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Grant McAlister
Editor's Notes
- #28: More flexibility but more overhead and problems finding freeable blocks
- #32: Cold cache on restart PostgreSQL uses it for a lot of items, prefetch, lots of different buffering, temporary tables buffering, etc
- #42: Also note that hugepages are not swapable so less likely to get into thrashing
- #54: You can also have parallel which could make this a lot larger
- #58: Other objects can also use a lot of ram per session, older sessions need to be pruned
- #68: Try to keep the most important items in memory Minimize swapping (i.e. avoid thrashing) Avoid OOM (Out of Memory)