Data Organisation: Table Partitioning in PostgreSQL

Mastering data organisation:
Deep dive into PostgreSQL table partitioning
Presented by
Mohammad Zaid Patel
Mydbops
Mydbops MyWebinar - 30
Dec 23rd, 2023

About Me
● PostgreSQL Database consultant
● Experienced in PostgreSQL and related DB
technologies
● Active Blogger
● Tech Speaker
● Likes Cricket, Music & Comedy

Consulting
Services
Consulting
Services
Managed
Services
● Database Management and
consultancy provider
● Founded in 2016
● Assisted 800+ happy customers
● AWS partners
● PCI & ISO certiﬁed
About Us

● Why data organisation ?
● Advantages of data organisation
● Techniques of data organisation
● Table partitioning in PostgreSQL
● Table partition techniques
● Demo for table partitioning using pg_partman
● Limitations of table partitioning
● Best practices for table partitioning
Agenda

● Databases are used for storing and retrieving data
● Database becomes a black box
● Piled up with un-organised data
● Performance degradation of the database
Why data organization?

● Organization of data is very essential for
the database functionality
● Plays crucial roles in database eﬃciency
● Assures data integrity
● Improves the usability of the stored data
Why data organization?

Unorganized data
Organized data

Advantages of organizing your data

Advantages of organizing your data
● Better data retrieval
● Improved query performance
● Data integrity
● Eﬃcient storage and resource utilization
● Ease in maintenance
● Ease in scalability of the database
● Data analysis and reporting

Data organization techniques
Index creation :
- Creates a data structure that allows quick access to rows based
on specific column values
- Faster data retrieval for conditions involving indexed columns
- Types: B-tree, GIN,Hash indexes etc
Data Archival:
- Process of moving the old data to a different location
- Clean up database
- Helps in managing the data efficiently over time

Schemas:
- Collection of database objects organized into a named namespace
- Provides a way to logically group and organize database objects within a
database
- Multiple schemas with each schema having its own set of objects.
Functional naming of database objects:
- Helps in noting the type of database objects e.g. indexes, views ,
sequences etc
- Easier for the users to understand the database structure and work with
the objects more eﬀectively

Relationships among database objects:
- Avoids creation of "orphaned" records and maintaining consistency across
related tables.
- Eﬃcient queries that involve data from multiple tables like JOIN
operations
Table partitioning:
- Involves dividing large tables into smaller, more manageable pieces
- Improves query performance, data management, and maintenance tasks
- Data distribution across partitions.

Table Partitioning in PostgreSQL

● Database design technique that involves dividing large tables into smaller,
more manageable segments called partitions
id Name Cricket club
01 Virat RCB
02 Dhoni CSK
03 Rohit MI
04 Hardik MI
05 Siraj RCB
06 Jadeja CSK
id Name Cricket club
01 Virat RCB
05 Siraj RCB
02 Dhoni CSK
06 Jadeja CSK
03 Rohit MI
04 Hardik MI
Child table-1
Child table-2
Child table-3
Normal Table Partitioned Table
Parent table

● Partition key, determines how data is distributed across partitions
● Eﬃcient data management, especially for operations like data insertion, updates, and
deletions.
● Enhances query performance by allowing the database to selectively scan only
relevant partitions when queries involve conditions on the partition key
● Ease in data archival

Types of table partitioning in PostgreSQL

PostgreSQL supports diﬀerent partitioning methods including range partitioning , list
partitioning, and hash partitioning.
Range Partitioning:
- Divides a table into partitions based on ranges of values in a chosen partition key
column
- Partitioning by a date column, each partition may represent a speciﬁc time period,
such as months or years
Creating a child table :
CREATE TABLE child_table_2022_01_01 PARTITION OF
parent_table
FOR VALUES FROM ('2022-01-01') TO ('2023-02-01');

id Username date_of_creation
01 Jake 01-01-2023
05 Ryan 01-01-2023
02 Anne 02-01-2023
06 Austin 02-01-2023
03 Daniel 03-01-2023
04 Taylor 03-01-2023
Child table-1
Child table-2
Child table-3
Range based partitioned table

List based Partitioning:
- Divides a table into partitions based on speciﬁc values in a chosen partition key
column.
- Rows with values falling within speciﬁc ranges of this key are grouped into individual
partitions
CREATE TABLE new_york_child_table PARTITION OF parent_table
FOR VALUES IN (‘New York’);

id Username City
01 Jake New York
05 Ryan New York
02 Anne Tokyo
06 Austin Tokyo
03 Daniel London
04 Taylor London
Child table-1
Child table-2
Child table-3
List based partitioned table

Hash based Partitioning:
- Divides a table into partitions based on the hash value of a chosen partition key
column
- Uniform distribution of data across partitions by using a hash function
CREATE TABLE child_table_1 PARTITION OF parent_table FOR
VALUES WITH (MODULUS 3,REMAINDER 0);

id Username City
01 Anne New York
05 Taylor Tokyo
02 Jake Tokyo
06 Austin London
03 Daniel London
04 Ryan New York
Child table-1
Child table-2
Child table-3
List based partitioned table
Remainder
value
0001
0001
0002
0002
0003
0003

Partitioning techniques in PostgreSQL

1. Manual Partitioning:
- Creating parent tables and child tables manually
- Maintenance of the child tables should be taken care of
E.g
Creating a parent table :
CREATE TABLE parent_table (
id int,date_column DATE, value INTEGER,
CONSTRAINT pkey PRIMARY KEY (id,date_column)
) PARTITION BY RANGE (date_column);

E.g
CREATE TABLE child_table_2022 PARTITION OF parent_table
FOR VALUES FROM ('2022-01-01') TO ('2023-01-01');

2. Partitioning using pg_partman extension:
- Extension designed to create and oversee sets of partitioned tables
- The creation of child tables is fully managed by the extension itself
- The extension includes a background worker (BGW) process to streamline partition
maintenance
- Optional retention policy that can automatically discard partitions that are no longer
necessary
- Maintenance function takes cares of partition management on
timely basis

Creating a parent table :
CREATE TABLE parent_table (
id integer ,created_date date,
CONSTRAINT table01_pkey PRIMARY KEY
(id,created_date)
) partition by range(created_date);
SELECT partman.create_parent( p_parent_table =>
'public.parent_table’, p_control => 'created_date',
p_interval=> '1 day', p_premake => 2);

Limitations of Table partitioning

Limitations of Table partitioning
● Useful only when the partition key is used
● Regular supervision is required for the pg_partman tool for partition management
● Complexities in data migration services
● Complexities in tables with foreign key relationships
● Child tables that are stored as backup needs to be taken care
● Data accumulation in default table fails the pg_partman functions

Best practices for partitioned table maintenance

● Choose the Right Partition Key
● Understand Query Patterns
● Monitor and Tune Performance
● Choose Appropriate Partitioning Method
● Limit the Number of Partitions
● Implement Data Archiving
● Regularly Update PostgreSQL Version
Best practices for partitioned table maintenance

Consulting
Services
Consulting
Services
Connect with us !
Reach us at : info@mydbops.com

Data Organisation: Table Partitioning in PostgreSQL

Recommended

More Related Content

Similar to Data Organisation: Table Partitioning in PostgreSQL (20)

More from Mydbops (20)

Recently uploaded (20)

Data Organisation: Table Partitioning in PostgreSQL