SlideShare a Scribd company logo

Generating & Querying Calendar Tables in Posgresql

Workhorse Computing, profile picture
Workhorse Computing

This document discusses PostgreSQL features for defining and working with date ranges and intervals to generate calendars and periods. It includes: 1) Defining calendars by starting with a base year and interval, generating date ranges for different period types like annual, quarterly, monthly etc. 2) Selecting date ranges from the generated calendar that contain or intersect with given dates. 3) Creating indexes on the calendar to efficiently search by date range.

Technology
1 of 53
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
PostgreSQL Ranges & Intervals: Data structures for calendars. Steven Lembark Workhorse Computing lembark@wrkhors.com
Generating a calendar. Periods by type: Annual, Quarterly. Effective window by start/end date. Hierarchy of periods: Quarters within Years.
Generating a calendar. Search by publication date. Classify by effective period. Search for documents by time. Define document expiration. Define searchable window.
Defining a calendar. Start with two relations: Start date + Calendar Size Period types. Fiscal calendar has arbitrary start. Wallclock calendar starts on 01-Jan.
Defining a calendar. Start with two relations: ( start, window ) ( type, size )
Defining a calendar. Start with two relations: ( date, interval ) ( enum, interval )
Defining a date “date” type. Stringy definitions: ’01-Feb-2021’ ’2021.02.01’ Has no time. Conversion to timestamp with ‘00:00:00’
Handling enums One way: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’month’ ) … );
Handling enums One way: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’month’ ) … ); Oops: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’mnoth’ ) … );
Handling enums Better way: create type date_t enum ( ‘year’, ‘month’ ); table foo ( date_type date_t, … ); table bar ( date_type date_t, … );
Handling enums Additions in one place: create type date_t enum ( ‘year’, ‘month’, ‘quarter’ ); table foo ( date_type date_t, … ); table bar ( date_type date_t, … );
Intervals in time Intervals define periods between dates and times. Offset timestamp, date, time values.
Intervals in time YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND
Intervals in time with rounding YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND
Interval input interval ‘1 year’ interval ‘2 years’ interval ‘2 years 1 month 3 days 5 hours 6 seconds’ PG handles leap year for you: ‘31-Jan-2020’::date + interval ‘29 days’
Definine the period window The definition of start date + window. Should have only one row. Minimal definition of start time. Minimal definition of end time.
Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: Single definition.
Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: First year: 2000 to current. Fiscal calendar uses date.
Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: First year: Interval: Defined in years.
Define time units Start with units of time. Convienent names: create type period_t as enum ( ‘annual’ , ‘quarterly’ , ‘monthly’ , ‘weekly’ , ‘daily’ ) ;
Define time units Start with units of time. Map name to intervals: create type period_t as enum ( ‘annual’ , ‘quarterly’ , ‘monthly’ , ‘weekly’ , ‘daily’ ) ; create table period_interval ( period_type period_t primary key , period_size interval not null , unique ( period ) );
Loading initial data Not much data: Integer + window. Few basic types. insert into period_window ( base_year , final_window ) values ( 2018, interval '10 years' ); insert into period_interval ( period_size , period_type ) values ( '1 year' , ‘annual’ ) , ( '3 months' , ‘quarterly’ ) , ( '1 month' , ‘monthly’ ) , ( ‘7 days’ , ‘weekly’ ) , ( '1 day' , ‘daily’ )
Generate a calendar “generated...” Replace sequence type. Allows cloning a table. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
Generate a calendar Re-use enum via type. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
Generate a calendar Periods are ranges. Periods are unique. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
First step: Start date. We have an integer: base_year. Intervals offset timestamps. select make_timestamp ( base_year, 1, 1, 0, 0, 0 ) "base_date" , final_window from period_window Generate a calendar
“Common Table Expression” “base_date” is a pseudo-table. Defined for one query. with base_date as ( select make_timestamp ( base_year, 1, 1, 0, 0, 0 ) "base_date" , final_window from period_window ) Generate a calendar
“cross join”: start timesteamp end interval name name width interval with base_date as ... ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
generate_series: Multiple rows. From base to final. Units of period interval. Generate a calendar with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z
‘annual’ , interval ‘1 year’ , ‘01-Jan-2020’ ... ‘annual’ , interval ‘1 year’ , ‘01-Jan-2021’ with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
‘quarterly’ , interval ‘3 months’ , ‘01-Jan-2020’ ... ‘annual’ , interval ‘3 months’ , ‘01-Apr-2020’ with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
Syntax: Sub-queries get names. with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
Sequence of values: “daterange” composite: lower upper with base_date as ... select period_type , daterange ( start::date , ( start + period )::date , '[)' ) from ( select period_type , period_size , generate_series ... “start” ... )z Generate a calendar
Result: Periods by width. period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
Result: ‘[ … , … )’ Non-inclusive upper. Ranges don’t intersect. period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
Result: ‘[ … , … )’ “2019-01-01” matches #2 & #16 on [2019-01-01, … ) period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
Selecting from the calendar “Range contains” ‘@>’ select period_sk , period_type , period from period where period @> '15-Mar-2021'::date order by period_type , period ;
Selecting from the calendar All period types. No match on: ‘15-Mar-2021’ ) select period_sk , period_type , period from period where period @> '15-Mar-2021'::date order by period_type , period ; period_sk | period_type | period -----------+-------------+------------------------- 4 | annual | [2021-01-01,2022-01-01) 24 | quarterly | [2021-01-01,2021-04-01) 91 | monthly | [2021-03-01,2021-04-01) 341 | weekly | [2021-03-15,2021-03-22) 1865 | single | [2021-03-15,2021-03-16) (5 rows)
Selecting from the calendar Intersecting: ‘&&’ select period_sk, period_type, period from curr_report_period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' order by period_type , period ; period_sk | period_type | period -----------+-------------+------------------------- 24 | quarterly | [2021-01-01,2021-04-01) 25 | quarterly | [2021-04-01,2021-07-01) 26 | quarterly | [2021-07-01,2021-10-01) 27 | quarterly | [2021-10-01,2022-01-01) 28 | quarterly | [2022-01-01,2022-04-01) 29 | quarterly | [2022-04-01,2022-07-01) 30 | quarterly | [2022-07-01,2022-10-01)
Indexing the calendar GiST indexes describe arbitrary trees. Originally designed for full-text search. Search where ‘=’ doesn’t apply.
Indexing the calendar GiST indexes Good for ranges and areas. create index on period using gist ( period ) include ( period_sk ) where active ; create index on period using gist ( period , period_type ) include ( period_sk ) where active ;
Indexing the calendar GiST indexes Adds un-indexed data to the index leaf nodes. Query on period gets SK. create index on period using gist ( period ) include ( period_sk ) where active ; create index on period using gist ( period , period_type ) include ( period_sk ) where active ;
Using the index Find partitions with quarterly reports including these dates. explain select period_sk from period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' ;
Using the index Find partitions with quarterly reports including these dates. Partition of data can use period_sk. explain select period_sk from period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' ;
Plan without index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows)
Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows) Works in PG: Data types & extensibility. Range type & GiST index.
Summary Tables for window, intervals: Declarative definition of data. No iteration, no loops. Just sets & SQL.
Summary Types for calendar: interval daterange & tsrange & tstzrange Series of values: generate_series( start, end, step );
Summary SQL: Common Table Expression (“CTE”). Inlined in Postgres – fast. Name subqueries. Cross-join == Cartesian Product.
Bedside reading https://www.postgresql.org/docs/13/... textsearch-indexes.html GIN & GiST rangetypes.html Data types datatype-datetime.html Date & Time spec’s functions-datetime.html Date op’s & func’s. functions-range.html Range op’s & func’s.

More Related Content

PDF
SessionSeven_WorkingWithDatesandTime
Hellen Gakuruh
 
DOCX
Builtinfunctions in vbscript and its types.docx
Ramakrishna Reddy Bijjam
 
PDF
How to work with dates and times in swift 3
allanh0526
 
DOCX
Design a class named Month. The class should have the following priv
trappiteboni
 
PDF
17 ruby date time
Walker Maidana
 
PPTX
YTD, QTD & MTD.pptx
Sandeep Bhaskar
 
PDF
Python and PostgreSQL: Let's Work Together! | PyConFr 2018 | Dimitri Fontaine
Citus Data
 
PDF
Dax queries.pdf
ntrnbk
 
SessionSeven_WorkingWithDatesandTime
Hellen Gakuruh
 
Builtinfunctions in vbscript and its types.docx
Ramakrishna Reddy Bijjam
 
How to work with dates and times in swift 3
allanh0526
 
Design a class named Month. The class should have the following priv
trappiteboni
 
17 ruby date time
Walker Maidana
 
YTD, QTD & MTD.pptx
Sandeep Bhaskar
 
Python and PostgreSQL: Let's Work Together! | PyConFr 2018 | Dimitri Fontaine
Citus Data
 
Dax queries.pdf
ntrnbk
 

Similar to Generating & Querying Calendar Tables in Posgresql (20)

PDF
Change to oop formatimport java.util.Scanner;import java.io.;.pdf
MAYANKBANSAL1981
 
PPT
Project presentation(View calender)
Ikhtiar Khan Sohan
 
PDF
Cara membuat time schedule dan kurva s
SugarPlum8
 
PPTX
Date and time functions.pptx
NaveenChilukuri5
 
PDF
LCC Asia Pacific_financialmodelling_excelshortcuts1
LCC Asia Pacific Corporate Finance
 
PDF
Visual Programming Lacture Nine 9 Structure.pdf
ubaidullahkhan5546
 
PPT
27. mathematical, date and time functions in VB Script
VARSHAKUMARI49
 
PPTX
MYSQL DATA TYPES for newbies helpful.pptx
Farhat991731
 
PPTX
How To Create Calendar View In Odoo 16
Celine George
 
PPTX
Advanced Views - Calendar View in Odoo 17
Celine George
 
PDF
Data Wrangling: Working with Date / Time Data and Visualizing It
kanaugust
 
PPT
AIN102D Access date functions sample queries
Dan D'Urso
 
PDF
Calculation Groups - color 1 slide per page.pdf
PBIMINERADC
 
PPTX
TSQL Functions (SQL Server)
Steve Stedman
 
PDF
Date and Timestamp Types In Snowflake (By Faysal Shaarani)
Faysal Shaarani (MBA)
 
PPTX
Intro to DAX Patterns
Eric Bragas
 
PPTX
Date object.pptx date and object v
22x026
 
PDF
Business Intelligence Portfolio
eileensauer
 
PDF
Business Intelligence Portfolio
eileensauer
 
PPTX
unit 5_Real time Data Analysis vsp.pptx
prakashvs7
 
Change to oop formatimport java.util.Scanner;import java.io.;.pdf
MAYANKBANSAL1981
 
Project presentation(View calender)
Ikhtiar Khan Sohan
 
Cara membuat time schedule dan kurva s
SugarPlum8
 
Date and time functions.pptx
NaveenChilukuri5
 
LCC Asia Pacific_financialmodelling_excelshortcuts1
LCC Asia Pacific Corporate Finance
 
Visual Programming Lacture Nine 9 Structure.pdf
ubaidullahkhan5546
 
27. mathematical, date and time functions in VB Script
VARSHAKUMARI49
 
MYSQL DATA TYPES for newbies helpful.pptx
Farhat991731
 
How To Create Calendar View In Odoo 16
Celine George
 
Advanced Views - Calendar View in Odoo 17
Celine George
 
Data Wrangling: Working with Date / Time Data and Visualizing It
kanaugust
 
AIN102D Access date functions sample queries
Dan D'Urso
 
Calculation Groups - color 1 slide per page.pdf
PBIMINERADC
 
TSQL Functions (SQL Server)
Steve Stedman
 
Date and Timestamp Types In Snowflake (By Faysal Shaarani)
Faysal Shaarani (MBA)
 
Intro to DAX Patterns
Eric Bragas
 
Date object.pptx date and object v
22x026
 
Business Intelligence Portfolio
eileensauer
 
Business Intelligence Portfolio
eileensauer
 
unit 5_Real time Data Analysis vsp.pptx
prakashvs7
 
Ad

More from Workhorse Computing (20)

PDF
Object::Trampoline: Follow the bouncing object.
Workhorse Computing
 
PDF
Wheels we didn't re-invent: Perl's Utility Modules
Workhorse Computing
 
PDF
mro-every.pdf
Workhorse Computing
 
PDF
Paranormal statistics: Counting What Doesn't Add Up
Workhorse Computing
 
PDF
The $path to knowledge: What little it take to unit-test Perl.
Workhorse Computing
 
PDF
Unit Testing Lots of Perl
Workhorse Computing
 
PDF
Hypers and Gathers and Takes! Oh my!
Workhorse Computing
 
PDF
BSDM with BASH: Command Interpolation
Workhorse Computing
 
PDF
Findbin libs
Workhorse Computing
 
PDF
Memory Manglement in Raku
Workhorse Computing
 
PDF
BASH Variables Part 1: Basic Interpolation
Workhorse Computing
 
PDF
Effective Benchmarks
Workhorse Computing
 
PDF
Metadata-driven Testing
Workhorse Computing
 
PDF
The W-curve and its application.
Workhorse Computing
 
PDF
Keeping objects healthy with Object::Exercise.
Workhorse Computing
 
PDF
Perl6 Regexen: Reduce the line noise in your code.
Workhorse Computing
 
PDF
Smoking docker
Workhorse Computing
 
PDF
Getting Testy With Perl6
Workhorse Computing
 
PDF
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Workhorse Computing
 
PDF
Neatly folding-a-tree
Workhorse Computing
 
Object::Trampoline: Follow the bouncing object.
Workhorse Computing
 
Wheels we didn't re-invent: Perl's Utility Modules
Workhorse Computing
 
mro-every.pdf
Workhorse Computing
 
Paranormal statistics: Counting What Doesn't Add Up
Workhorse Computing
 
The $path to knowledge: What little it take to unit-test Perl.
Workhorse Computing
 
Unit Testing Lots of Perl
Workhorse Computing
 
Hypers and Gathers and Takes! Oh my!
Workhorse Computing
 
BSDM with BASH: Command Interpolation
Workhorse Computing
 
Findbin libs
Workhorse Computing
 
Memory Manglement in Raku
Workhorse Computing
 
BASH Variables Part 1: Basic Interpolation
Workhorse Computing
 
Effective Benchmarks
Workhorse Computing
 
Metadata-driven Testing
Workhorse Computing
 
The W-curve and its application.
Workhorse Computing
 
Keeping objects healthy with Object::Exercise.
Workhorse Computing
 
Perl6 Regexen: Reduce the line noise in your code.
Workhorse Computing
 
Smoking docker
Workhorse Computing
 
Getting Testy With Perl6
Workhorse Computing
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Workhorse Computing
 
Neatly folding-a-tree
Workhorse Computing
 
Ad

Recently uploaded (20)

PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
PPTX
Machine Learning_overview_presentation.pptx
kondavamsidharreddy2
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
Getting Started with Data Integration: FME Form 101
Safe Software
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PPTX
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
AgnesMunisi
 
MYSQL Presentation for SQL database connectivity
Swati270511
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Encapsulation theory and applications.pdf
gurumoop
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
Machine Learning_overview_presentation.pptx
kondavamsidharreddy2
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
Getting Started with Data Integration: FME Form 101
Safe Software
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
Approach and Philosophy of On baking technology
30anthuong17
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
SOPHOS-XG Firewall Administrator PPT.pptx
AgnesMunisi
 

Generating & Querying Calendar Tables in Posgresql

  • 1. PostgreSQL Ranges & Intervals: Data structures for calendars. Steven Lembark Workhorse Computing [email protected]
  • 2. Generating a calendar. Periods by type: Annual, Quarterly. Effective window by start/end date. Hierarchy of periods: Quarters within Years.
  • 3. Generating a calendar. Search by publication date. Classify by effective period. Search for documents by time. Define document expiration. Define searchable window.
  • 4. Defining a calendar. Start with two relations: Start date + Calendar Size Period types. Fiscal calendar has arbitrary start. Wallclock calendar starts on 01-Jan.
  • 5. Defining a calendar. Start with two relations: ( start, window ) ( type, size )
  • 6. Defining a calendar. Start with two relations: ( date, interval ) ( enum, interval )
  • 7. Defining a date “date” type. Stringy definitions: ’01-Feb-2021’ ’2021.02.01’ Has no time. Conversion to timestamp with ‘00:00:00’
  • 8. Handling enums One way: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’month’ ) … );
  • 9. Handling enums One way: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’month’ ) … ); Oops: table foo ( date_type enum ( ‘year’,’month’ ) … ); table bar ( date_type enum ( ‘year’,’mnoth’ ) … );
  • 10. Handling enums Better way: create type date_t enum ( ‘year’, ‘month’ ); table foo ( date_type date_t, … ); table bar ( date_type date_t, … );
  • 11. Handling enums Additions in one place: create type date_t enum ( ‘year’, ‘month’, ‘quarter’ ); table foo ( date_type date_t, … ); table bar ( date_type date_t, … );
  • 12. Intervals in time Intervals define periods between dates and times. Offset timestamp, date, time values.
  • 13. Intervals in time YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND
  • 14. Intervals in time with rounding YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND
  • 15. Interval input interval ‘1 year’ interval ‘2 years’ interval ‘2 years 1 month 3 days 5 hours 6 seconds’ PG handles leap year for you: ‘31-Jan-2020’::date + interval ‘29 days’
  • 16. Definine the period window The definition of start date + window. Should have only one row. Minimal definition of start time. Minimal definition of end time.
  • 17. Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: Single definition.
  • 18. Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: First year: 2000 to current. Fiscal calendar uses date.
  • 19. Definine the period window create table period_window ( one_row integer not null primary key default 0 check( one_row = 0 ) , base_year integer not null check ( base_year > 2000 and base_year <= date_part('year', now() ) ) , final_window interval year not null check ( final_window > interval '0 year' and final_window < interval '20 year' ) ); One row: First year: Interval: Defined in years.
  • 20. Define time units Start with units of time. Convienent names: create type period_t as enum ( ‘annual’ , ‘quarterly’ , ‘monthly’ , ‘weekly’ , ‘daily’ ) ;
  • 21. Define time units Start with units of time. Map name to intervals: create type period_t as enum ( ‘annual’ , ‘quarterly’ , ‘monthly’ , ‘weekly’ , ‘daily’ ) ; create table period_interval ( period_type period_t primary key , period_size interval not null , unique ( period ) );
  • 22. Loading initial data Not much data: Integer + window. Few basic types. insert into period_window ( base_year , final_window ) values ( 2018, interval '10 years' ); insert into period_interval ( period_size , period_type ) values ( '1 year' , ‘annual’ ) , ( '3 months' , ‘quarterly’ ) , ( '1 month' , ‘monthly’ ) , ( ‘7 days’ , ‘weekly’ ) , ( '1 day' , ‘daily’ )
  • 23. Generate a calendar “generated...” Replace sequence type. Allows cloning a table. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
  • 24. Generate a calendar Re-use enum via type. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
  • 25. Generate a calendar Periods are ranges. Periods are unique. create table period ( period_sk integer generated by default as identity primary key , period_type period_t references period_interval , period daterange not null , unique ( period ) );
  • 26. First step: Start date. We have an integer: base_year. Intervals offset timestamps. select make_timestamp ( base_year, 1, 1, 0, 0, 0 ) "base_date" , final_window from period_window Generate a calendar
  • 27. “Common Table Expression” “base_date” is a pseudo-table. Defined for one query. with base_date as ( select make_timestamp ( base_year, 1, 1, 0, 0, 0 ) "base_date" , final_window from period_window ) Generate a calendar
  • 28. “cross join”: start timesteamp end interval name name width interval with base_date as ... ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
  • 29. generate_series: Multiple rows. From base to final. Units of period interval. Generate a calendar with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z
  • 30. ‘annual’ , interval ‘1 year’ , ‘01-Jan-2020’ ... ‘annual’ , interval ‘1 year’ , ‘01-Jan-2021’ with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
  • 31. ‘quarterly’ , interval ‘3 months’ , ‘01-Jan-2020’ ... ‘annual’ , interval ‘3 months’ , ‘01-Apr-2020’ with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
  • 32. Syntax: Sub-queries get names. with base_date as ... from ( select period_type , period , generate_series ( base_date , base_date + final_window , period_size ) "start" from base_date cross join period_interval ) z Generate a calendar
  • 33. Sequence of values: “daterange” composite: lower upper with base_date as ... select period_type , daterange ( start::date , ( start + period )::date , '[)' ) from ( select period_type , period_size , generate_series ... “start” ... )z Generate a calendar
  • 34. Result: Periods by width. period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
  • 35. Result: ‘[ … , … )’ Non-inclusive upper. Ranges don’t intersect. period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
  • 36. Result: ‘[ … , … )’ “2019-01-01” matches #2 & #16 on [2019-01-01, … ) period_sk | period_type | period ----------+-------------+------------------------- 1 | annual | [2018-01-01,2019-01-01) 2 | annual | [2019-01-01,2020-01-01) 3 | annual | [2020-01-01,2021-01-01) 4 | annual | [2021-01-01,2022-01-01) 5 | annual | [2022-01-01,2023-01-01) 6 | annual | [2023-01-01,2024-01-01) 7 | annual | [2024-01-01,2025-01-01) 8 | annual | [2025-01-01,2026-01-01) 9 | annual | [2026-01-01,2027-01-01) 10 | annual | [2027-01-01,2028-01-01) 11 | annual | [2028-01-01,2029-01-01) 12 | quarterly | [2018-01-01,2018-04-01) 13 | quarterly | [2018-04-01,2018-07-01) 14 | quarterly | [2018-07-01,2018-10-01) 15 | quarterly | [2018-10-01,2019-01-01) 16 | quarterly | [2019-01-01,2019-04-01) 17 | quarterly | [2019-04-01,2019-07-01) 18 | quarterly | [2019-07-01,2019-10-01) Generate a calendar
  • 37. Selecting from the calendar “Range contains” ‘@>’ select period_sk , period_type , period from period where period @> '15-Mar-2021'::date order by period_type , period ;
  • 38. Selecting from the calendar All period types. No match on: ‘15-Mar-2021’ ) select period_sk , period_type , period from period where period @> '15-Mar-2021'::date order by period_type , period ; period_sk | period_type | period -----------+-------------+------------------------- 4 | annual | [2021-01-01,2022-01-01) 24 | quarterly | [2021-01-01,2021-04-01) 91 | monthly | [2021-03-01,2021-04-01) 341 | weekly | [2021-03-15,2021-03-22) 1865 | single | [2021-03-15,2021-03-16) (5 rows)
  • 39. Selecting from the calendar Intersecting: ‘&&’ select period_sk, period_type, period from curr_report_period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' order by period_type , period ; period_sk | period_type | period -----------+-------------+------------------------- 24 | quarterly | [2021-01-01,2021-04-01) 25 | quarterly | [2021-04-01,2021-07-01) 26 | quarterly | [2021-07-01,2021-10-01) 27 | quarterly | [2021-10-01,2022-01-01) 28 | quarterly | [2022-01-01,2022-04-01) 29 | quarterly | [2022-04-01,2022-07-01) 30 | quarterly | [2022-07-01,2022-10-01)
  • 40. Indexing the calendar GiST indexes describe arbitrary trees. Originally designed for full-text search. Search where ‘=’ doesn’t apply.
  • 41. Indexing the calendar GiST indexes Good for ranges and areas. create index on period using gist ( period ) include ( period_sk ) where active ; create index on period using gist ( period , period_type ) include ( period_sk ) where active ;
  • 42. Indexing the calendar GiST indexes Adds un-indexed data to the index leaf nodes. Query on period gets SK. create index on period using gist ( period ) include ( period_sk ) where active ; create index on period using gist ( period , period_type ) include ( period_sk ) where active ;
  • 43. Using the index Find partitions with quarterly reports including these dates. explain select period_sk from period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' ;
  • 44. Using the index Find partitions with quarterly reports including these dates. Partition of data can use period_sk. explain select period_sk from period where period && daterange( '15-Mar-2021', '20-Aug-2022' ) and period_type = 'quarterly' ;
  • 45. Plan without index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows)
  • 46. Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
  • 47. Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
  • 48. Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Bitmap Heap Scan on period (cost=15.87..80.54 rows=1 width=4) Recheck Cond: active Filter: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) -> Bitmap Index Scan on period_period_sk_idx (cost=0.00..15.87 rows=1578 width=0) (4 rows) QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows)
  • 49. Plan with index QUERY PLAN ------------------------------------------------------------------------------------------- Index Only Scan using period_period_period_type_period_sk_idx on period (cost=0.15..2.37 rows=1 width=4) Index Cond: ((period && '[2021-03-15,2022-08-20)'::daterange) AND (period_type = 'quarterly'::period_t)) (2 rows) Works in PG: Data types & extensibility. Range type & GiST index.
  • 50. Summary Tables for window, intervals: Declarative definition of data. No iteration, no loops. Just sets & SQL.
  • 51. Summary Types for calendar: interval daterange & tsrange & tstzrange Series of values: generate_series( start, end, step );
  • 52. Summary SQL: Common Table Expression (“CTE”). Inlined in Postgres – fast. Name subqueries. Cross-join == Cartesian Product.
  • 53. Bedside reading https://www.postgresql.org/docs/13/... textsearch-indexes.html GIN & GiST rangetypes.html Data types datatype-datetime.html Date & Time spec’s functions-datetime.html Date op’s & func’s. functions-range.html Range op’s & func’s.