John Melesky - Federating Queries Using Postgres FDW @ Postgres Open

Federating Queries Using postgres fdw
john melesky
Rentrak, Inc
September 17, 2013

Who Am I?
A long-time programmer, working with PostgreSQL in the
cloud

Who Am I?
cloud my butt

Who Am I?
cloud my butt
Now, a DBA, working with PostgreSQL on real machines with
real disks

Who Am I?
cloud my butt
Now, a DBA, working with PostgreSQL on real machines
VMWare with real disks

Who Am I?
cloud my butt
Now, a DBA, working with PostgreSQL on real machines
VMWare with real disks NetApps

PostgreSQL inheritance partitioning
create table transactions (
id serial,
user_id bigint,
time_utc timestamp,
int_value bigint,
txt_value text,
primary key (id)
);
create table transactions_201306 (
like transactions including indexes,
check
(time_utc >= ’2013-06-01’ and
time_utc < ’2013-07-01’)
) inherits (transactions);

create table transactions (
id serial,
user_id bigint,
time_utc timestamp,
int_value bigint,
txt_value text,
primary key (id)
);
create table transactions_201306 (
like transactions including indexes,
check
(time_utc >= ’2013-06-01’ and
time_utc < ’2013-07-01’)
) inherits (transactions);
You know this already

Old-school partitioning
create view transactions as (
select * from transactions_201301
union all
union all
union all
union all
...
);

Why don’t we still use this?

1. No insert triggers on views

2. No ”inherit indexes” without additional misdirection

2. No ”inherit indexes” without additional misdirection
3. Basically, we have a better option with inheritence partitioning

Postgres Foreign Data Wrapper
-- just once
create extension postgres_fdw;
-- once per data node
create server node0 foreign data wrapper postgres_fdw
options (connection stuff);
create user mapping for app_user server node0;
-- once per table per node
create foreign table transactions_node0
(table definition)
server node0
options (table_name ’transactions’);

Federating, Old-school
create view transactions as (
select * from transactions_node0
union all
union all
union all
union all
...
);

Querying
primary=# explain select count(*) from transactions;
QUERY PLAN
---------------------------------------------------------
Aggregate (cost=1767.38..1767.39 rows=1 width=0)
-> Append (cost=100.00..1699.12 rows=27304 width=0)
-> Foreign Scan on transactions_node0
(cost=100.00..212.39 rows=3413 width=0)
(cost=100.00..212.39 rows=3413 width=0)
(cost=100.00..212.39 rows=3413 width=0)
(cost=100.00..212.39 rows=3413 width=0)
(cost=100.00..212.39 rows=3413 width=0)
...
(10 rows)
Time: 1.226 ms

Querying
primary=# explain verbose select count(*) from transactions;
QUERY PLAN
-----------------------------------------------------------------
Output: count(*)
-> Foreign Scan on public.transactions_node0
(cost=100.00..212.39 rows=3413 width=0)
Remote SQL: SELECT NULL FROM public.transactions
(cost=100.00..212.39 rows=3413 width=0)
(cost=100.00..212.39 rows=3413 width=0)
...
(19 rows)
Time: 1.273 ms

Querying
primary=# select count(*) from transactions;
count
---------
1095336
(1 row)
Time: 3035.054 ms

Round-robin
primary
node 0
(id % 4 = 0)
node 1
(id % 4 = 1)
node 2
(id % 4 = 2)
node 3
(id % 4 = 3)

Round-robin
primary=# create foreign table transactions_node0 (
primary(# id serial,
primary(# user_id bigint,
primary(# time_utc timestamp,
primary(# int_value bigint,
primary(# txt_value text,
primary(# check ((id % 8) = 0)
primary(# ) server node0
primary(# options (table_name ’transactions’);
ERROR: constraints are not supported on foreign tables
LINE 6: check ((id % 8) = 0)) server node0 ...

Domain-based (aka ”sharding”)
primary
node 0
(customer = 'bigone')
node 1
(customer in ('bigtwo', 'bigthree')
node 2
(customer in (...))
node 3
(customer in (...))

Range-based
primary
node 0
(date between '2013-01-01' and '2013-01-31')
node 1
(date between ...)
node 2
(date between ...)
node 3
date between ...)

Table-based
primary
node 0
(users table(s))
node 1
(transactions table)
node 2
(session tables)

Multi-head
primary1
node0 node1node2 node3
primary2

Multi-head
primary1
node0 node1node2 node3
primary2primary3 primary4

Limitations: Network traﬃc
primary=# select count(*) from transactions_local;
count
---------
1095336
(1 row)
Time: 209.097 ms

Limitations: Network traﬃc
primary=# select count(*) from transactions_local;
count
---------
1095336
(1 row)
Time: 209.097 ms
primary=# select count(*) from transactions_primary;
count
---------
1095336
(1 row)
Time: 2867.385 ms

Limitations: Dumb queries
QUERY PLAN
-----------------------------------------------------------------
Output: count(*)
(cost=100.00..212.39 rows=3413 width=0)
...

QUERY PLAN
-----------------------------------------------------------------
Output: count(*)
(cost=100.00..212.39 rows=3413 width=0)
...
primary=# explain verbose select avg(int_value) from transactions;
QUERY PLAN
--------------------------------------------------------------------------
Output: avg(transactions_node0.int_value)
(cost=100.00..186.80 rows=2560 width=8)
Output: transactions_node0.int_value
Remote SQL: SELECT int_value FROM public.transactions
...

select type, count(*)
from users
group by type
order by 2 desc;

Limitations: Joins
select count(*)
from transactions t, users u
where t.user_id = u.id
and u.type = ’mistaken’;

Limitations: Keys
’Nuﬀ said

Limitations: Constraint exclusion
Remember this?
ERROR: constraints are not supported on foreign tables
LINE 6: check ((id % 8) = 0)) server node0 ....

Limitations: Single-threaded executer

How many nodes do you have?

How many nodes do you have?
Do you know what they’re doing?

Strategies
Large working set, small nodes
Node-level partitioning
Heavy distributed processing
Multi-head

Strategy: Large working set, small nodes

Your working set is larger than one node’s RAM

... but you have lots of nodes

(and network is faster than disk)

(and network is faster than disk)
This might be worth looking into if you’re on AWS, but
please, please test it ﬁrst

Strategy: Node-level partitioning
Like parititioning, but with a separate node per partition group!

Strategy: Node-level partitioning
Like parititioning, but with a separate node per partition group!
As a total strategy, this is probably not worthwhile. However, it
can work with a fast ”current data” node combining with slower
”archived data” nodes.

Take advantage of lots of CPUs

Works well when you have node-discrete workloads

Lock management can become a bit hairier

Lock management can become a bit hairier
This might actually be a useful use case

Multi-headed
Like replication, but with no overhead or delay!

Multi-headed
Also, no storage overhead!

Multi-headed
Might work well with the distributed processing setup

Multi-headed
Might work well with the distributed processing setup
In fact, given the overhead that lands on the head node, it
might be necessary for a working FDW federation setup

Pan-Strategy Advice
Think very carefully about what tables should live where

Pan-Strategy Advice
Think very carefully about tuning settings (especially on your
head node)
work mem
shared buﬀers
temp buﬀers

Pan-Strategy Advice
head node)
work mem
shared buﬀers
temp buﬀers
Think very carefully about how many data nodes you want

Pan-Strategy Advice
head node)
work mem
shared buﬀers
temp buﬀers
Think very carefully about network vs. disk vs. dumb-query
costs

Pan-Strategy Advice
head node)
work mem
shared buﬀers
temp buﬀers
Think very carefully about network vs. disk vs. dumb-query
costs
Think very carefully!

Questions?
Any questions?
John, do you use this approach for your databases?

Questions?
Any questions?
John, do you use this approach for your databases?
Why not?

Thanks!
Plug: Stephen Frost has another postgres fdw talk tomorrow

Thanks!
Plug: Stephen Frost has another postgres fdw talk tomorrow
Also: Rentrak is hiring: programmers, sysadmins, and devops

Federating Queries Using postgres fdw
Introduction
Who am I?
Partitioning
Old-school partitioning
Federating Queries
Federation Strategies Overview
Trial and Error
Demo
Limitations
Strategies
Wrap-up

John Melesky - Federating Queries Using Postgres FDW @ Postgres Open

Recommended

More Related Content

What's hot (20)

Similar to John Melesky - Federating Queries Using Postgres FDW @ Postgres Open (20)

More from PostgresOpen (18)

Recently uploaded (20)

John Melesky - Federating Queries Using Postgres FDW @ Postgres Open