PostgreSQL and CockroachDB SQL

@cockroachdb
PostgreSQL meetup, November 2015
CockroachDB
presented by Peter Mattis / Co-Founder

@cockroachdb
1.Overview of CockroachDB
2.SQL Data Model
3.Logical Data Storage
4.Online/Concurrent Schema Change
Agenda

@cockroachdb
What is CockroachDB?
■Scale out SQL
■Distributed
■Survivable
■Consistent
■Open source

@cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
SQL
Transactional KV
Monolithic Map
Raft

@cockroachdb
Transactional KV
Monolithic Map
Raft
GraphSQL

@cockroachdb
Transactional KV
Monolithic Map
Raft
SQL Graph Doc

@cockroachdb
Transactional KV
Monolithic Map
Raft
SQL

@cockroachdb
Transactional KV
Monolithic Map
Raft
SQL
Physical

@cockroachdb
■Tables
SQL Data Model

@cockroachdb
■Tables
SQL Data Model
Inventory

@cockroachdb
■Tables
■Rows
SQL Data Model
Inventory

@cockroachdb
■Tables
■Rows
■Columns
SQL Data Model
Inventory
ID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66

@cockroachdb
■Tables
■Rows
■Columns
■Indexes
SQL Data Model
Inventory
ID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66
Name
Ball
Bat
Glove
Shirt
Shoes
Shorts
Name_Idx

@cockroachdb
PostgreSQL: Logical Data Storage

@cockroachdb
■Rows are stored in an unordered heap
■Indexes are btrees
■Primary key is a unique index
PostgreSQL: Data Storage

@cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
PostgreSQL: Example Table

@cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);

@cockroachdb
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
test (heap)

@cockroachdb
(0, 1) (1, “ball”, 3.33)
Index Key Tuple ID
1 (0, 1)
test (heap)test_pkey (btree)

@cockroachdb
INSERT INTO test VALUES (2, “glove”, 4.44);
(0, 1) (1, “ball”, 3.33)
(0, 2) (2, “glove”, 4.44)
Index Key Tuple ID
1 (0, 1)
2 (0, 2)
test (heap)test_pkey (btree)

@cockroachdb
CockroachDB: Logical Data Storage

@cockroachdb
■Keys and values are strings
■Monolithic, sorted map
CockroachDB: KV

@cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
CockroachDB: KV Primitives

@cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
Del(key)
CockroachDB: KV Primitives

@cockroachdb
■All tables have a primary key
■One key/value pair per column
CockroachDB: Row Storage

@cockroachdb
■All tables have a primary key
■One key/value pair per column
■Key anatomy:
/<table>/<index>/<pkey>/<column>
CockroachDB: Row Storage

@cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
CockroachDB: Example Table

@cockroachdb
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”
/test/primary/1/price 2.22

@cockroachdb
/test/primary/2/name “glove”

@cockroachdb
.../price 2.22
.../2/name “glove”
.../price 3.33

@cockroachdb
/1000/1/1/1 “ball”
.../2 2.22
.../2/1 “glove”
.../2 3.33

@cockroachdb
■Key encoding
■NULL column values
■Unique indexes
■Non-unique indexes
CockroachDB: The Details

@cockroachdb
■Columns are typed data
■???
CockroachDB: Key Encoding

@cockroachdb
■NULL indicates value does not exist
■NULL is weird: NULL != NULL
CockroachDB: NULL Column Values

@cockroachdb
■NULL indicates value does not exist
■CockroachDB: NULL values are not explicitly stored

@cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);

@cockroachdb
INSERT INTO test VALUES (2, NULL, NULL);
??? ???

@cockroachdb
Key: /<table>/<index>/<key>[/<column>] Value
/test/primary/1 Ø
/test/primary/2 Ø

@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
■Multiple table rows with equal indexed values are
not allowed
CockroachDB: Unique Indexes

@cockroachdb
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1

@cockroachdb
/test/bar/”glove” 2

@cockroachdb

@cockroachdb
CockroachDB: Unique Indexes (NULL Values)

@cockroachdb
/test/bar/NULL 3

@cockroachdb
/test/bar/NULL 3
/test/bar/NULL 4

@cockroachdb
Key: /<table>/<index>/<key>[/<pkey>] Value
/test/bar/NULL/3 Ø

@cockroachdb
Key: /<table>/<index>/<key>[/<pkey>] Value
/test/bar/NULL/3 Ø
/test/bar/NULL/4 Ø

@cockroachdb
CREATE INDEX foo ON test (name);
allowed
CockroachDB: Non-Unique Indexes

@cockroachdb
allowed
■Primary key is a unique index

@cockroachdb
Key: /<table>/<index>/<key>/<pkey> Value
/test/foo/”ball”/1 Ø

@cockroachdb
/test/foo/”glove”/2 Ø

@cockroachdb

@cockroachdb
■NULL column values
■Unique indexes
■Non-unique indexes
CockroachDB: Logical Data Storage

@cockroachdb
Logical Data Storage
PostgreSQL CockroachDB
Keys are composite structures Keys are strings
Heap storage for rows Required primary key
Per-table heap/indexes Monolithic map

@cockroachdb
Online Schema Change

@cockroachdb
Schema Change Operations
CREATE INDEX foo ON test (col1, col2, …);
ALTER TABLE test DROP col1;
ALTER TABLE test ADD col3 INTEGER;
...

@cockroachdb
Schema Change (the easy way)
1. Lock table
2. Adjust table data (add column, populate index, etc.)
3. Unlock table

@cockroachdb
Schema Change (the easy way)
1. Apologize for down time
2. Lock table
3. Adjust table data (add column, populate index, etc.)
4. Unlock table

@cockroachdb
Schema Change (the MySQL way)
1. Create new table with altered schema
2. Capture changes from source to the new table
3. Copy rows from the source to the new table
4. Synchronize source and new table
5. Swap/rename source and new table

@cockroachdb
Schema Change (the PostgreSQL way)
1. CREATE INDEX CONCURRENTLY

@cockroachdb
CockroachDB: Schema Change
■TableDescriptor contains table schema
■TableDescriptor replicated on every node
■Distributed atomic updates are difficult
■Distributed locking is difficult
■The easy way isn’t feasible

@cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Backfill index entries
2. Add index to TableDescriptor

@cockroachdb
T1 T2
CREATE INDEX foo ON test… INSERT INTO test…

@cockroachdb
1. Add index to TableDescriptor as write-only
3. Mark index as read-write

@cockroachdb
T1 T2
CREATE INDEX foo ON test… INSERT INTO test…
or
UPDATE test…
or
DELETE FROM test…

@cockroachdb
1. Add index to TableDescriptor as delete-only
2. Wait for descriptor propagation
3. Mark index as write-only
4. Wait for descriptor propagation
6. Mark index as read-write

@cockroachdb
Online Schema Change
Online schema change is difficult
The database should do the heavy lifting

@cockroachdb
The End
SQL databases are KV stores on steroids

@cockroachdb
github.com/cockroachdb/cockroach
CockroachLabs.com
@cockroachdb
Thank You

PostgreSQL and CockroachDB SQL

Recommended

More Related Content

What's hot (20)

Viewers also liked (15)

Similar to PostgreSQL and CockroachDB SQL (20)

Recently uploaded (20)

PostgreSQL and CockroachDB SQL

Editor's Notes