Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of Couchbase

Bringing the SQL back to NoSQL
Querying JSON w/N1QL

©2014 Couchbase Inc.
Titanium Sponsors
Platinum Sponsors
Gold Sponsors

About Me: Jeff Morris
 Software Engineer @Couchbase
 Maintain several open source projects…
 Official Couchbase .NET SDK
 Linq2Couchbase
 CouchbaseASP.NET Extensions
 Former Enterprise Architect
 Work primarily with .NET, but some Java and arch. ex. with
PHP
 Family, Surfing and Fishing
3

Agenda
 Intro
 Couchbase 101, Data in Couchbase
 N1QL
 N1QL vs SQL, Language Structure, Queries and Results
 Advanced
 How do queries work, 2i, Predicate filters
 SDK Support
 RESTAPI,Ad hoc queries, Linq, Java DSL
 Q&A
4

Couchbase 101
Prerequisites…
5

Couchbase Server: About
 High throughput, low latency
 Built-in distributed cache
 Commodity hardware
 Distributed, shared nothing architecture
 Nodes are peers
 Strong consistency
 Elastic scalability
 XDCR

Storage Engine
Cluster Manager
Data Service
Projector & Router
New Services in Couchbase Server 4.0
Query ServiceIndex Service
Supervisor
Index maintenance &
Scan coordinator
Index#2Index#1
Query Processor
cbq-engine
Bucket#1 Bucket#2
DCP Stream
Index#4Index#3
...
B
u
c
k
e
t
#
2
B
u
c
k
e
t
#
1
1809311211 18901
ManagedCache

Full Cluster Architecture
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 2
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service

Couchbase Server: Core concepts
 Node
 Cluster
 Bucket
 Views and Design Documents
 Key/Value
 Cluster Map
 “Smart” clients
 Multi-dimensional scaling (> 4.0)
9

Introducing N1QL
Just the basics…

Data In Couchbase
 Data is stored as a binary blob or a document
 Every datum has a primary key
 A document is a JSON object
 Documents are of arbitrary length and depth
 Documents contain scalars, other objects and arrays or
collections
 Each document has it’s own implied schema
 Documents are agile; they can evolve over time

N1QL and SQL differences
 Data model
 Static vs Fluid
 Data is constrained by a “key space”
 Projection differences
 Columns vs “rich model”
 Reshaping of data is possible
 Selection differences
 Table vs Keyspace
 New operators: “.” and “[]”
 Filtering differences
 Irregular shape: IS MISSING, IS NULL
 ANY, SOME, EVERY 12

Language Structure
 Statements
 Data Definition Language (DDL)
 Data Manipulations Language (DML)
 Query statements
 Expressions
 Comments
13

Queries and Results
 Submit queries using SELECT statements
 Nested subqueries are possible
 Query across multiple documents with JOINs
 Operators and function allow navigation through arrays
 Use predicates in theWHERE clause to return ill regular data
 GROUP BY, ORDER BY, LIMIT and OFFSET and functions to transform
results
 N1QL provide paths to support nested data
 Paths are used with arrays and nested objects to get attributes
 Array syntax is used to get the information: orders[0].product_id
14

Queries and Results: example
15
SELECT name, brewery_id from `beer-sample`
WHERE brewery_id IS NOT MISSING
LIMIT 2;

Query results
 Returned as a JSON document
 Results are nested within additional data
 Request Id
 Signature
 Status
 Metrics
 Errors messages are also returned with “hints”
16

Query Results: Example
17
{
"requestID": "fbea9e79-a2e2-4ab8-9fdc-14e098838cc1",
"signature": { "brewery_id": "json", "name": "json" },
"results": [
{ "brewery_id": "big_horn_brewing_the_ram_2", "name": "Schaumbergfest" },
{ "brewery_id": "ballast_point_brewing", "name": "Wahoo Wheat Beer" } ],
"status": "success",
"metrics": { "elapsedTime": "131.492184ms",
"executionTime": "131.261322ms",
"resultCount": 2,
"resultSize": 205 }
}

System Information
 N1QL has a system catalogue that stores metadata
 The system catalog is a namespace called “system”
 Each artifact has a keyspace
 Logical hierarchy
 Data stores
 Namespaces
 Keyspaces
 Indexes
 Duals
 All system keyspaces are queryable!
18

Advanced N1QLTopics
What you need to know…

UNNEST
21

Index Hints
22

EXPLAIN
23

Creating Indexes
24

REST API, SDK support, ODBC/JDBC and
Connectors
How to access it…

REST API

SDK Support
27

ODBC and JDBC Drivers
28

Connectors and 3rd part integration
29

Q&A
You ask, I answer…mmmaybe

Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of Couchbase

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of Couchbase (20)

More from Data Con LA (20)

Recently uploaded (20)

Big Data Day LA 2015 - Introducing N1QL: SQL for Documents by Jeff Morris of Couchbase