MongoDB .local Chicago 2019: Practical Data Modeling for MongoDB: Tutorial

#MDBlocal
Austin Zellner
Practical Data Modelling
CHICAGO

#MDBLocal
Why MongoDB?
Best way to work
with data
Intelligently put data
where you need it
Freedom
to run anywhere
Intelligent Operational Data Platform

#MDBLocal
Data Modeling Refresher
Conceptual
• What are we capturing?
Logical
• What are the relationships?
Physical
• How do we want to store it?

#MDBLocal
Example: Customer & Address
Database
Index
MongoDB
Customer
Collection
Tables
Document

#MDBLocal
MongoDB Approach
Utility of data leveraged with through query and access pattern
Data
(BSON)
Query
( CRUD /
Aggregation
Pipeline )
Access
Pattern
( Indexing )
Utility

#MDBLocal
BSON Explored
Typed Data
Capture Field Value pairs, capture Arrays, capture Sub Documents

#MDBLocal
Easy: Document data model
• Naturally maps to objects in code
• Represent data of any structure
• Strongly typed for ease of processing
– Over 20 binary encoded JSON data types
• Access by idiomatic drivers in all
major programming language
{
"_id" : ObjectId("5ad88534e3632e1a35a58d00"),
"name" : {
"first" : "John",
"last" : "Doe" },
"address" : [
{ "location" : "work",
"address" : {
"street" : "16 Hatfields",
"city" : "London",
"postal_code" : "SE1 8DJ"},
"geo" : { "type" : "Point", "coord" : [
51.5065752,-0.109081]}},
+ {...}
],
"phone" : [
"number" : "+44-1234567890"},
+ {...}
],
"dob" : ISODate("1977-04-01T05:00:00Z"),
"retirement_fund" : NumberDecimal("1292815.75")
}

#MDBLocal
{
"name" : {
"first" : "John",
"last" : "Doe" },
"address" : [
"address" : {
"city" : "London",
51.5065752,-0.109081]}},
+ {...}
],
"dob" : ISODate("1977-04-01T05:00:00Z"),
}
Flexible: Adapt to change
Add new fields dynamically at runtime
{
"name" : {
"first" : "John",
"last" : "Doe" },
"address" : [
"address" : {
"city" : "London",
51.5065752,-0.109081]}},
+ {...}
],
"phone" : [
"number" : "+44-1234567890"},
+ {...}
],
"dob" : ISODate("1977-04-01T05:00:00Z"),
}

#MDBLocal
Versatile: Multiple data models
JSON Documents Tabular Key-Value Text GraphGeospatial

#MDBLocal
JSON
Stored directly into BSON
Most common entry point for
customers using MongoDB

#MDBLocal
Tabular ( Relational )
Capture column and value as Field Value pairs
Instead of left to right, top to bottom
Sub tables are captured as sub documents

#MDBLocal
Key Value
Key value captured directly as Field Value
If caching, can use TTL index
For attributes, exists as sub document

#MDBLocal
Text
Captured directly into Field Value

#MDBLocal
Graph
Captured in Array
{ "_id" : 0, "airport" : "JFK", "connects" : [ "BOS", "ORD" ] }
{ "_id" : 1, "airport" : "BOS", "connects" : [ "JFK", "PWM" ] }
{ "_id" : 2, "airport" : "ORD", "connects" : [ "JFK" ] }
{ "_id" : 3, "airport" : "PWM", "connects" : [ "BOS", "LHR" ] }
{ "_id" : 4, "airport" : "LHR", "connects" : [ "PWM" ] }

#MDBLocal
Geospatial
Stored in GeoJSON
<field>: { type: <GeoJSON type> , coordinates: <coordinates> }
Types: Point, LineString, Polygon, Multipoint, MultiLinestring, MultiPolygon,
GeometryCollection
Examples
• { type: "Point", coordinates: [ 40, 5 ] }
• { type: "LineString", coordinates: [ [ 40, 5 ], [ 41, 6 ] ] }
• {
type: "Polygon",
coordinates: [ [ [ 0 , 0 ] , [ 3 , 6 ] , [ 6 , 1 ] , [ 0 , 0 ] ] ]
}

#MDBLocal
Combine Multiple Models in One Document
“storenumber” 101
“wifi_fence” type: "Polygon", coordinates: [ [ [ 0 , 0 ] , [ 3 , 6 ] , [ 6 , 1 ] , [ 0 , 0 ] ] ]
“amenities”
“playland” Y
“curbside_pickup” Y
“delivery” Y
“storelocation” type: "Point", [ 123, 456 ]
“sister_stores” [ 102, 104, 503 ]
“description” MongoDB’s document model is…
“employees”
“name” Bob
“name” Sue
“name” Ed
Geo
Key:value
Text
Graph
Tabular

#MDBLocal
Combine Data and State in the Same Document
• Relational keeps data and state information separate
• MongoDB – combine data and state in same document
• Read stateful data from Primary, read stateless data from
Secondary

#MDBLocal
CRUD Operations
Create
• db.collection.insert( )
• db.collection.save( )
• db.collection.update( , , { upsert: true } )
Read
• db.collection.find( , )
• db.collection.findOne( , )
Update
• db.collection.update( , , )
Delete
• db.collection.remove( , )

#MDBLocal
db.customers.aggregate([
{
$unwind: "$address",
},
{
$match: {"address.location": "home"}
},
{
$group: {
_id: "$address.city",
totalSpend: {$sum: "$annualSpend"},
averageSpend: {$avg: "$annualSpend"},
maximumSpend: {$max: "$annualSpend"},
customers: {$sum: 1}
}
}
])
Versatile: Complex queries fast to create, optimize, & maintain
MongoDB’s aggregation framework has the flexibility you need to get
value from your data, but without the complexity and fragility of SQL
These “phases” are distinct and
easy to understand
They can be thought about in
order… no nesting.

#MDBLocal
Versatile: Rich query functionality
MongoDB
{ customer_id : 1,
first_name : "Mark",
last_name : "Smith",
city : "San Francisco",
phones: [ {
number : "1-212-777-1212",
type : "work"
},
{
number : "1-212-777-1213",
type : "cell"
}]
……...
Expressive
Queries
• Find anyone with phone # “1-212…”
• Check if the person with number “555…” is on the “do not call” list
Geospatial
• Find the best offer for the customer at geo coordinates of 42nd St.
and 6th Ave
Text Search • Find all tweets that mention the firm within the last 2 days
Aggregation
• Count and sort number of customers by city, compute min, max,
and average spend
Native Binary
JSON Support
• Add an additional phone number to Mark Smith’s record without
rewriting the document
• Update just 2 phone numbers out of 10
• Sort on the modified date
JOIN
($lookup)
• Query for all San Francisco residences, lookup their transactions,
and sum the amount by person
Graph Queries
($graphLookup)
• Query for all people within 3 degrees of separation from Mark

#MDBLocal
Aggregation to leverage Utility

#MDBLocal
Design Queries to express meaningfulness of data
Since we are layering multiple dimensions, leverage queries to
express the concepts represented in the data.
Example:
Customer Information
Address Information
State in System
Metadata for reporting
Query – Represent customer
and address
Query – Get status
Query – Pull data for reporting
CRUD – Get All

#MDBLocal
Types of indexes
Compound Index
Single Field Index
Text Index
Geospatial Index (lat/lon pairs)
Unique Index (ensures uniqueness)
TTL Index (Time To Live - automatically delete
after elapsed time)

#MDBLocal
Fully Indexable
Fully featured secondary indexes
• Primary Index
– Every Collection has a primary key index
• Compound Index
– Index against multiple keys in the document
• MultiKey Index
– Index into arrays
• Wildcard Index
– Auto-index all matching fields, sub-documents &
arrays
• Text Indexes
– Support for text searches
• GeoSpatial Indexes
– 2d & 2dSphere indexes for spatial geometries
• Hashed Indexes
– Hashed based values for sharding
Index Types
• TTL Indexes
– Single Field indexes, when expired delete the document
• Unique Indexes
– Ensures value is not duplicated
• Partial Indexes
– Expression based indexes, allowing indexes on subsets of data
• Case Insensitive Indexes
• supports text search using case insensitive search
• Sparse Indexes
– Only index documents which have the given field
Index Features

#MDBLocal
{
"_id" : ObjectId("5c1d358bf383fbee028aea0b"),
"product_name" : "Blaster Gauntlet",
"product_attributes" : {
"elements" : [ "Fire" , "Water" ],
"price" : 250
...
}
},
{
"_id" : ObjectId("5c1d358bf383fbee028aea0c"),
"product_name" : "Super Suit",
"product_attributes" : {
"superFlight" : true,
"resistance" : [ "Bludgeoning", "Piercing", "Slashing" ]
...
},
}
Wildcard Indexes
Allow more natural data modeling, avoids
pre-defining indexes for every access
pattern
• Polymorphic document structures:
Product catalogs, CMS
• Ad-hoc queries & data exploration
Define a filter that indexes all matching fields,
sub-documents, and arrays
• Sparse index, omit specific fields
• Covered queries & collations
• Strongly consistent: updated
atomically with base data
Index all sub-documents &
arrays under Product Attributes

#MDBLocal
Special place for
those who do not
Index…

#MDBLocal
One Big Document
Use Case A
Use Case B
Use Case C

#MDBLocal
Application Specific Domain
Customer
Orders
Status
Customer
Orders
Inventory
Focus from Customer point of view
OK because logical relationships flow
This problematic, because Inventory
should be in it’s own logical domain…

#MDBLocal
Easy: MongoDB Multi-Document ACID Transactions
Just like relational transactions
• Multi-statement, familiar relational syntax
• Easy to add to any application
• Multiple documents in 1 or many collections and databases
ACID guarantees
• Snapshot isolation, all or nothing execution
• No performance impact for non-transactional operations

#MDBLocal
Syntax
with client.start_session() as s:
s.start_transaction()
collection_one.insert_one(doc_one, session=s)
collection_two.insert_one(doc_two, session=s)
s.commit_transaction()
Natural for developers
• Idiomatic to the programming
language
• Familiar to relational
developers
• Simple

#MDBLocal
Master and Working Collections
Master: Customer
Order
Use
Case
Working: Order
Read
Read/Write
Master: Inventory
Read
• Master collections are stateless
• Working collections are stateful
• When initiate working document,
read as needed from Master
• Duplicate only for performance
• When Working state changes,
write back to Master
• Example: inventory consumed
• If duplicated data changes in Master,
write update to Working file
• Example: shipping address
• Use Transactions to keep it all straight
1
1
2
2

#MDBLocal
https://university.mongodb.com/courses/M320/about
Data Modeling Patterns Use Cases

#MDBlocal
Practical Data Modeling
for MongoDB: Tutorial
https://www.surveymonkey.com/r/6GZ2GQR
Every session you rate enters you into a drawing for a $250
Visa gift card, sponsored by

MongoDB .local Chicago 2019: Practical Data Modeling for MongoDB: Tutorial

Recommended

More Related Content

What's hot (20)

Similar to MongoDB .local Chicago 2019: Practical Data Modeling for MongoDB: Tutorial (20)

More from MongoDB (20)

Recently uploaded (20)

MongoDB .local Chicago 2019: Practical Data Modeling for MongoDB: Tutorial