SlideShare a Scribd company logo

Building your first java application with MongoDB

Download as PPTX, PDF
MongoDB, profile picture
MongoDB

This document provides a detailed overview of building a location-based surf reporting application using MongoDB, highlighting its high performance, availability, and scalability. It outlines the data model, including the concepts of documents, collections, and replica sets, and also details the implementation of queries to retrieve surf conditions. Additionally, it discusses sharding for scaling and features like the aggregation framework for analyzing surf data.

TechnologyNews & Politics
1 of 40
Downloaded 23 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Building Your First Application in Java Bryan Reinero Bryan.reinero@10gen.com September 2012 1
 High performance  Highly available  Easily scalable  Easy to use  Feature rich Document store ©2012 Jaspersoft Corporation. Proprietary and Confidential 2
Data Model  A Mongo system holds a set of databases  A database holds a set of collections  A collection holds a set of documents  A document is a set of fields  A field is a key-value pair  A key is a name (string)  A value is a basic type like string, integer, float, timestamp, binary, etc., a document, or an array of values ©2012 Jaspersoft Corporation. Proprietary and Confidential 3
High Availability: Replica Sets  Initialize -> Election  Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 4
High Availability: Failure  Primary down/network failure  Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 5
High Availability: Failover  New primary elected  Replication established from new primary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 6
Durability  Fire and forget  Wait for error  Wait for journal sync  Wait for fsync  Wait for replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 7
Read Preferences  PRIMARY  PRIMARY PREFERRED  SECONDARY  SECONDARY PREFERRED  NEAREST ©2012 Jaspersoft Corporation. Proprietary and Confidential 8
Let’s build a location based surf reporting app! ©2012 Jaspersoft Corporation. Proprietary and Confidential 9
Let’s build a location based surf reporting app! • Report current conditions
Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions
Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions • Determine best conditions per beach
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "test", "location" : { "coordinates" : [ -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Auto-indexed -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, Indexed for "date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live }
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1})
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
Results { "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } } { "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } } { "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
Scaling  Sharding is the partitioning of data among multiple machines  Balancing occurs when the load on any one node grows out of proportion ©2012 Jaspersoft Corporation. Proprietary and Confidential 23
Scaling MongoDB Sharded cluster MongoDB Single Instance Or Replica Set Client Application
The Mechanism of Sharding Complete Data Set Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean BeachRockaway Shard 1 Shard 2 Shard 3 Shard 4
The Mechanism of Sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 29
The Mechanism of Sharding Client Query: Linda Mar Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 30
The Mechanism of Sharding Client Query: Maverick’s Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 31
Analysis Features: Aggregation Framework What are the best conditions for my local beach?
Pipelining Operations $match Match “Linda Mar” $project Only interested in conditions $group Group by rating, averaging wave height and wave period $sort Order by best conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] }
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Match “Linda Mar”
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Only interested in conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Group by rating & average conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Show me best conditions first
Other Features…  Native MapReduce  Hadoop Connector  Tagging  Drivers for all major languages ©2012 Jaspersoft Corporation. Proprietary and Confidential 39
Thanks! Office Hours Thursdays 4-6 pm 555 University Ave. Palo Alto We’re Hiring ! Bryan.reinero@10gen.com

More Related Content

PPTX
Building your first Java Application with MongoDB
MongoDB
 
PPTX
Building Your First Java Application with MongoDB
MongoDB
 
PDF
MongoDB: Optimising for Performance, Scale & Analytics
Server Density
 
PDF
Real Time Big Data (w/ NoSQL)
Stein Writes Inc.
 
PDF
Intro to the Hadoop Stack @ April 2011 JavaMUG
David Engfer
 
PDF
ROracle
Mohamed Magdy
 
PPTX
Ops Jumpstart: MongoDB Administration 101
MongoDB
 
KEY
An introduction to CouchDB
David Coallier
 
Building your first Java Application with MongoDB
MongoDB
 
Building Your First Java Application with MongoDB
MongoDB
 
MongoDB: Optimising for Performance, Scale & Analytics
Server Density
 
Real Time Big Data (w/ NoSQL)
Stein Writes Inc.
 
Intro to the Hadoop Stack @ April 2011 JavaMUG
David Engfer
 
ROracle
Mohamed Magdy
 
Ops Jumpstart: MongoDB Administration 101
MongoDB
 
An introduction to CouchDB
David Coallier
 

Viewers also liked (7)

PPTX
MongoDB Roadmap
MongoDB
 
PPT
Giftivo mongodb
MongoDB
 
PPT
A Morning with MongoDB - Helsinki
MongoDB
 
PPTX
Indexing and Query Optimisation
MongoDB
 
PPTX
Webinar: Replication and Replica Sets
MongoDB
 
PPTX
Branf final bringing mongodb into your organization - mongo db-boston2012
MongoDB
 
KEY
Discover MongoDB - Israel
Michael Fiedler
 
MongoDB Roadmap
MongoDB
 
Giftivo mongodb
MongoDB
 
A Morning with MongoDB - Helsinki
MongoDB
 
Indexing and Query Optimisation
MongoDB
 
Webinar: Replication and Replica Sets
MongoDB
 
Branf final bringing mongodb into your organization - mongo db-boston2012
MongoDB
 
Discover MongoDB - Israel
Michael Fiedler
 
Ad

Similar to Building your first java application with MongoDB (20)

PDF
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
MongoDB
 
PDF
Improving Operational Space Responsiveness
Pat Cappelaere
 
PPTX
Operational Intelligence with MongoDB Webinar
MongoDB
 
PPTX
MongoDB for Time Series Data Part 3: Sharding
MongoDB
 
PDF
Webinar: Building Your First App with MongoDB and Java
MongoDB
 
PPTX
High-Volume Data Collection and Real Time Analytics Using Redis
cacois
 
PPT
Meetup#1: 10 reasons to fall in love with MongoDB
Minsk MongoDB User Group
 
PDF
Getting started with Spark & Cassandra by Jon Haddad of Datastax
Data Con LA
 
PDF
10 Key MongoDB Performance Indicators
iammutex
 
PDF
Composable Data Processing with Apache Spark
Databricks
 
PDF
2013 london advanced-replication
Marc Schwering
 
PDF
Mongodb workshop
Harun Yardımcı
 
PDF
A Century Of Weather Data - Midwest.io
Randall Hunt
 
PPTX
Average- An android project
Ipsit Dash
 
PDF
HMS: Scalable Configuration Management System for Hadoop
DataWorks Summit
 
PDF
Spring Data MongoDB 介紹
Kuo-Chun Su
 
PDF
NoSQL Infrastructure
Server Density
 
PDF
Maintenance for MongoDB Replica Sets
Igor Donchovski
 
PPTX
Data-centric Invocable Services
Rick Warren
 
KEY
OpenStack Folsom Summit: Melange overview
troytoman
 
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
MongoDB
 
Improving Operational Space Responsiveness
Pat Cappelaere
 
Operational Intelligence with MongoDB Webinar
MongoDB
 
MongoDB for Time Series Data Part 3: Sharding
MongoDB
 
Webinar: Building Your First App with MongoDB and Java
MongoDB
 
High-Volume Data Collection and Real Time Analytics Using Redis
cacois
 
Meetup#1: 10 reasons to fall in love with MongoDB
Minsk MongoDB User Group
 
Getting started with Spark & Cassandra by Jon Haddad of Datastax
Data Con LA
 
10 Key MongoDB Performance Indicators
iammutex
 
Composable Data Processing with Apache Spark
Databricks
 
2013 london advanced-replication
Marc Schwering
 
Mongodb workshop
Harun Yardımcı
 
A Century Of Weather Data - Midwest.io
Randall Hunt
 
Average- An android project
Ipsit Dash
 
HMS: Scalable Configuration Management System for Hadoop
DataWorks Summit
 
Spring Data MongoDB 介紹
Kuo-Chun Su
 
NoSQL Infrastructure
Server Density
 
Maintenance for MongoDB Replica Sets
Igor Donchovski
 
Data-centric Invocable Services
Rick Warren
 
OpenStack Folsom Summit: Melange overview
troytoman
 
Ad

More from MongoDB (20)

PDF
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
PDF
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
PDF
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
PDF
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
PDF
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
PDF
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
PDF
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
PDF
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
PDF
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 

Recently uploaded (20)

PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PPTX
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Approach and Philosophy of On baking technology
30anthuong17
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
MYSQL Presentation for SQL database connectivity
Swati270511
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
Teaching material agriculture food technology
LiaRayya
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 

Building your first java application with MongoDB

  • 1. Building Your First Application in Java Bryan Reinero [email protected] September 2012 1
  • 2. High performance  Highly available  Easily scalable  Easy to use  Feature rich Document store ©2012 Jaspersoft Corporation. Proprietary and Confidential 2
  • 3. Data Model  A Mongo system holds a set of databases  A database holds a set of collections  A collection holds a set of documents  A document is a set of fields  A field is a key-value pair  A key is a name (string)  A value is a basic type like string, integer, float, timestamp, binary, etc., a document, or an array of values ©2012 Jaspersoft Corporation. Proprietary and Confidential 3
  • 4. High Availability: Replica Sets  Initialize -> Election  Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 4
  • 5. High Availability: Failure  Primary down/network failure  Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 5
  • 6. High Availability: Failover  New primary elected  Replication established from new primary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 6
  • 7. Durability  Fire and forget  Wait for error  Wait for journal sync  Wait for fsync  Wait for replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 7
  • 8. Read Preferences  PRIMARY  PRIMARY PREFERRED  SECONDARY  SECONDARY PREFERRED  NEAREST ©2012 Jaspersoft Corporation. Proprietary and Confidential 8
  • 9. Let’s build a location based surf reporting app! ©2012 Jaspersoft Corporation. Proprietary and Confidential 9
  • 10. Let’s build a location based surf reporting app! • Report current conditions
  • 11. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions
  • 12. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions • Determine best conditions per beach
  • 13. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "test", "location" : { "coordinates" : [ -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 14. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Auto-indexed -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 15. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 16. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, Indexed for "date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live }
  • 17. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1})
  • 18. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports
  • 19. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
  • 20. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
  • 21. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
  • 22. Results { "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } } { "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } } { "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
  • 23. Scaling  Sharding is the partitioning of data among multiple machines  Balancing occurs when the load on any one node grows out of proportion ©2012 Jaspersoft Corporation. Proprietary and Confidential 23
  • 24. Scaling MongoDB Sharded cluster MongoDB Single Instance Or Replica Set Client Application
  • 25. The Mechanism of Sharding Complete Data Set Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 26. The Mechanism of Sharding Chunk Chunk Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 27. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 28. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean BeachRockaway Shard 1 Shard 2 Shard 3 Shard 4
  • 29. The Mechanism of Sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 29
  • 30. The Mechanism of Sharding Client Query: Linda Mar Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 30
  • 31. The Mechanism of Sharding Client Query: Maverick’s Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 31
  • 32. Analysis Features: Aggregation Framework What are the best conditions for my local beach?
  • 33. Pipelining Operations $match Match “Linda Mar” $project Only interested in conditions $group Group by rating, averaging wave height and wave period $sort Order by best conditions
  • 34. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] }
  • 35. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Match “Linda Mar”
  • 36. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Only interested in conditions
  • 37. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Group by rating & average conditions
  • 38. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Show me best conditions first
  • 39. Other Features…  Native MapReduce  Hadoop Connector  Tagging  Drivers for all major languages ©2012 Jaspersoft Corporation. Proprietary and Confidential 39
  • 40. Thanks! Office Hours Thursdays 4-6 pm 555 University Ave. Palo Alto We’re Hiring ! [email protected]