SlideShare a Scribd company logo
Getting Started with MongoDB
Using the Microsoft Stack
johnerandolph@gmail.com
John Randolph
Gexa Energy
Overview
• Background
• From pilot to production
• Database access and organization ideas
• C# tips & tricks for MongoDB
• Conclusions & Questions
2
Background
3
Gexa Energy
• Retail electric provider for
commercial and residential
customers.
• Subsidiary of NextEra Energy.
• Headquarters in Houston Texas.
• Development is primarily
• On-Premise
• Oracle
• .Net
• A lot of EDI
4
Why MongoDB?
• We needed to create several document storage and management
applications
• Not JSON; PDFs, XML, Excel, Word and text documents
• Other alternatives were not attractive
• Oracle and high end EMC storage
• Reading original files directly from disk
• Specialized document management systems
5
Document Management using MongoDB
First project was EDI Document management
• No method to access original source documents
• Wanted to provide one
Second project was archiving legacy documents
• Retiring systems
• Multiple types of files per business transaction (XML, PDF, XLS)
• Needed to store and provide access
6
Pilot
M101N: MongoDB for .NET Developers
M102: MongoDB for DBAs
Deployed to Test Server
Happy Developers &
Business Users
New Requirements
Management
Approval
• Hardware
• Commercial agreement with MongoDB
• MongoDB for consulting engagement
• Found SQL Server DBA who wanted to
take on MongoDB
From Pilot to Production
• Engagement with MongoDB Consulting
• 4 day working session
• Prerequisites
• Hardware was in house and built
• Availability of team members from Infrastructure, Applications, Database
• Goals for engagement
• Install MongoDB
• Have operational, functioning production and test systems
• Have an application review
8
Mongo Environment
• CISCO C240 (5)
• 2U height
• Up to 24 SFF 12GB SAS
• Only bought 7
• 2x300GB R1 for OS
• 5x600GB R5 (2.1TB)
• 64GB Memory
• Windows Server 2012
• MongoDB 3.2
9
Database Access and
Organization Ideas
There is no such thing as code that can’t be improved.
10
MongoDBContext
• Only one database connection object needed per application.
• Responsibility
• Obtain connection string
• Hold the connection
• Support SSL
• Access collections
11
Data Access Layer
Interface Implementation
12
EDI Database Organization Requirements
• Large numbers of TSV text documents ~ 80M, 1M per month
• Loaded in batch, never updated
• Less than 1000 of documents are > 16MB.
• Require GridFS to store
13
EDI Database Organization
• @DocInfo collection
• Indexes, key fields, id’s of actual documents
• Data is partitioned by date & size
• Small index collection
• Data collections are partitioned
• Transparently store large docs
• Old collections not updated
14
Extend DA Layer for GridFS & Dynamic Names
Interface Implementation
15
Added GridFS collection
Added collectionName
to Documents Collection
Adding Documents
16
Access Data Transparently from Application
17
Extending the ObjectId
Subsequent projects needed to store multiple files per document
18
ObjectId
• Designed to support simultaneous updates from many clients
• If that is not your use case, you can do a lot with 12 bytes
19
Leveraging the ObjectId
• Leveraging a portion of the ObjectId
20
Using the Extended ObjectId
• Multiple documents per
transaction
21
Saving and Accessing Data
Interface
Use id to find
22
C# Tips & Tricks for MongoDB
Ideas to save you time getting started
23
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
24
Using the DbContext in an MVC Application
25
Add ControllerFactory in App_Start directory
Update Global.asax.cs to use the new ControllerFactory
Inject dbContext when creating controller
Using the DbContext in a DotNet Core MVC
Application
26
Add to Startup.cs
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
27
Saving and Retrieving Files
Console Application
28
Web Application
Importing a file in an MVC application - View
29
enctype = “multipart/form-data”
<input type=“file”
Importing a file in an MVC application - Action
30
Displaying Files (PDFs)
Display in Browser
Query
Download
31
Returning Other Document Types
Search for “MIME Types” or “Media Types”
File Extension
Word (doc) application/msword
Word (docx) application/vnd.openxmlformats-officedocument.wordprocessingml.document
Excel (xls) application/vnd.ms-excel
Excel (xlsx) application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Powerpoint (.ppt) application/vnd.ms-powerpoint
Powerpoint (.pptx) application/vnd.openxmlformats-officedocument.presentationml.presentation
Text application/text
PDF application/pdf
XML application/xml
32
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
33
http://gexadocstore/markettransaction/getDoc/58332b73e38b35214c005710
http://gexadocstore/markettransaction/getDoc/58332c73e39b35214c0057011
http://gexadocstore/markettransaction/getDoc/58332d73e40b35214c0057012
http://gexadocstore/markettransaction/details/58332b73e00b35214c00570f
34
Paging List View with Links
Paging in a MVC application
• PagedList.MVC NuGet Package
• Return IEnumerable
• Works Great!
• Sub second response time on
100M records
db.getCollection('Customer')
.find({Name:’bob’})
.skip(0)
.limit(25)
Action
Query
35
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
36
Serialization / Data Annotation
37
Altering Field Names in Database
• Good meaningful variable names
• Smaller, more database friendly
field names
38
Unexpected Fields in Database
39
New field
Exception thrown
Unexpected Fields in Database
[BsonIgnoreExtraElements]
40
[BsonIgnoreExtraElements]
No Exception
Prevent Fields From Being Written
[BsonIgnore]
41
Not serialized
Prevent Null Field From Being Written
[BsonIgnoreIfNull]
42
[BsonIgnoreIfNull] elements not written
Dates [BsonDateTimeOption]
public DateTime startDate {get;set;}
3/4/2017 8:06:41 PM
3/5/2017 2:06:41 AM
[BsonDateTimeOptions(Kind = DateTimeKind.Local)]
public DateTime startDate {get;set;}
Console.WriteLine(dt)
3/4/2017 8:06:41 PM
3/4/2017 8:06:41 PM
43
Insert DateTime.Now into database
Read into dt
Console.WriteLine(DateTime.Now);
Console.WriteLine(dt)
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
44
Retrieving Select Fields
• Linq doesn’t provide you to exclude certain fields
45
Dynamic Queries
• Builders allow you to build dynamic queries
46
C# Tips & Tricks for MongoDB
• Using the database context
• Saving and retrieving files
• Paging
• Serialization and data annotation
• Builders vs Linq
• Two other tips
47
Async Example
Action
Query
48
Finding Documentation Hint
• .Net drivers have evolved, searches often return out of date information
• Start here: http://mongodb.github.io/mongo-csharp-driver/
49
Conclusion & Questions
50
In Conclusion
• Getting started with MongoDB is incredibly easy.
• Treat it as you’d treat a traditional database
• MongoDB consulting was a great way to get operationally ready.
• As you’re developing:
• Understand the reading and updating requirements for your data
• Index what you need to search for
• Hide information in index key value
• Reduce fields, indexes
• GridFS is not all or nothing
51

More Related Content

What's hot (20)

Sizing MongoDB Clusters
Sizing MongoDB Clusters
MongoDB
 
MongoDB on Azure
MongoDB on Azure
Norberto Leite
 
eHarmony - Messaging Platform with MongoDB Atlas
eHarmony - Messaging Platform with MongoDB Atlas
MongoDB
 
MongoDB Launchpad 2016: MongoDB 3.4: Your Database Evolved
MongoDB Launchpad 2016: MongoDB 3.4: Your Database Evolved
MongoDB
 
Webinar: What's New in MongoDB 3.2
Webinar: What's New in MongoDB 3.2
MongoDB
 
Mongo db 3.4 Overview
Mongo db 3.4 Overview
Norberto Leite
 
MongoDB Europe 2016 - Building WiredTiger
MongoDB Europe 2016 - Building WiredTiger
MongoDB
 
MongoDB: Agile Combustion Engine
MongoDB: Agile Combustion Engine
Norberto Leite
 
MongoDB World 2019: Finding the Right MongoDB Atlas Cluster Size: Does This I...
MongoDB World 2019: Finding the Right MongoDB Atlas Cluster Size: Does This I...
MongoDB
 
Jumpstart: Your Introduction to MongoDB
Jumpstart: Your Introduction to MongoDB
MongoDB
 
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
MongoDB
 
Introducing Stitch
Introducing Stitch
MongoDB
 
Webinar : Nouveautés de MongoDB 3.2
Webinar : Nouveautés de MongoDB 3.2
MongoDB
 
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB
 
Document Validation in MongoDB 3.2
Document Validation in MongoDB 3.2
MongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
MongoDB .local Toronto 2019: Finding the Right Atlas Cluster Size: Does this ...
MongoDB .local Toronto 2019: Finding the Right Atlas Cluster Size: Does this ...
MongoDB
 
Webinar: Choosing the Right Shard Key for High Performance and Scale
Webinar: Choosing the Right Shard Key for High Performance and Scale
MongoDB
 
Webinar: Schema Patterns and Your Storage Engine
Webinar: Schema Patterns and Your Storage Engine
MongoDB
 
Introduction To MongoDB
Introduction To MongoDB
ElieHannouch
 
Sizing MongoDB Clusters
Sizing MongoDB Clusters
MongoDB
 
eHarmony - Messaging Platform with MongoDB Atlas
eHarmony - Messaging Platform with MongoDB Atlas
MongoDB
 
MongoDB Launchpad 2016: MongoDB 3.4: Your Database Evolved
MongoDB Launchpad 2016: MongoDB 3.4: Your Database Evolved
MongoDB
 
Webinar: What's New in MongoDB 3.2
Webinar: What's New in MongoDB 3.2
MongoDB
 
MongoDB Europe 2016 - Building WiredTiger
MongoDB Europe 2016 - Building WiredTiger
MongoDB
 
MongoDB: Agile Combustion Engine
MongoDB: Agile Combustion Engine
Norberto Leite
 
MongoDB World 2019: Finding the Right MongoDB Atlas Cluster Size: Does This I...
MongoDB World 2019: Finding the Right MongoDB Atlas Cluster Size: Does This I...
MongoDB
 
Jumpstart: Your Introduction to MongoDB
Jumpstart: Your Introduction to MongoDB
MongoDB
 
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
MongoDB
 
Introducing Stitch
Introducing Stitch
MongoDB
 
Webinar : Nouveautés de MongoDB 3.2
Webinar : Nouveautés de MongoDB 3.2
MongoDB
 
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB
 
Document Validation in MongoDB 3.2
Document Validation in MongoDB 3.2
MongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
MongoDB .local Toronto 2019: Finding the Right Atlas Cluster Size: Does this ...
MongoDB .local Toronto 2019: Finding the Right Atlas Cluster Size: Does this ...
MongoDB
 
Webinar: Choosing the Right Shard Key for High Performance and Scale
Webinar: Choosing the Right Shard Key for High Performance and Scale
MongoDB
 
Webinar: Schema Patterns and Your Storage Engine
Webinar: Schema Patterns and Your Storage Engine
MongoDB
 
Introduction To MongoDB
Introduction To MongoDB
ElieHannouch
 

Similar to Getting Started with MongoDB Using the Microsoft Stack (20)

Techorama - Evolvable Application Development with MongoDB
Techorama - Evolvable Application Development with MongoDB
bwullems
 
MongoDB 2.4 and spring data
MongoDB 2.4 and spring data
Jimmy Ray
 
Introduction to MongoDB
Introduction to MongoDB
MongoDB
 
Lessons Learned from Building a Multi-Tenant Saas Content Management System o...
Lessons Learned from Building a Multi-Tenant Saas Content Management System o...
MongoDB
 
MongoDB.local Sydney: An Introduction to Document Databases with MongoDB
MongoDB.local Sydney: An Introduction to Document Databases with MongoDB
MongoDB
 
Mongodb, Node.js and You: PART I
Mongodb, Node.js and You: PART I
Mitch Pirtle
 
Mongodb
Mongodb
ichangbai
 
9. Document Oriented Databases
9. Document Oriented Databases
Fabio Fumarola
 
Introduction to MongoDB and its best practices
Introduction to MongoDB and its best practices
AshishRathore72
 
Building your first app with MongoDB
Building your first app with MongoDB
Norberto Leite
 
Introduction to MongoDB a brief intro(1).pptx
Introduction to MongoDB a brief intro(1).pptx
mehfooz968268
 
Mongo db rev001.
Mongo db rev001.
Rich Helton
 
MongoDB NoSQL database a deep dive -MyWhitePaper
MongoDB NoSQL database a deep dive -MyWhitePaper
Rajesh Kumar
 
Introduction to mongo db
Introduction to mongo db
Lawrence Mwai
 
Everything You Need to Know About MongoDB Development.pptx
Everything You Need to Know About MongoDB Development.pptx
75waytechnologies
 
Mongodb Introduction
Mongodb Introduction
Nabeel Naqeebi
 
NDC London 2013 - Mongo db for c# developers
NDC London 2013 - Mongo db for c# developers
Simon Elliston Ball
 
Best Practices for Migrating RDBMS to MongoDB
Best Practices for Migrating RDBMS to MongoDB
Sheeri Cabral
 
Mongo db intro.pptx
Mongo db intro.pptx
JWORKS powered by Ordina
 
Dev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDB
MongoDB
 
Techorama - Evolvable Application Development with MongoDB
Techorama - Evolvable Application Development with MongoDB
bwullems
 
MongoDB 2.4 and spring data
MongoDB 2.4 and spring data
Jimmy Ray
 
Introduction to MongoDB
Introduction to MongoDB
MongoDB
 
Lessons Learned from Building a Multi-Tenant Saas Content Management System o...
Lessons Learned from Building a Multi-Tenant Saas Content Management System o...
MongoDB
 
MongoDB.local Sydney: An Introduction to Document Databases with MongoDB
MongoDB.local Sydney: An Introduction to Document Databases with MongoDB
MongoDB
 
Mongodb, Node.js and You: PART I
Mongodb, Node.js and You: PART I
Mitch Pirtle
 
9. Document Oriented Databases
9. Document Oriented Databases
Fabio Fumarola
 
Introduction to MongoDB and its best practices
Introduction to MongoDB and its best practices
AshishRathore72
 
Building your first app with MongoDB
Building your first app with MongoDB
Norberto Leite
 
Introduction to MongoDB a brief intro(1).pptx
Introduction to MongoDB a brief intro(1).pptx
mehfooz968268
 
Mongo db rev001.
Mongo db rev001.
Rich Helton
 
MongoDB NoSQL database a deep dive -MyWhitePaper
MongoDB NoSQL database a deep dive -MyWhitePaper
Rajesh Kumar
 
Introduction to mongo db
Introduction to mongo db
Lawrence Mwai
 
Everything You Need to Know About MongoDB Development.pptx
Everything You Need to Know About MongoDB Development.pptx
75waytechnologies
 
NDC London 2013 - Mongo db for c# developers
NDC London 2013 - Mongo db for c# developers
Simon Elliston Ball
 
Best Practices for Migrating RDBMS to MongoDB
Best Practices for Migrating RDBMS to MongoDB
Sheeri Cabral
 
Dev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDB
MongoDB
 
Ad

More from MongoDB (20)

MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDB
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDB
MongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDB
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDB
MongoDB
 
Ad

Recently uploaded (20)

Your startup on AWS - How to architect and maintain a Lean and Mean account
Your startup on AWS - How to architect and maintain a Lean and Mean account
angelo60207
 
Data Validation and System Interoperability
Data Validation and System Interoperability
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
FME for Distribution & Transmission Integrity Management Program (DIMP & TIMP)
FME for Distribution & Transmission Integrity Management Program (DIMP & TIMP)
Safe Software
 
The State of Web3 Industry- Industry Report
The State of Web3 Industry- Industry Report
Liveplex
 
Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
Floods in Valencia: Two FME-Powered Stories of Data Resilience
Floods in Valencia: Two FME-Powered Stories of Data Resilience
Safe Software
 
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
 
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Alliance
 
Mastering AI Workflows with FME - Peak of Data & AI 2025
Mastering AI Workflows with FME - Peak of Data & AI 2025
Safe Software
 
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
Muhammad Rizwan Akram
 
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
 
June Patch Tuesday
June Patch Tuesday
Ivanti
 
FIDO Seminar: New Data: Passkey Adoption in the Workforce.pptx
FIDO Seminar: New Data: Passkey Adoption in the Workforce.pptx
FIDO Alliance
 
Agentic AI: Beyond the Buzz- LangGraph Studio V2
Agentic AI: Beyond the Buzz- LangGraph Studio V2
Shashikant Jagtap
 
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
 
Oracle Cloud Infrastructure Generative AI Professional
Oracle Cloud Infrastructure Generative AI Professional
VICTOR MAESTRE RAMIREZ
 
Your startup on AWS - How to architect and maintain a Lean and Mean account J...
Your startup on AWS - How to architect and maintain a Lean and Mean account J...
angelo60207
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
High Availability On-Premises FME Flow.pdf
High Availability On-Premises FME Flow.pdf
Safe Software
 
Your startup on AWS - How to architect and maintain a Lean and Mean account
Your startup on AWS - How to architect and maintain a Lean and Mean account
angelo60207
 
Data Validation and System Interoperability
Data Validation and System Interoperability
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
FME for Distribution & Transmission Integrity Management Program (DIMP & TIMP)
FME for Distribution & Transmission Integrity Management Program (DIMP & TIMP)
Safe Software
 
The State of Web3 Industry- Industry Report
The State of Web3 Industry- Industry Report
Liveplex
 
Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
Floods in Valencia: Two FME-Powered Stories of Data Resilience
Floods in Valencia: Two FME-Powered Stories of Data Resilience
Safe Software
 
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
 
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Alliance
 
Mastering AI Workflows with FME - Peak of Data & AI 2025
Mastering AI Workflows with FME - Peak of Data & AI 2025
Safe Software
 
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
Muhammad Rizwan Akram
 
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
 
June Patch Tuesday
June Patch Tuesday
Ivanti
 
FIDO Seminar: New Data: Passkey Adoption in the Workforce.pptx
FIDO Seminar: New Data: Passkey Adoption in the Workforce.pptx
FIDO Alliance
 
Agentic AI: Beyond the Buzz- LangGraph Studio V2
Agentic AI: Beyond the Buzz- LangGraph Studio V2
Shashikant Jagtap
 
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
 
Oracle Cloud Infrastructure Generative AI Professional
Oracle Cloud Infrastructure Generative AI Professional
VICTOR MAESTRE RAMIREZ
 
Your startup on AWS - How to architect and maintain a Lean and Mean account J...
Your startup on AWS - How to architect and maintain a Lean and Mean account J...
angelo60207
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
High Availability On-Premises FME Flow.pdf
High Availability On-Premises FME Flow.pdf
Safe Software
 

Getting Started with MongoDB Using the Microsoft Stack

  • 1. Getting Started with MongoDB Using the Microsoft Stack [email protected] John Randolph Gexa Energy
  • 2. Overview • Background • From pilot to production • Database access and organization ideas • C# tips & tricks for MongoDB • Conclusions & Questions 2
  • 4. Gexa Energy • Retail electric provider for commercial and residential customers. • Subsidiary of NextEra Energy. • Headquarters in Houston Texas. • Development is primarily • On-Premise • Oracle • .Net • A lot of EDI 4
  • 5. Why MongoDB? • We needed to create several document storage and management applications • Not JSON; PDFs, XML, Excel, Word and text documents • Other alternatives were not attractive • Oracle and high end EMC storage • Reading original files directly from disk • Specialized document management systems 5
  • 6. Document Management using MongoDB First project was EDI Document management • No method to access original source documents • Wanted to provide one Second project was archiving legacy documents • Retiring systems • Multiple types of files per business transaction (XML, PDF, XLS) • Needed to store and provide access 6
  • 7. Pilot M101N: MongoDB for .NET Developers M102: MongoDB for DBAs Deployed to Test Server Happy Developers & Business Users New Requirements Management Approval • Hardware • Commercial agreement with MongoDB • MongoDB for consulting engagement • Found SQL Server DBA who wanted to take on MongoDB
  • 8. From Pilot to Production • Engagement with MongoDB Consulting • 4 day working session • Prerequisites • Hardware was in house and built • Availability of team members from Infrastructure, Applications, Database • Goals for engagement • Install MongoDB • Have operational, functioning production and test systems • Have an application review 8
  • 9. Mongo Environment • CISCO C240 (5) • 2U height • Up to 24 SFF 12GB SAS • Only bought 7 • 2x300GB R1 for OS • 5x600GB R5 (2.1TB) • 64GB Memory • Windows Server 2012 • MongoDB 3.2 9
  • 10. Database Access and Organization Ideas There is no such thing as code that can’t be improved. 10
  • 11. MongoDBContext • Only one database connection object needed per application. • Responsibility • Obtain connection string • Hold the connection • Support SSL • Access collections 11
  • 12. Data Access Layer Interface Implementation 12
  • 13. EDI Database Organization Requirements • Large numbers of TSV text documents ~ 80M, 1M per month • Loaded in batch, never updated • Less than 1000 of documents are > 16MB. • Require GridFS to store 13
  • 14. EDI Database Organization • @DocInfo collection • Indexes, key fields, id’s of actual documents • Data is partitioned by date & size • Small index collection • Data collections are partitioned • Transparently store large docs • Old collections not updated 14
  • 15. Extend DA Layer for GridFS & Dynamic Names Interface Implementation 15 Added GridFS collection Added collectionName to Documents Collection
  • 17. Access Data Transparently from Application 17
  • 18. Extending the ObjectId Subsequent projects needed to store multiple files per document 18
  • 19. ObjectId • Designed to support simultaneous updates from many clients • If that is not your use case, you can do a lot with 12 bytes 19
  • 20. Leveraging the ObjectId • Leveraging a portion of the ObjectId 20
  • 21. Using the Extended ObjectId • Multiple documents per transaction 21
  • 22. Saving and Accessing Data Interface Use id to find 22
  • 23. C# Tips & Tricks for MongoDB Ideas to save you time getting started 23
  • 24. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 24
  • 25. Using the DbContext in an MVC Application 25 Add ControllerFactory in App_Start directory Update Global.asax.cs to use the new ControllerFactory Inject dbContext when creating controller
  • 26. Using the DbContext in a DotNet Core MVC Application 26 Add to Startup.cs
  • 27. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 27
  • 28. Saving and Retrieving Files Console Application 28 Web Application
  • 29. Importing a file in an MVC application - View 29 enctype = “multipart/form-data” <input type=“file”
  • 30. Importing a file in an MVC application - Action 30
  • 31. Displaying Files (PDFs) Display in Browser Query Download 31
  • 32. Returning Other Document Types Search for “MIME Types” or “Media Types” File Extension Word (doc) application/msword Word (docx) application/vnd.openxmlformats-officedocument.wordprocessingml.document Excel (xls) application/vnd.ms-excel Excel (xlsx) application/vnd.openxmlformats-officedocument.spreadsheetml.sheet Powerpoint (.ppt) application/vnd.ms-powerpoint Powerpoint (.pptx) application/vnd.openxmlformats-officedocument.presentationml.presentation Text application/text PDF application/pdf XML application/xml 32
  • 33. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 33
  • 35. Paging in a MVC application • PagedList.MVC NuGet Package • Return IEnumerable • Works Great! • Sub second response time on 100M records db.getCollection('Customer') .find({Name:’bob’}) .skip(0) .limit(25) Action Query 35
  • 36. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 36
  • 37. Serialization / Data Annotation 37
  • 38. Altering Field Names in Database • Good meaningful variable names • Smaller, more database friendly field names 38
  • 39. Unexpected Fields in Database 39 New field Exception thrown
  • 40. Unexpected Fields in Database [BsonIgnoreExtraElements] 40 [BsonIgnoreExtraElements] No Exception
  • 41. Prevent Fields From Being Written [BsonIgnore] 41 Not serialized
  • 42. Prevent Null Field From Being Written [BsonIgnoreIfNull] 42 [BsonIgnoreIfNull] elements not written
  • 43. Dates [BsonDateTimeOption] public DateTime startDate {get;set;} 3/4/2017 8:06:41 PM 3/5/2017 2:06:41 AM [BsonDateTimeOptions(Kind = DateTimeKind.Local)] public DateTime startDate {get;set;} Console.WriteLine(dt) 3/4/2017 8:06:41 PM 3/4/2017 8:06:41 PM 43 Insert DateTime.Now into database Read into dt Console.WriteLine(DateTime.Now); Console.WriteLine(dt)
  • 44. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 44
  • 45. Retrieving Select Fields • Linq doesn’t provide you to exclude certain fields 45
  • 46. Dynamic Queries • Builders allow you to build dynamic queries 46
  • 47. C# Tips & Tricks for MongoDB • Using the database context • Saving and retrieving files • Paging • Serialization and data annotation • Builders vs Linq • Two other tips 47
  • 49. Finding Documentation Hint • .Net drivers have evolved, searches often return out of date information • Start here: http://mongodb.github.io/mongo-csharp-driver/ 49
  • 51. In Conclusion • Getting started with MongoDB is incredibly easy. • Treat it as you’d treat a traditional database • MongoDB consulting was a great way to get operationally ready. • As you’re developing: • Understand the reading and updating requirements for your data • Index what you need to search for • Hide information in index key value • Reduce fields, indexes • GridFS is not all or nothing 51

Editor's Notes

  • #2: Introduce self Ask about audience
  • #3: I’m going to discuss the 1st two projects I developed at Gexa. What the problems we were trying to solve. How we went from idea to production. The rest is very code based. The two projects are similar, but evolutionary. Talk about db org & show how we do data access and how that evolved I’ll show c# things I learned that would have saved me time if I had known them, Wrap & Questions
  • #5: 3 parts to deliverying electiicity Use Service providers Utilities We make money pricing fairly and accurately so we can buy EDI is how we comuicate changes of service historical usage demand Then IT good at processing relational data Eventually put it back together in a data warehouse The problem we have is this is complex data and often the anylists need to see the original
  • #7: First project was EDI Documents We do what all good IT orgs do, we take the transactions and obliterate them, loading the pieces into relational database. MongoDB allows us to store complete documents Multiple indexes can be created to quickly access For example, with 100 million documents, we can present a list of all of a customers documents with sub-second response time View the document in a number of ways Make documents available to other applications via a WebApi Another use case is archiving legacy systems data Legacy systems generate documents on the fly (PDFs, supporting XLS & XML documents) Solution: Generate all historical documents and load into MongoDB Like EDI, index in a variety of ways to provide quick access to PDF and supporting documents Make available to other applications via a WebApi Finally, we’ve implemented print management Similar to #2, but with new, not legacy data
  • #8: DBA Classes M102  mongodb for dba’s M201 mongodb performance M122 Getting started with mongodb cluster management
  • #9: Should I talk the results on the next slide
  • #10: Results Mongo deployed on 5 servers, 4 prod, 1 test One node was non-voting standby member, used for Disaster Recover. SSL everywhere, both to the clients & between replica set members UAT env that mirrors production. Gives the DBA a place to test his stuff Dev, QA, & STG single node systems SSL everywhere, between servers & out to clinets Ops manager installed Application review LDAP was a no go Up for a year. Only one outage that was our fault.
  • #15: Backup – don’t‘ have to backup collections that aren’t changing as often If you have high cost storage, you could locate other collections on lower cost medai
  • #17: You don’t have to decide/check. Handle rare exception.
  • #19: 2nd project has multiple documents per transaction. If I used the method I just showed, I’d have to store both the Id & collection name. I’m going to show how to extend the object id to fix that.
  • #20: Put file date in ObjectId so I don’t have to maintain another field & index
  • #23: In later slide
  • #25: I’m going to show ideas which are basically building up parts of my applications
  • #26: You could just use a static class. Advantage over static is for testing
  • #27: Advantage over static is for testing
  • #36: Couldn’t figure out how to get explain
  • #46: Need to confirm linq actually brings back full object
  • #47: Need to confirm linq actually brings back full object
  • #50: Mongo drivers have evolved Often pull back out of data doc Start here. You can find the area. Use the key works to search futher
  • #52: Address fault tolerance and disaster recover Develop operational plans Change passwords & ports Secure connections Scanning a database is fine for analytics, not for speedy retrieval of documents