SlideShare a Scribd company logo
Unit-3
 Non-relational
 Flexible schema
 Other or additional query languages than SQL
 Distributed – horizontal scaling
 Less structured data
 Supports big data
2
INTRODUCTION OF NOSQL
When compared to relational databases, NoSQL databases are more scalable and provide
superior performance, and their data model addresses several issues that the relational
model is not designed to address:
◦ Geographically distributed architecture instead of expensive,
monolithic architecture
◦ Large volumes of rapidly changing structured, semi-structured, and
unstructured data
◦ Agile sprints, quick schema iteration, and frequent code pushes
◦ Object-oriented programming that is easy to use and flexible
3
 It’s Not No SQL it’s NOT ONLY SQL.
 It’s not even a replacement to RDBMS.
As compared to the good olden days we are
saving more and more data.
Connection between the data is growing in
which we require an architecture that takes
advantage of these two key issues.
MongoDB is a cross-platform, document
oriented database that provides
 High performance.
 High availability.
 Easy scalability.
MongoDB works on concept of collection and
document.
Architecture : -
Database
ContainerDocum
ent
When your requirements has these properties :
 You absolutely must store unstructured data. Say
things coming from 3rd-party API you don’t
control, logs whose format may change any
minute, user-entered metadata, but you want
indexes on a subset of it.
 You need to handle more reads/writes than
single server can deal with and master-slave
architecture won’t work for you.
 You change your schema very often on a large
dataset.
 Stands for No-SQL or Not Only SQL??
 Class of non-relational data storage
systems
 E.g. BigTable, Dynamo, PNUTS/Sherpa, ..
 Usually do not require a fixed table schema
nor do they use the concept of joins
 Distributed data storage systems
 All NoSQL offerings relax one or more of
the ACID properties (will talk about the CAP
theorem)
 Basic API access:
 get(key) -- Extract the value given a key
 put(key, value) -- Create or update the
value given its key
 delete(key) -- Remove the key and its
associated value
 execute(key, operation, parameters) --
Invoke an operation to the value (given
its key) which is a special data structure
(e.g. List, Set, Map .... etc).
NoSQL Data Storage: Classification
 Uninterpreted key/value or ‘the big hash
table’.
 Amazon S3 (Dynamo)
 Flexible schema
 BigTable, Cassandra, Base (ordered keys, semi-
structured data),
 Sherpa/PNuts (unordered keys, JSON)
 MongoDB (based on JSON)
 CouchDB (name/value in text)
 Cheap, easy to implement (open source)
 Data are replicated to multiple nodes (therefore
identical and fault-tolerant) and can be
partitioned
 When data is written, the latest version is on at least
one node and then replicated to other nodes
 No single point of failure
 Easy to distribute
 Don't require a schema
What does NoSQL Not Provide?
 Joins
 Group by
 But PNUTS provides interesting
materialized view approach to
joins/aggregation.
 ACID transactions
 SQL
 Integration with applications that are
based on SQL
Key/value (Dynamo)
Columnar/tabular (Base)
Document (mongoDB)
Big data technology unit 3
mongoDB SQL
Document Tuple
Collection Table/View
PK: _id Field PK: Any Attribute(s)
Uniformity not Required Uniform Relation Schema
Index Index
Embedded Structure Joins
Shard Partition
RDBMS MongoDB
Database ➜ Database
Table ➜ Collection
Row ➜ Document
Index ➜ Index
Join ➜ Embedded Document
Foreign Key ➜ Reference
 Has two phases:
A map stage that processes each document
and emits one or more objects for each input document
A reduce phase that combines the output of the map
operation.
An optional finalize stage for final modifications to the
result
 Uses Custom JavaScript functions
Provides greater flexibility but is less efficient and
more complex than the aggregation pipeline
 Can have output sets that exceed the 16 megabyte
output limitation of the aggregation pipeline.
 It’s Not No SQL it’s NOT ONLY SQL.
 It’s not even a replacement to RDBMS.
As compared to the good olden days we are saving
more and more data.
Connection between the data is growing in which
we require an architecture that takes advantage of
these two key issues.
 Key Value pair
Dynamo DB
Azure Table Storage
(ATS )
Graph
database
 Document Based
Mango Db
AmazonSimple DB
Couch DB
 Column Oriented database
(#key,#value)
(Name, Tom)
(Age,25)
(Role, Student)
(University, CU)
[
{
"Name":
"Tom",
"Age": 30,
"Role":
"Student",
"University":
"CU",
}
]
Student
Tom
CU
25
Masters
Ottawa Location
• Neo4j
• Infogrid
Row Id Columns
1
Name Tom
Age 25
Role Studen
t
Bigtable(Google)
Base
MongoDB is a cross-platform, document oriented
database that provides
 High performance.
 High availability.
 Easy scalability.
MongoDB works on concept of collection and
document.
 All the modern applications deals with huge data.
 Development with ease is possible with mongo DB.
 Flexibility in deployment.
 Rich Queries.
 Older database systems may not be compatible with
the design.
And it’s a document oriented storage:- Data is stored in
the form of JSON Style.
Architecture : -
Database
ContainerDocum
ent
XML JSON
It is a markup language. It is a way of representing
objects.
This is more verbose than
JSON.
This format uses less words.
It is used to describe the
structured data.
It is used to describe
unstructured data which
include arrays.
JavaScript functions like
eval(), parse()
doesn’t work here.
When eval method is applied
to JSON it returns the
described object.
Example:
<car>
<company>Volkswagen</c
ompany>
{
"company": Volkswagen,
"name": "Vento",
 What is it ?
 How does it work ?
 Hadoop
 Tools
 Architecture
 Distributed database management system
 Designed for big data
 Scalable
 Fault tolerant
 No single point of failure
 Has an SQL like query language
 NoSQL
 Organises data into tables
 Uses Cassandra Query Language ( CQL )
 Does not allow sub queries or joins
 Supports Hadoop Map Reduce
 Uses asynchronous masterless replication
◦ Gives low latency
 Allows indexing
 Allows batch analysis via Hadoop
How does Cassandra integrate with Hadoop
 Support for Map Reduce
 Integration with
◦ Apache Pig
◦ Apache Hive
 Can also act as a back end for Solr !
 User Interface ( GUI )
◦ Cassandra GUI
◦ Toad for cloud db's
 Administration
◦ OpsCentre
◦ Cassandra Cluster Admin
 Other
◦ Client libraries
◦ Java, Python, .Net, Perl etc
 A peer to peer cluster
 No single point of failure
 Tunable consistency
◦ Is performance or accuracy more important ?
 Query by key or key range
 Row oriented data storage
 Rows can hold up to 2 billion columns
Big data technology unit 3

More Related Content

PPT
Hadoop mapreduce and yarn frame work- unit5
PPTX
Big Data Unit 4 - Hadoop
PPTX
Introduction of big data unit 1
PPTX
Data lake-itweekend-sharif university-vahid amiry
PPTX
عصر کلان داده، چرا و چگونه؟
PPTX
Big Data and Hadoop
PPTX
Big data analytics: Technology's bleeding edge
PPTX
Big data ppt
Hadoop mapreduce and yarn frame work- unit5
Big Data Unit 4 - Hadoop
Introduction of big data unit 1
Data lake-itweekend-sharif university-vahid amiry
عصر کلان داده، چرا و چگونه؟
Big Data and Hadoop
Big data analytics: Technology's bleeding edge
Big data ppt

What's hot (20)

PPTX
Big data vahidamiri-tabriz-13960226-datastack.ir
PPTX
Big data vahidamiri-datastack.ir
PPTX
Intro to bigdata on gcp (1)
PPT
Big Data Analytics 2014
PPTX
Database awareness
PPTX
Intro to Big Data Hadoop
PPS
Big data hadoop rdbms
PPTX
Big Data with SQL Server
PPTX
Big data concepts
PDF
Hdfs Dhruba
PDF
Big Data , Big Problem?
PPTX
Big Data Concepts
PPTX
Big data ppt
PDF
Big Data technology Landscape
PPTX
Introduction to Apache Hadoop Eco-System
PPTX
PPTX
Big Data Technology Stack : Nutshell
PDF
Vikram Andem Big Data Strategy @ IATA Technology Roadmap
PPTX
Big data
PDF
RDBMS vs Hadoop vs Spark
Big data vahidamiri-tabriz-13960226-datastack.ir
Big data vahidamiri-datastack.ir
Intro to bigdata on gcp (1)
Big Data Analytics 2014
Database awareness
Intro to Big Data Hadoop
Big data hadoop rdbms
Big Data with SQL Server
Big data concepts
Hdfs Dhruba
Big Data , Big Problem?
Big Data Concepts
Big data ppt
Big Data technology Landscape
Introduction to Apache Hadoop Eco-System
Big Data Technology Stack : Nutshell
Vikram Andem Big Data Strategy @ IATA Technology Roadmap
Big data
RDBMS vs Hadoop vs Spark
Ad

Similar to Big data technology unit 3 (20)

PDF
Nosql part1 8th December
PPT
Mongo Bb - NoSQL tutorial
PPTX
Introduction to NoSQL and MongoDB
PPTX
NoSQL Basics and MongDB
PPTX
Nosql
PPTX
PDF
Baisc introduction of mongodb for beginn
PDF
20-NoSQLMongoDbiig data analytics hB.pdf
PPTX
Introduction to Data Science NoSQL.pptx
PDF
No sql databases
PPTX
NoSQL(MongoDB and DynamoDB) Overview.pptx
PPTX
Muskan Kumari (1276) Intro to NoSql.pptx. pptx
PDF
NoSql and it's introduction features-Unit-1.pdf
PPTX
MongoDB NoSQL - Developer Guide
PPTX
MongoDB 2.4 and spring data
PDF
the rising no sql technology
PPTX
No sq lv2
PPTX
PDF
NOsql Presentation.pdf
PPTX
cours database pour etudiant NoSQL (1).pptx
Nosql part1 8th December
Mongo Bb - NoSQL tutorial
Introduction to NoSQL and MongoDB
NoSQL Basics and MongDB
Nosql
Baisc introduction of mongodb for beginn
20-NoSQLMongoDbiig data analytics hB.pdf
Introduction to Data Science NoSQL.pptx
No sql databases
NoSQL(MongoDB and DynamoDB) Overview.pptx
Muskan Kumari (1276) Intro to NoSql.pptx. pptx
NoSql and it's introduction features-Unit-1.pdf
MongoDB NoSQL - Developer Guide
MongoDB 2.4 and spring data
the rising no sql technology
No sq lv2
NOsql Presentation.pdf
cours database pour etudiant NoSQL (1).pptx
Ad

Recently uploaded (20)

PDF
Microbial disease of the cardiovascular and lymphatic systems
PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
PDF
Classroom Observation Tools for Teachers
PDF
STATICS OF THE RIGID BODIES Hibbelers.pdf
PDF
Anesthesia in Laparoscopic Surgery in India
PDF
RTP_AR_KS1_Tutor's Guide_English [FOR REPRODUCTION].pdf
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
PDF
Paper A Mock Exam 9_ Attempt review.pdf.
PDF
2.FourierTransform-ShortQuestionswithAnswers.pdf
PDF
RMMM.pdf make it easy to upload and study
PPTX
Introduction-to-Literarature-and-Literary-Studies-week-Prelim-coverage.pptx
PPTX
202450812 BayCHI UCSC-SV 20250812 v17.pptx
PDF
A GUIDE TO GENETICS FOR UNDERGRADUATE MEDICAL STUDENTS
PDF
Complications of Minimal Access Surgery at WLH
DOC
Soft-furnishing-By-Architect-A.F.M.Mohiuddin-Akhand.doc
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
PDF
LNK 2025 (2).pdf MWEHEHEHEHEHEHEHEHEHEHE
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
PDF
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
Microbial disease of the cardiovascular and lymphatic systems
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
Classroom Observation Tools for Teachers
STATICS OF THE RIGID BODIES Hibbelers.pdf
Anesthesia in Laparoscopic Surgery in India
RTP_AR_KS1_Tutor's Guide_English [FOR REPRODUCTION].pdf
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
Paper A Mock Exam 9_ Attempt review.pdf.
2.FourierTransform-ShortQuestionswithAnswers.pdf
RMMM.pdf make it easy to upload and study
Introduction-to-Literarature-and-Literary-Studies-week-Prelim-coverage.pptx
202450812 BayCHI UCSC-SV 20250812 v17.pptx
A GUIDE TO GENETICS FOR UNDERGRADUATE MEDICAL STUDENTS
Complications of Minimal Access Surgery at WLH
Soft-furnishing-By-Architect-A.F.M.Mohiuddin-Akhand.doc
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
LNK 2025 (2).pdf MWEHEHEHEHEHEHEHEHEHEHE
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf

Big data technology unit 3

  • 2.  Non-relational  Flexible schema  Other or additional query languages than SQL  Distributed – horizontal scaling  Less structured data  Supports big data 2 INTRODUCTION OF NOSQL
  • 3. When compared to relational databases, NoSQL databases are more scalable and provide superior performance, and their data model addresses several issues that the relational model is not designed to address: ◦ Geographically distributed architecture instead of expensive, monolithic architecture ◦ Large volumes of rapidly changing structured, semi-structured, and unstructured data ◦ Agile sprints, quick schema iteration, and frequent code pushes ◦ Object-oriented programming that is easy to use and flexible 3
  • 4.  It’s Not No SQL it’s NOT ONLY SQL.  It’s not even a replacement to RDBMS. As compared to the good olden days we are saving more and more data. Connection between the data is growing in which we require an architecture that takes advantage of these two key issues.
  • 5. MongoDB is a cross-platform, document oriented database that provides  High performance.  High availability.  Easy scalability. MongoDB works on concept of collection and document.
  • 7. When your requirements has these properties :  You absolutely must store unstructured data. Say things coming from 3rd-party API you don’t control, logs whose format may change any minute, user-entered metadata, but you want indexes on a subset of it.  You need to handle more reads/writes than single server can deal with and master-slave architecture won’t work for you.  You change your schema very often on a large dataset.
  • 8.  Stands for No-SQL or Not Only SQL??  Class of non-relational data storage systems  E.g. BigTable, Dynamo, PNUTS/Sherpa, ..  Usually do not require a fixed table schema nor do they use the concept of joins  Distributed data storage systems  All NoSQL offerings relax one or more of the ACID properties (will talk about the CAP theorem)
  • 9.  Basic API access:  get(key) -- Extract the value given a key  put(key, value) -- Create or update the value given its key  delete(key) -- Remove the key and its associated value  execute(key, operation, parameters) -- Invoke an operation to the value (given its key) which is a special data structure (e.g. List, Set, Map .... etc).
  • 10. NoSQL Data Storage: Classification  Uninterpreted key/value or ‘the big hash table’.  Amazon S3 (Dynamo)  Flexible schema  BigTable, Cassandra, Base (ordered keys, semi- structured data),  Sherpa/PNuts (unordered keys, JSON)  MongoDB (based on JSON)  CouchDB (name/value in text)
  • 11.  Cheap, easy to implement (open source)  Data are replicated to multiple nodes (therefore identical and fault-tolerant) and can be partitioned  When data is written, the latest version is on at least one node and then replicated to other nodes  No single point of failure  Easy to distribute  Don't require a schema
  • 12. What does NoSQL Not Provide?  Joins  Group by  But PNUTS provides interesting materialized view approach to joins/aggregation.  ACID transactions  SQL  Integration with applications that are based on SQL
  • 15. mongoDB SQL Document Tuple Collection Table/View PK: _id Field PK: Any Attribute(s) Uniformity not Required Uniform Relation Schema Index Index Embedded Structure Joins Shard Partition
  • 16. RDBMS MongoDB Database ➜ Database Table ➜ Collection Row ➜ Document Index ➜ Index Join ➜ Embedded Document Foreign Key ➜ Reference
  • 17.  Has two phases: A map stage that processes each document and emits one or more objects for each input document A reduce phase that combines the output of the map operation. An optional finalize stage for final modifications to the result  Uses Custom JavaScript functions Provides greater flexibility but is less efficient and more complex than the aggregation pipeline  Can have output sets that exceed the 16 megabyte output limitation of the aggregation pipeline.
  • 18.  It’s Not No SQL it’s NOT ONLY SQL.  It’s not even a replacement to RDBMS. As compared to the good olden days we are saving more and more data. Connection between the data is growing in which we require an architecture that takes advantage of these two key issues.
  • 19.  Key Value pair Dynamo DB Azure Table Storage (ATS ) Graph database  Document Based Mango Db AmazonSimple DB Couch DB  Column Oriented database (#key,#value) (Name, Tom) (Age,25) (Role, Student) (University, CU) [ { "Name": "Tom", "Age": 30, "Role": "Student", "University": "CU", } ] Student Tom CU 25 Masters Ottawa Location • Neo4j • Infogrid Row Id Columns 1 Name Tom Age 25 Role Studen t Bigtable(Google) Base
  • 20. MongoDB is a cross-platform, document oriented database that provides  High performance.  High availability.  Easy scalability. MongoDB works on concept of collection and document.
  • 21.  All the modern applications deals with huge data.  Development with ease is possible with mongo DB.  Flexibility in deployment.  Rich Queries.  Older database systems may not be compatible with the design. And it’s a document oriented storage:- Data is stored in the form of JSON Style.
  • 23. XML JSON It is a markup language. It is a way of representing objects. This is more verbose than JSON. This format uses less words. It is used to describe the structured data. It is used to describe unstructured data which include arrays. JavaScript functions like eval(), parse() doesn’t work here. When eval method is applied to JSON it returns the described object. Example: <car> <company>Volkswagen</c ompany> { "company": Volkswagen, "name": "Vento",
  • 24.  What is it ?  How does it work ?  Hadoop  Tools  Architecture
  • 25.  Distributed database management system  Designed for big data  Scalable  Fault tolerant  No single point of failure  Has an SQL like query language  NoSQL
  • 26.  Organises data into tables  Uses Cassandra Query Language ( CQL )  Does not allow sub queries or joins  Supports Hadoop Map Reduce  Uses asynchronous masterless replication ◦ Gives low latency  Allows indexing  Allows batch analysis via Hadoop
  • 27. How does Cassandra integrate with Hadoop  Support for Map Reduce  Integration with ◦ Apache Pig ◦ Apache Hive  Can also act as a back end for Solr !
  • 28.  User Interface ( GUI ) ◦ Cassandra GUI ◦ Toad for cloud db's  Administration ◦ OpsCentre ◦ Cassandra Cluster Admin  Other ◦ Client libraries ◦ Java, Python, .Net, Perl etc
  • 29.  A peer to peer cluster  No single point of failure  Tunable consistency ◦ Is performance or accuracy more important ?  Query by key or key range  Row oriented data storage  Rows can hold up to 2 billion columns