No SQL, No Problem: Use Azure DocumentDB

Editor's Notes

• #5: Document databases pair each key with a complex data structure known as a document. Documents can contain many different key-value pairs, or key-array pairs, or even nested documents. Mongo, Azure DocumentDB. Work best with hierarchical documents that are entirely or almost entirely self contained. Graph stores are used to store information about networks of data, such as social connections. Graph stores include Neo4J and Giraph. Key-value stores are the simplest NoSQL databases. Every single item in the database is stored as an attribute name (or 'key'), together with its value. Examples of key-value stores are Riak and Berkeley DB. Some key-value stores, such as Redis, allow each value to have a type, such as 'integer', which adds functionality. Wide-column stores such as Cassandra and HBase are optimized for queries over large datasets, and store columns of data together, instead of rows.
• #6: In DocumentDB, persist whole object in DB ; query the whole object and display it Service: no need for containers or VMs to run it; provision the service; has fine grain control b/c of this Indexing: builds the structure to start right away; can shape the data; no need to spend time on ERD or large schemas to persist data; indexes evolve as models do DocumentDB database can grow to hundred of terabytes or event petabytes; thousands of nodes in data centers
• #8: Table Storage (Key Value) Azure Blob Storage can be used to store full user profiles including images. Azure Tables is cheap, scalable No SQL solution Good for key value patterns to be scored at scale Runs on spinning disk and has higher latency when getting to 95th percentile But no secondary indexes either
• #9: Mongo protocol support now supports Mongo drivers Can leverage existing skills and tools to interact with a DocumentDB service This supports the native MongoDB wire protocol
• #10: Complex relationships – obvious (denormalized data and flex schema) Rigid schema – for instance if your data was imported from XML docs Complex transactions – supports some but can’t cross boundaries Aggregation – since some properties could be present or not, it’s not a good fit for data aggregation Encrypted data storage – protocol is encrypted, data is not. No current in-built mechanisms for encrypted data storage Moving entire datastore to Azure DocumentDB – use as supplement not replacement Azure specific – although adding protocol support for MongoDB
• #11: UGC in social media applications is a blend of free form text, properties, tags and relationships not bounded by rigid structure. Content such as chats, comments, and posts can be stored in DocumentDB without requiring transformations or complex object to relational mapping layers. Data properties can be added or modified easily to match requirements as developers iterate over the application code, thus promoting rapid development. Catalog Data - Attributes for this data may vary and can change over time to fit application requirements. Consider an example of a product catalog for an automotive parts supplier. Every part may have its own attributes in addition to the common attributes that all parts share. Furthermore, attributes for a specific part can change the following year when a new model is released. As a JSON document store, DocumentDB supports flexible schemas and allows you to represent data with nested properties, and thus it is well suited for storing product catalog data. Logging - Perform ad-hoc queries over a subset of data for troubleshooting. Subset of data is first retrieved from the logs, typically by time series. Then, a drill-down is performed by filtering the dataset with error levels or error messages. Long running data analytics jobs performed offline over a large volume of log data. Examples of this use case include server availability analysis, application error analysis, and clickstream data analysis. Typically, Hadoop is used to perform these types of analyses with data from DocumentDB using Hadoop connector Gaming - Handle updating profile and stats from millions of simultaneous gamers, millisecond reads and writes to help avoid any lags, automatic indexing allows for filtering against multiple different properties in real-time User preferences data - most modern web and mobile applications come with complex views and experiences. These views and experiences are usually dynamic, catering to user preferences or moods and branding needs. Hence, applications need to be able to retrieve personalized settings effectively in order to render UI elements and experiences quickly. IoT - Bursts of data can be ingested by Azure Event Hubs as it offers high throughput data ingestion with low latency. Data ingested that needs to be processed for real time insight can be funneled to Azure Stream Analytics for real time analytics. Data can be loaded into DocumentDB for ad-hoc querying. Once the data is loaded into DocumentDB, the data is ready to be queried. The data in DocumentDB can be used as reference data as part of real time analytics. In addition, data can further be refined and processed by connecting DocumentDB data to HDInsight for Pig, Hive or Map/Reduce jobs. Refined data is then loaded back to DocumentDB for reporting.
• #12: Provides keys and resource location for databases and contents
• #14: Container of JSON documents and the associated JavaScript application logic (stored proc, trigger, UDF) Collections are isolated and independent from one another. Queries run against a single collection and returns documents from that collection
• #15: Read consistency level of documents follows the consistency policy on the database account
• #16: DocumentDB cannot have poor performing code affect the server so it uses bounded execution to limit how long code can run
• #19: Requires multiple reads Typically provides faster write speeds
• #20: Data from entities are queried together Only making one round trip to server for all the data
• #21: One to many relationships (unbounded) Blog article comments array could have millions of entries Many-to-many relationships For example, one speaker could have multiple sessions and each session could have multiple speakers Related data changes with differing volatility Document does not change but document with views or likes does
• #22: Use Amazon/Best Buy product example Can have embedded product info that also contains current review summary so don’t have to retrieve each time When you want the rating summary or review you have different query
• #24: Trade-off between performance and consistency Strong: Strong consistency guarantees that a write is only visible after it is committed durably by the majority quorum of replicas. Strong consistency provides absolute guarantees on data consistency, but offers the lowest level of read and write performance Session: ability to read your own writes. A read request for session consistency is issued against a replica that can serve the client requested version (part of the session cookie). Session consistency provides predictable read data consistency for a session while offering the lowest latency writes. Reads are also low latency as except in the rare cases, the read will be served by a single replica. Bounded staleness: provides more predictable behavior for read consistency while offering the lowest latency writes. As reads are acknowledged by a majority quorum, read latency is not the lowest offered by the system. This provides a stronger guarantee than Session or Eventual. Eventual: weakest form of consistency wherein a client may get the values which are older than the ones it had seen before, over time. In the absence of any further writes, the replicas within the group will eventually converge. The read request is served by any secondary index. Eventual consistency provides the weakest read consistency but offers the lowest latency for both reads and writes.
• #27: DocumentDB supports a RESTful protocol Any DocumentDB operation can be performed from any HTTP client as long as request URL points to valid DocumentDB resource and the headers contain the required authentication info
• #30: Throughput = amount of items passing through a system
• #32: Every request you do has a response that displays the RequestUnit charge whether action succeeds or fails Can see charge in QueryExplorer Writes are more expensive than reads
• #33: Standard – pay as you go, allows partitioning, storage measured in GB, 400-250K RUs, 250GB storage max (can be increased) S1, S2, S3 – predefined. 10GB storage max, throughput max 2.5K Rus Can have mix of S1, S2 and S3 collections in a DB; can archive data in an S1 and keep active data in an S3 Once you create an account there is a Minimum S1 charge ($25) even with no collections created But when you create a collection you satisfy the charge and no longer have the $25 fee