Choosing the Right Database for My Workload: Purpose-Built Databases
1 like794 views
The document discusses choosing the right database for different types of workloads. It covers operational databases like Amazon DynamoDB, Amazon RDS, Amazon ElastiCache and Amazon Neptune that are well-suited for transactional workloads. It also discusses analytic databases like Amazon Redshift, Amazon Athena, Amazon Kinesis Analytics and Amazon Elasticsearch Service that are well-suited for large-scale analytics and business intelligence workloads. The document emphasizes that AWS offers a variety of purpose-built databases and there is no need to pick just one, as different databases can be combined to solve different aspects of a problem.
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
My [insert your favorite DB] works for everything
General purpose Special purpose
One size fits all Efficiency at scale](https://image.slidesharecdn.com/purpose-builtdatabases-180907062900/85/Choosing-the-Right-Database-for-My-Workload-Purpose-Built-Databases-8-320.jpg)
Choosing the Right Database for My Workload: Purpose-Built Databases
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Ozkan Can, Senior Solutions Architect, AWS Choosing the Right Database for My Workload: Purpose-Built Databases
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Looking at the Specifics Purpose of a database Your application needs
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A Short Break from Generalities Relational Non-Relational NoSQL SQL Schema Schema-free Unstructured Structured
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Database Workloads Data Considerations Shape Size Compute
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Shape Purpose-Built For Optimized for When you need to Example Workload Row Store Operate on a record or group of records Payroll Column Store Aggregations, scans and joins Analytics Key-Value Store Query by key with high throughput & fast ingestion Tracking devices Document Store Index & store documents for query on any property Patient data Graph Store Persist and retrieve relationships Recommendations Time-Series Store Store and process data sequence Process Engine telemetry Unstructured Store Get and put of objects Store user reviews
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Size Considerations Example Workload Size at limit – bounded or unbounded Number of employees – bounded Number of sensors – unbounded Working set size & caching 10-years of sales data but only the last 12-months is queried Session data for users of a streaming service Retrieval size Get one row Get one thousand rows Partitionable or monolithic Storage and processing of car location data is partitionable Company payroll data has no natural partition boundary
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Compute Considerations Example Workload Compute functions Sum of sales for the last 12-months Get & Put data Throughput Million users browsing a product catalogue every second 50 doctors looking at 300 patient records per day Latency Get the location of a car in 5 milliseconds Get the min, max & average deal size for the last 12-months in 5 seconds Change rate Inventory counts are frequently updated Sales records are never updated Rate of ingestion Location telemetry from cars added to the database every minute New employees records being added to the database
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. My [insert your favorite DB] works for everything General purpose Special purpose One size fits all Efficiency at scale
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. But Which Database to Use When? Decision points and considerations
- 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Managed database services DevOps Build Deploy Operate code, integrate, test provision, configure, rollout secure, monitor, scale, HA All conveniently located at the end of an API call
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. But Which Databases to Use When? Why pick just one?
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Our Strategy
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Operational Analytics transactional retrospective system of record streaming content management predictive Back to Generalities
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Operational Analytics transactional retrospective system of record predictive content management streaming
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. General characteristics • Usually a good fit for caching • Small compute size – few rows, items, documents per request • Low-latency • High-throughput • High-concurrency • Mission critical HA, DR and data protection Primary dimensions to consider • Size at limit – bounded or unbounded • Rows, key-values or documents • Need relational capabilities or not • partitioned or monolithic • Push-down compute requirements • Change velocity • Ingestion requirements Operational workloads
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Relational or Not Referential integrity with strong consistency, transactions, and hardened scale GraphKey-value Document ; Relational Low-latency key based queries with high throughput and fast ingestion of data Indexing and storing documents with support for query on any property Creating and navigating relations between data easily and quickly Complex query support via SQL Simple query methods with filters Simple query with filters, projections and aggregates Easily express queries in terms of relations
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Relational Database Service (RDS) Managed relational database service with a choice of six popular database engines Easy to administer Highly scalable Available & Durable Fast No need for infrastructure provisioning, installing and maintaining database software Scale database compute and storage with a few mouse clicks with no downtime Multi-AZ: Automatically replicates data to in a different AZ. Automated backup, snapshots, failover Chose between 2 SSD- backed storage for high performance OLTP
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon DynamoDB F a s t a n d f l e x i b l e N o S Q L d a t a b a s e s e r v i c e f o r a n y s c a l e NoSQL database that supports both document and key-value structures Fast, consistent performance Highly scalable Fully managed Business Critical Reliability Consistently single-digit millisecond latencies at any scale. DAX speeds up times to microseconds. Auto-scaling tables serving millions of requests per second, storing hundreds of terabytes of data. Automatic provisioning and infrastructure management. Data is replicated across fault tolerant availability zones, with fine-grained access control.
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fully managed, in- memory cache for DynamoDB. Reduces DynamoDB response times from milliseconds to microseconds. Amazon DynamoDB Accelerator (DAX) Fully managed write-through cache for DynamoDB
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon ElastiCache Extreme Performance Secure & hardened Easily scalable Highly available & reliable In-memory data store and cache using optimized stack to deliver sub-millisecond response times VPC for cluster isolation, encryption at rest/transit, and HIPAA compliance Read scaling with replicas. Write and memory scaling with sharding. Non disruptive scaling Multi-AZ with automatic failover Managed, in-memory data store service. Redis or Memcached to power real-time apps with sub-millisecond latency.
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Neptune F u l l y m a n a g e d g r a p h d a t a b a s e Fast & Scalable Reliable Open Store billions of relationships and query with milliseconds latency Six replicas of your data across three AZs with full backup and restore Build powerful queries easily with Gremlin and SPARQL Supports Apache TinkerPop & W3C RDF graph models Gremlin SPARQL Easy
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Operational database dimensions Size at limit – bounded Size at limit – unbounded key-values or documents Rows Graph Data Need relational capabilities Partitioned Push-down compute requirements Change Velocity Amazon DynamoDB Amazon RDS Amazon Neptune
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Operational Analytics transactional retrospective system of record predictive content management streaming
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. General characteristics • Almost always a columnar • Large and usually partitioned • large compute size – millions of items involved in query • Heavy compute push down • Batch writes or trickle inserts • Little to no updates • Needs a lot of memory and often in-memory compute capabilities Primary dimensions to consider • Streaming or not • Latency requirements • ETL or no ETL • Serverless or dedicated compute • Always active or occasionally active • Data formats Analytic workloads
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Athena – Interactive analysis Interactive query service to analyze data in Amazon S3 using standard SQL No infrastructure to setup or manage and no data to load $ SQL Query Instantly Zero setup cost. Just point to S3 and start querying Pay per query Pay only for queries run. Save 30-90% on per query costs through compression Open ANSI SQL interface, JDBC/ODBC drivers, Multiple formats, compression types, and complex Joins and data types Easy Serverless. Zero Infrastructure. Zero Administration
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Redshift – Data Warehousing Fast, powerful, and simple data warehousing at 1/10 the cost Massively parallel, petabyte scale $ Fast Inexpensive Scalable Secure Columnar storage technology to improve I/O efficiency and parallelize queries. Data load scales linearly. As low as $1,000 per terabyte per year, 1/10th the cost of traditional data warehouse solutions. Resize your cluster up and down as your performance and capacity needs change. Data encrypted at rest and transit. Isolate clusters with VPC. Manage your own keys with AWS KMS.
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Kinesis Analytics Process and Analyze Streaming Data in Real-time with SQL Capture streaming data with Kinesis Streams or Kinesis Firehose Run standard SQL queries against data streams Kinesis Analytics can send processed data to analytics tools so you can create alerts and respond in real-time KINESIS FIREHOSE KINESIS STREAMS KINESIS ANALYTICS
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Elasticsearch Service Easy to Use Fully-managed. Deploy production-ready clusters in minutes. Open Direct access to Elasticsearch open-source APIs. Supports Logstash and Kibana. Secure Secure access with VPC to keep all traffic within AWS network. Available Zone awareness replicates data between two AZs. Automatically monitors & replaces failed nodes. Easy to deploy, secure, operate, and scale Elasticsearch Customers use Elasticsearch for log analytics, full text search, & application monitoring
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Analytics database dimensions Streaming analytics Serverless ad-hoc query Process, prepare and index in-place Low-latency for reporting and BI dashboards Pay per query Data warehouse with multiple enterprise data sources Query data directly in S3 without format conversions Directly query CSV, JSON, TSV or text files Amazon Redshift AthenaKinesis Analytics
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Well Modelled DataData Exploration Non-SQL Analytics Real-Time analytics Managed Storage Delivery Amazon Elasticsearch Service Kinesis Analytics Athena Amazon Redshift Amazon S3 Why pick one when you can use all three?
- 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DW | Big Data Processing | Interactive AWS Databases and Analytics B r o a d a n d d e e p p o r t f o l i o , p u r p o s e - b u i l t f o r b u i l d e r s Business Intelligence & Machine Learning Data Movement Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams QuickSight Relational Databases RDS Aurora Data Lake S3/Glacier Glue (ETL & Data Catalog) SageMaker Non-Relational Databases Analytics DynamoDB ElastiCache (Redis, Memcached) Neptune (Graph) Redshift EMR Athena Kinesis Analytics Elasticsearch Service Real-time Operational Databases
- 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. In Closing AWS offers a myriad of services designed to help you solve your toughest problems at scale – no need to just pick one service When selecting a data service, consider the dimensions and pick the best match for each component of your application
- 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Go Build Something! Q&A