Amazon Aurora (MySQL, Postgres)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ian Meyers
August 30, 2018
Database Web Day
Amazon Aurora 101 and What’s New

Agenda
• History of Amazon Aurora
• How Aurora achieves high performance,
availability, security, and monitoring?
• What’s new on Aurora roadmap in 2018?
• Common use cases
• Migration tools and resources

Amazon Aurora ……
Databases reimagined for the cloud
Delivered as a managed service
 Speed and availability of high-end commercial databases
 Simplicity and cost-effectiveness of open source databases
 Drop-in compatibility with MySQL and PostgreSQL
 Simple pay as you go pricing

Amazon Aurora is…
Fully managed service – automate administrative tasks
1
2
3
Scale-out, distributed, multi-tenant design
Service-oriented architecture leveraging AWS services

Scale-out, distributed, multi-tenant architecture
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Shared storage volume
Storage nodes with SSDs
Master
SQL
Transactions
Caching
• Purpose-built log-structured
distributed storage system
designed for databases
Replica Replica
SQL
Transactions
Caching
SQL
Transactions
Caching
• Storage volume is striped across
hundreds of storage nodes
distributed over 3 different
Availability Zones
• Master and replicas all point to the
same storage
• Six copies of data, two copies in
each Availability Zone to protect
against AZ+1 failures

Leveraging cloud eco-system
AWS Lambda
Amazon S3
IAM
Amazon CloudWatch
Invoke AWS Lambda events from stored procedures/triggers.
Load data from Amazon S3, store snapshots and backups in S3.
Use AWS Identity & Access Management (IAM) roles to manage database
access control.
Upload systems metrics and audit logs to Amazon CloudWatch.

Automate administrative tasks
Schema design
Query construction
Query optimization
Automatic fail-over
Backup & recovery
Isolation & security
Industry compliance
Push-button scaling
Automated patching
Advanced monitoring
Routine maintenance
Takes care of your time-consuming database management tasks, freeing you
to focus on your applications and business
You
AWS

Aurora is used by ¾ of the top 100 AWS customers
Aurora customer adoption
Fastest growing service in AWS history

Agenda
• How Aurora achieves high
performance, availability, security, and
monitoring?
• What’s new on Aurora roadmap in 2018?

Amazon Aurora is fast …
5x faster than MySQL

WRITE PERFORMANCE READ PERFORMANCE
MySQL SysBench results
R3.8XL: 32 cores / 244 GB RAM
5X faster than RDS MySQL 5.6 & 5.7
Five times higher throughput than stock MySQL
based on industry standard benchmarks.
0
25.000
50.000
75.000
100.000
125.000
150.000
0
100.000
200.000
300.000
400.000
500.000
600.000
700.000
Aurora MySQL 5.6 MySQL 5.7

With user connection With number of tables
With database size - SYSBENCH With database size - TPCC
Connections
Amazon
Aurora
RDS MySQL
w/ 30K IOPS
50 40,000 10,000
500 71,000 21,000
5,000 110,000 13,000
Tables
Amazon
Aurora
MySQL
I2.8XL
local SSD
RDS MySQL
w/ 30K IOPS
(single AZ)
10 60,000 18,000 25,000
100 66,000 19,000 23,000
1,000 64,000 7,000 8,000
10,000 54,000 4,000 5,000
8x
U P T O
F A S T E R
11x
U P T O
F A S T E R
DB Size
Amazon
Aurora
RDS MySQL
w/ 30K IOPS
1GB 107,000 8,400
10GB 107,000 2,400
100GB 101,000 1,500
1TB 26,000 1,200
DB Size Amazon Aurora
RDS MySQL
w/ 30K IOPS
80GB 12,582 585
800GB 9,406 69
21
U P T O
F A S T E R
136x
U P T O
F A S T E R
Aurora Performance Scales with Heavy Workloads

Do fewer I/Os
Minimize network packets
Cache prior results
Offload the database engine
DO LESS WORK
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
DATABASES ARE ALL ABOUT I/O
NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND
HIGH-THROUGHPUT PROCESSING IS ALL ABOUT CONTEXT SWITCHES
How does Aurora Achieve Higher Performance?

BINLOG DATA DOUBLE-WRITELOG FRM FILES
TYPE OF WRITE
MYSQL WITH REPLICA
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Instance
Replica
Instance
1
2
3
4
5
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
ASYNC
4/6 QUORUM
DISTRIBUTED
WRITES
Replica
Instance
AMAZON AURORA
780K transactions
7,388K I/Os per million txns (excludes mirroring, standby)
Average 7.4 I/Os per transaction
MySQL I/O profile for 30 min Sysbench run
27,378K transactions 35X MORE
0.95 I/Os per transaction (6X amplification) 7.7X LESS
Aurora IO profile for 30 min Sysbench run
Comparison of Aurora I/O Profile

What about availability
“Performance only matters if your database is up”

Aurora has a 6-way replicated storage for HA
Six copies across three Availability Zones
• 4 out of 6 write quorum; 3 out of 6 read quorum
• Peer-to-peer replication for repairs
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Read availabilityRead and write availability

Also have up to 15 promotable read replicas
Master
Read
Replica
Read
Replica
Read
Replica
Shared distributed storage volume
Reader end-point
► Up to 15 promotable read replicas across multiple availability zones
► Re-do log based replication leads to low replica lag – typically < 10ms
► Reader end-point with load balancing and auto-scaling * NEW *

When Database fails – recovery is fast <30 seconds
0,00%
5,00%
10,00%
15,00%
20,00%
25,00%
30,00%
35,00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
0 - 5s – 30% of fail-overs
0,00%
5,00%
10,00%
15,00%
20,00%
25,00%
30,00%
35,00%
40,00%
45,00%
50,00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
0%
10%
20%
30%
40%
50%
60%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
0%
5%
10%
15%
20%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

Amazon Aurora is easy to use
Automated storage management, security and compliance,
advanced monitoring, database migration.

Everything you get from Amazon RDS . . .
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High availability
DB software installs
OS installation
App optimization
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High availability
OS installation
App optimization
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High availability
OS installation
App optimization
Database on-premises Database on EC2 Amazon RDS
Managed
by you
Managed
by AWS

…and more
 Automatic storage scaling up to 64 TB—no performance impact
Up to 64TB of storage – auto-incremented in 10GB units
up to 64 TB
 Continuous, incremental backups to Amazon S3
 Instantly create user snapshots—no performance impact
 Automatic restriping, mirror repair, hot spot management, encryption

Fast database cloning
Create a copy of a database without
duplicate storage costs
• Creation of a clone is nearly instantaneous –
we don’t copy data
• Data copy happens only on write – when
original and cloned volume data differ
Typical use cases:
• Clone a production DB to run tests
• Reorganize a database
• Save a point in time snapshot for analysis
without impacting production system.
Production database
Clone Clone
Clone
Dev/test
applications
Benchmarks
Production
applications
Production
applications

Security and Monitoring

Security and compliance
 Network isolation with Amazon Virtual Private
Cloud (VPC)
 AWS Identify and Access Management (IAM)
based resource-level permission controls
 Encryption to secure data at rest using
customer managed keys
• AES-256; hardware accelerated
• All blocks on disk and in Amazon S3 are encrypted
• Key management via AWS KMS
 Encrypted cross-region replication, snapshot
copy - SSL to secure data in transit
 Advanced auditing and logging without any
performance impact
Data Key 1 Data Key 2 Data Key 3 Data Key 4
Customer Master
Key(s)
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Database
Engine

Database activity monitoring and insights
Search: Look for specific events across log files.
Metrics: Measure activity in your Aurora DB cluster.
 Continuously monitor activity in your DB clusters by sending these audit logs to CloudWatch Logs.
 Export to S3 for long term archival; analyze logs using Athena; visualize logs with QuickSight.
Visualizations: Create activity dashboards
Alarms: Get notified or take actions
Amazon Aurora Amazon CloudWatch
Amazon Athena
Amazon QuickSight
S3

Industry certifications
 Amazon Aurora gives each database
instance IP firewall protection
 Aurora offers transparent encryption at rest
and SSL protection for data in transit
 Amazon VPC lets you isolate and control
network configuration and connect
securely to your IT infrastructure
 AWS Identity and Access Management
provides resource-level permission
controls

Amazon Aurora saves you money
1/10th of the cost of commercial databases
Cheaper than even MySQL

Cost of ownership: Aurora vs. MySQL
MySQL configuration hourly cost
Primary
r3.8XL
Standby
r3.8XL
Replica
r3.8XL
Replica
R3.8XL
Storage
6 TB / 10 K PIOP
Storage
6 TB / 10 K PIOP
Storage
6 TB / 5 K PIOP
Storage
6 TB / 5 K PIOP
$1.33/hr
$1.33/hr
$1.33/hr $1.33/hr
$2,42/hr
$2,42/hr $2,42/hr
Instance cost: $5.32 / hr
Storage cost: $8.30 / hr
Total cost: $13.62 / hr
$2,42/hr

Aurora configuration hourly cost
Total cost: $9.29 / hr
Primary
r3.8XL
Replica
r3.8XL
Replica
R3.8XL
Storage / 6 TB
$1.62 / hr $1.62 / hr $1.62 / hr
$4.43 / hr
*At a macro level Aurora saves over 50% in
storage cost compared to RDS MySQL.
31.8%
Savings
 No idle standby instance
 Single shared storage volume
 No PIOPs – pay for use I/O
 Reduction in overall IOP

Further opportunity for saving
Total cost: $6.86 / hrStorage IOPs assumptions:
1. Average IOPs is 50% of Max IOPs
2. 50% savings from shipping logs vs. full pages
49.6%
Savings
Primary
r3.8XL
Replica
r3.8XL
Replica
r3.8XL
Storage / 6TB
$0.81 / hr $0.81 / hr $0.81 / hr
$4.43 / hr
r3.4XL r3.4XL r3.4XL
 Use smaller instance size
 Pay-as-you-go storage

Higher performance, lower Cost
 Fewer instances needed
 Smaller instances can be used
Safe.com lowered their bill by 40% by switching from sharded
MySQL to a single Aurora instance.
Double Down Interactive (gaming) lowered their bill by 67%
while also achieving better latencies (most queries ran faster)
and lower CPU utilization.
 No need to pre-provision storage
 No additional storage for read replicas

Agenda
• How Aurora achieves high performance,
availability, security, and monitoring?
• What’s new on Aurora roadmap in
2018?

Aurora MySQL 5.7 compatibility Available Now
Aurora MySQL 1.x = MySQL 5.6 compatible
Aurora MySQL 2.x = MySQL 5.7 compatible
• Available now
o JSON support
o Generated Columns
o Spatial indexes (already supported in Aurora 5.6)
o …
o General performance improvements
• Coming later
o Performance Schema
o GTID
o Multi-source replication
o Selective replication
o …

“Backtrack” provides near-instantaneous restores
Available Now
Backtrack quickly brings the database to a desired point in time.
No restore from backup. No copying of data. Not destructive – can backtrack many times.
Quickly recover from unintentional DML/DDL operations.
T0 T1 T2
T0 T1
T2
T3 T4
T3
T4
REWIND TO T1
REWIND TO T3
INVISIBLE INVISIBLE

Parallel Query In Preview
Aurora storage has thousands of CPUs
 Presents opportunity to push down and parallelize
query processing using the storage fleet
 v1: single-table predicates (selections,
projections, 200+ SQL functions, case
statements, filters) and hash joins
 v2: group by, order by, aggregation
 Moving processing close to data reduces network
traffic and latency
DATABASE NODE
STORAGE NODES
PUSH DOWN
PREDICATES
AGGREGATE
RESULTS

Parallel Query – expect to bring orders of magnitude faster
OLAP queries
Latency (seconds)
Decision support benchmark, R3.8xlarge, cold buffer cache
Improvement factor
with Parallel Query
24.6x
18.3x
5.0x
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Aggregate + 2-table join
Aggregate query
Point query on non-indexed column
With Parallel Query Without Parallel Query

Aurora Serverless Available Now
When you provision a database,
Aurora Serverless:
• Creates an Aurora storage volume
• Provisions proxy endpoint in your
VPC for application connection
• Configures network load
balancing behind proxy
• Initializes request routers to route
database traffic
• Provisions initial capacity
Use cases include: Infrequently used
applications (e.g. low-volume blog
site); spiky workload; Test &
Development databases DATABASE STORAGE
APPLICATION
CUSTOMER VPC
VPC PROXY
ENDPOINTS
VPC
ENDPOINTS
NETWORK LOAD BALANCER
REQUEST
ROUTERS
INITIAL
CAPACITY

Performance Insights Available Now
Dashboard showing
Load on Database
• Easy
• Powerful
Identifies source of bottlenecks
• Top SQL
Adjustable time frame
• Hour, day, week , month
• Up to 35 days of data
Max CPU

Performance Insights for Aurora
Amazon RDS Performance Insights
• DB load: average active sessions
• Identifies database bottlenecks
• Easy
• Powerful
• Top SQL/most intensive queries
• Enables problem discovery
• Adjustable timeframe
• Hour, day, week, and longer
• Available now for Aurora PostgreSQL
and Aurora MySQL (Preview)
• Coming soon for all RDS engines

Who is moving to Aurora and why?
Higher performance – up to 5x
Better availability and durability
Reduces cost – up to 60%
Easy migration; no application change
One-tenth of the cost; no licenses
Integration with cloud eco-system
Comparable performance and availability
Migration tooling and services
Customers using
open source engines
Customers using
commercial engines

Enabling fast business growth and infrastructure scaling
• One of the fastest growing SaaS
company focused on marketing
automation
• Over 150,000 marketers use
ActiveCampaign to get better results
• 25x the number of customers in 5
years with increased performance
and user satisfaction
SaaS company focused
on marketing automation

Migrate from commercial databases to achieve
efficiency
Ryanair is the largest
European Airline
By migrating from Microsoft SQL Server to Amazon
Aurora, Ryanair can now run one of the largest email
marketing campaigns in Europe with higher performance
at a fraction of the cost, sending out 22 million emails
daily to customers about travel bookings or sales events.
“We’ve chosen to work with the world’s leading cloud to
develop and deliver services that will transform our
customers’ travel experiences. By rebuilding core
applications, converting data into actionable insights, and
creating intelligent applications, we are putting the
solutions in place to continue our leadership in the travel
industry,”
John Hurley, Chief Technology Officer at Ryanair.

Improved Performance, Scalability and Lower Cost
The Challenge
Needed real-time business intelligence and
analytics on a rapidly growing collection of
online travel marketplace data
SQL Server-based architecture was too
expensive. Performance degraded as data
volume grew.
The Solution
Aurora met their scale and performance
requirements at a much lower cost.
Travel Data Application inserts 300M rows of
data per day with a peak of 70,000 rows per
second.
Expedia is the world’s leading
online travel company with a
portfolio that includes 150
travel sites in 70 countries.

AWS Database Migration Service

AWS Schema Conversion Tool
Features
• Oracle and Microsoft SQL Server schema conversion to MySQL,
Amazon Aurora, MariaDB, and PostgreSQL
• Convert your schema between PostgreSQL and any MySQL engine
• Database Migration Assessment report for choosing the best target
engine
• Code browser that highlights places where manual edits are required
• Secure connections to your databases with SSL
• Cloud-native code optimization
Automates many database schema and code
conversion tasks when migrating between database
engines or data warehouse engines

Common Amazon Aurora migration options
Source database From where Recommended option
RDS
EC2, on premises
EC2, on premises, RDS
Console-based automated
snapshot ingestion and catch
up via binlog replication.
Binary snapshot ingestion
through S3 and catch up via
binlog replication.
Schema conversion using
SCT and data migration via
DMS.

Thank you!
Please enter questions in GoToWebinar

Amazon Aurora (MySQL, Postgres)

Recommended

More Related Content

Similar to Amazon Aurora (MySQL, Postgres) (20)

More from AWS Germany (20)

Recently uploaded (20)

Amazon Aurora (MySQL, Postgres)