Why MySQL High Availability Matters
2 likes1,885 views
The document discusses MySQL high availability (HA), explaining its importance, considerations, and design factors necessary for minimizing downtime and ensuring reliability. It covers various approaches to achieve HA, including replication techniques and MySQL Group Replication, which offers a robust framework for distributed database services. The content emphasizes the need for proper infrastructure and monitoring to meet business continuity requirements and supports the architecture of the MySQL InnoDB Cluster.
![Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication: What Sets It Apart?
• Built by the MySQL Engineering Team
– Natively integrated into Server: InnoDB, Replication, GTIDs, Performance Schema, SYS
– Built-in, no need for separate downloads
– Available on all platforms [Linux, Windows, Solaris, FreeBSD, etc]
• Better performance than similar offerings
– MySQL GCS has optimized network protocol that reduces the impact on latency
• Easier monitoring
– Simple Performance Schema tables for group and node status/stats
– Native support for Group Replication coming to MySQL Enterprise Monitor
• Modern full stack MySQL HA being built around it
– Native end-to-end easy to use sharded InnoDB clusters
22](https://image.slidesharecdn.com/whymysqlhighavailabilitymatters-170121154023/85/Why-MySQL-High-Availability-Matters-22-320.jpg)
Why MySQL High Availability Matters
- 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Why MySQL High Availability Matters Matt Lord MySQL Product Manager @mattalord
- 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 3
- 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 4 High Availability Explained Native MySQL High Availability Highly Available MySQL Reference Architectures What’s Coming Next for MySQL HA and Scale? 1 2 3 4
- 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 5 High Availability Explained Native MySQL High Availability Highly Available MySQL Reference Architectures What’s Coming Next for MySQL HA and Scale? 1 2 3 4
- 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Terms and Concepts • Availability – Outage and downtime – Mean Time Between Failures (MTBF) – Mean Time To Recover (MTTR) – Service Level Agreement (SLA) – The Nine’s Scale : 5 nines = 5 mins downtime per year • Capacity / Performance • Redundancy • Fault Tolerance • Disaster Tolerance/Recovery 6
- 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Considerations • SLA requirements – What is needed to support your business objectives • Operational capabilities – Will you have the workforce to implement and maintain it • Service agility – Can it grow and change as your company does • Time to market – Can you go to market quickly • Budgetary constraints – Cost of downtime versus cost of HA implementation 7 What To Use?
- 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Factors • Environment – Redundant servers in different datacenters and geographical areas will protect you against regional issues—power grid failures, hurricanes, earthquakes, etc. • Hardware – Each part of your hardware stack—networking, storage, servers—should be redundant • Software – Every layer of the software stack needs to be duplicated and distributed across separate hardware and environments • Data – Data loss and inconsistency/corruption must be prevented by having multiple copies of each piece of data, with consistency checks and guarantees for each change 8
- 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Design Factors • Reliability, Availability, and Serviceability (RAS) – Reliability • Quality hardware and software components that you can manage and trust – Availability • Multiple copies of each datum replicated to separate hardware • Multiple copies of each piece of the hardware and software stack • Multiple physical locations – Separate power grids – Enough physical distance to survive disasters such as floods, earthquakes, and power grid outages • Resiliency – Automation to transparently deal with any potential failure in the infrastructure, hardware, software – Serviceability • Ensure that you can monitor, diagnose, maintain, and repair it all 9
- 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Acceptable Performance • Performance outside of acceptable bounds is unavailability – Determine your acceptable application response times – Ensure that you can measure impact from each layer of stack – Determine failure versus load • Scale up/out or failover • Build elasticity into each layer – Quickly and easily scale up and down to handle changing load and traffic patterns 10
- 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: The Causes of Downtime 11 40% 40% 20% Software/Application Human Error Hardware * Source: Gartner Group 1998 survey A study by the Gartner Group projected that through 2015, 80% of downtime will be due to people and process issues
- 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: The Business Cost of Downtime • Calculate a cost per minute of downtime – Average revenue generated per-minute over a year – Cost of not meeting any customer SLAs – Factor in costs that are harder to quantify 1. Revenue 2. Reputation 3. Customer sentiment 4. Stock price 5. Service’s success 6. Company’s very existence 12 THIS is why HA matters!
- 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | High Availability: Database Needs • Management infrastructure – Monitoring of status, health, performance • Facilities for service changes, service transitions • Failure detection and handling – Identify and handle failures • Elasticity – Scale up to ensure acceptable performance is always maintained • Consistency guarantees – Conflict detection and handling ; data loss protection/prevention • Online maintenance 13
- 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | • Recovery Time Objective – Maximum length of downtime before there is break in “business continuity” • Recovery Point Objective – Point in time to which data must be recovered when service is re- established Tier 1 • Mission-critical services Tier 2 • Business-critical services Tier 3 • Task-critical services Tier 4 • Non-critical services 14 High Availability: Does The Application Require It?
- 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Master/Slave Replication Master/Slave with Automated Failover DRBD OS or VM vendor active/passive solutions MySQL Group Replication High Availability: Common MySQL Solutions 9 9 . 9 9 9 % 15 Continuous Availability
- 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 16 High Availability Explained Native MySQL High Availability Highly Available MySQL Reference Architectures What’s Coming Next for MySQL HA and Scale? 1 2 3 4
- 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | C B A C B ACRASH C B A Now B is the new master Uh Oh! It’s OK! A major building block for high availability What is Replication Used For? 17
- 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Group Replication 18 App Servers with MySQL Router MySQL Group Replication “High Availability becomes a core first class feature of MySQL!”
- 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: What Is It? • Group Replication library – Implementation of Replicated Database State Machine theory • MySQL GCS is based on Paxos (variant of Mencius) – Provides virtually synchronous replication for MySQL 5.7+ – Supported on all MySQL platforms • Linux, Windows, Solaris, OSX, FreeBSD “Multi-master update anywhere replication plugin for MySQL with built-in conflict detection and resolution, automatic distributed recovery, and group membership.” 19 App Servers with MySQL Router MySQL Group Replication
- 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | • A highly available distributed MySQL database service – Clustering eliminates single points of failure (SPOF) • Allows for online maintenance – Removes the need for manually handling server fail-over – Provides distributed fault tolerance and self-healing – Enables Active/Active update anywhere setups – Automates reconfiguration (adding/removing nodes, crashes, failures) – Makes it easy to scale up/down based on demand – Automatically ensures data consistency • Detects and handles conflicts • Prevents data loss • Prevents data corruption 20 MySQL Group Replication: What Does It Provide?
- 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: Use Cases • Elastic Replication – Environments that require a very fluid replication infrastructure, where the number of servers has to grow or shrink dynamically and with little pain as possible. • Highly Available Shards – Sharding is a popular approach to achieve write scale-out. Users can use MySQL Group Replication to implement highly available shards in a federated system. Each shard can map into a Replication Group. • Alternative to Master-Slave Replication – It may be that a single master server makes it a single point of contention. Writing to an entire group may prove more scalable under certain circumstances. 21
- 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: What Sets It Apart? • Built by the MySQL Engineering Team – Natively integrated into Server: InnoDB, Replication, GTIDs, Performance Schema, SYS – Built-in, no need for separate downloads – Available on all platforms [Linux, Windows, Solaris, FreeBSD, etc] • Better performance than similar offerings – MySQL GCS has optimized network protocol that reduces the impact on latency • Easier monitoring – Simple Performance Schema tables for group and node status/stats – Native support for Group Replication coming to MySQL Enterprise Monitor • Modern full stack MySQL HA being built around it – Native end-to-end easy to use sharded InnoDB clusters 22
- 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: Architecture Node Types R: Traffic routers/proxies: mysqlrouter, haproxy, sqlproxy, ... M: mysqld nodes participating in Group Replication 23
- 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: Stack • The Group Replication plugin is responsible for – Maintaining distributed execution context – Detecting and handling conflicts – Handling distributed recovery • Detect membership changes • Donate state if needed • Collect state if needed – Proposing transactions to other members – Receiving and handling transactions from other members – Deciding the ultimate fate of transactions • commit or rollback 24 GCS API Replication Plugin Plugin API MySQL Server Group Comm. System (Corosync) Group Com. Engine
- 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: Stack • The Group Communication System API: – Abstracts the underlying group communication system implementation from the plugin itself – Maps the interface to a specific group communication system implementation • The Group Communication System engine: – Variant of Paxos developed at MySQL – Building block to provide distributed agreement between servers 25 GCS API Replication Plugin Plugin API MySQL Server Group Comm. System (Corosync) Group Com. Engine
- 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication: Performance Comparison 26
- 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL InnoDB Cluster: The Full Stack M M M MySQL Connector Application MySQL Router MySQL Connector Application MySQL Router MySQL Shell HA Group Replication 27 Now GA!
- 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Shell 28 A single unified client for all administrative and operations tasks App Servers with MySQL Router MySQL Group Replication MySQL Shell Setup, Manage, Orchestrate • Multi-Language: JavaScript, Python, and SQL – Naturally scriptable • Supports both Document and Relational models • Exposes full Development and Admin API ”MySQL Shell provides the developer and DBA with a single intuitive, flexible, and powerful interface for all MySQL related tasks!” InnoDB Cluster Now GA!
- 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Shell: Admin API • mysql-js> dba.help() • The global variable 'dba' is used to access the MySQL AdminAPI • Perform DBA operations – Manage MySQL InnoDB clusters • Create clusters • Validate MySQL instances • Configure MySQL instances • Get cluster info • Modify clusters Database Administration Interface App Servers with MySQL Router MySQL Group Replication MySQL Shell Setup, Manage, Orchestrate 29 InnoDB Cluster Now GA!
- 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Router • Transparent client connection routing – Load balancing – Application connection failover • Stateless design offers easy HA client routing – A local Router becomes part of the application stack ”MySQL Router allows you to easily migrate your standalone MySQL instances to natively distributed and highly available InnoDB Clusters without affecting existing applications!” 30 Transparent access to HA databases for MySQL Applications App Servers with MySQL Router MySQL Group Replication MySQL Shell Setup, Manage, Orchestrate InnoDB Cluster Now GA!
- 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Router: 2.1 • Native support for InnoDB clusters – Understands Group Replication topology – Utilizes metadata schema stored on each member • Bootstraps itself and sets up client routing for the InnoDB cluster • Allows for intelligent client routing into the InnoDB cluster – Supports multi-master and single primary modes • Core improvements – Built-in keyring for easy and secure password management 31 App Servers with MySQL Router MySQL Group Replication MySQL Shell Setup, Manage, Orchestrate ”MySQL Router 2.1, with the new metadata_cache plugin, provides transparent client connection routing and failover into your InnoDB clusters!” InnoDB Cluster Now GA!
- 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 32 High Availability Explained Native MySQL High Availability Highly Available MySQL Reference Architectures What’s Coming Next for MySQL HA and Scale? 1 2 3 4
- 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hardware and Infrastructure Notes • 3, 5, or 7 machines per group – Isolate machine resources as much as possible – Limit virtualization layers – Machines configured for dedicated database server role • Recommended configuration for HPC setups – 32-64 vCPUs with fast CPU clock (2.5GHz+) – SSDs (for data and replication logs) – High quality network connection between each machine • Low latency, high throughput, reliable • Limit network switches, routers, and hubs as much as possible • Isolated and dedicated network when possible 33
- 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Shared Nothing Cluster – Single Data Center 34 Application Servers MySQL Router in Stack MySQL Database Service Group Replication
- 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Shared Nothing Cluster – Cross Data Center 35 MySQL Database Service Group Replication Data Center 1 Data Center 2 Clients
- 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Geographically Redundant Cluster 36 Async Replication Active Data Center Backup Data Center Clients
- 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Active/Active Multi-Data Center Setup 37 Async Replication Regional Data Center Regional Data Center Regional ClientsRegional Clients
- 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 38 High Availability Explained Native MySQL High Availability Highly Available MySQL Reference Architectures What’s Coming Next for MySQL HA and Scale? 1 2 3 4
- 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 39 MySQL HA Out-Of-Box HA Read Scale-Out Async Replication + Auto Failover Write Scale-Out ShardingS1 S2 S3 * InnoDB Cluster Now GA!
- 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | S1 S2 S3 S4 S… M M M MySQL Connector Application MySQL Router MySQL Connector Application MySQL Router MySQL Shell HA MySQL InnoDB Cluster: Architecture – Step 2 Group Replication Read-Only Slaves 40
- 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | S1 S2 S3 S4 S… M M M MySQL Connector Application MySQL Router MySQL Connector Application MySQL Router MySQL Shell HA ReplicaSet1 S1 S2 S3 S4 S… M M M MySQL Connector Application MySQL Router HA ReplicaSet2 ReplicaSet3 MySQL Connector Application MySQL Router MySQL InnoDB Cluster: Architecture – Step 3 S1 S2 S3 S4 M M M HA 41
- 42. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Sharded InnoDB Clusters • Group Replication – Each shard is a highly available replica set • MySQL Router – Manages shard mappings and related metadata – Manages client routing – Provides cross shard execution framework • On top of distributed query execution facilities present in Server • MySQL Shell – Exposes management and orchestration features 42
- 43. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Enterprise Monitor • Native holistic support for Group Replication / InnoDB clusters (GA in 3.4!) – Topology views – Detailed metrics and graphs – Best Practice advice • Monitoring of MySQL Routers 43
- 44. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 44 MySQL InnoDB Cluster: The End Goal M App M M MySQL Shell and Orchestration Tooling MApp M M Simple Shard Mapping, State and Extra Metadata Control, Coordinate, Provision ... Monitoring (MEM) MySQL Router Group Replication – Shard 1 Group Replication – Shard N MySQL Router
- 45. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Cloud: MySQL Cloud Service • MySQL Enterprise Edition • Web based console to manage your MySQL Cloud instances • Self-Service Provisioning • Elastic Scalability • Multi-Layered Security • Unified Cloud Management • Oracle PaaS and IaaS Integration • Premier Technical Support included 45 Integrated HA and DR service options – utilizing InnoDB Clusters – coming soon!