The Proper Care and Feeding of MySQL Databases
1 like137 views
The document provides guidance for Linux administrators on the effective management of MySQL databases. Key topics include hardware requirements, software updates, backup strategies, and replication setups to ensure database performance and reliability. Best practices are emphasized, including the importance of memory, security measures, and tools to facilitate administrative tasks.
The Proper Care and Feeding of MySQL Databases
- 1. The Proper Care and Feeding of MySQL Databases For Busy Linux Admins
- 2. Dave Stokes MySQL Community Manager Started programming with FORTRAN on Punch Cards Forced to UNIX/BSD/Linux Please buy my book 2
- 3. MySQL MySQL is 23 years old Old timers prefer βMy - ess - queue - ellβ Own by Oracle for eight years MySQL 8 is current Generally Available Release -- talk at 10:30 in Room A 3
- 4. Databases are ... Selfish Want entire system to self Messy Suck up memory, disk space, bandwidth, sanity Growing all the time Suck up a good part of your life! 4
- 5. Databases are the toddlers of software!! 5
- 6. Disclaimer And it is also the view of most admins (and developers) who also have DBA duties added to all their other regular tasks to help fill all their lavish spare timeThe last slide was my opinion 6
- 7. So What Makes a Happy MySQL Database? β Hardware β Software β Backups & Replication β Tools to make life easier β Configuration suggestions β Q&A 7
- 8. Hardware 8
- 9. Hardware Happiness Databases LOVE memory β Spend money on good memory β Lots of it β Second comes cores β Databases now also love cores, but not as much as memory 9
- 10. Why memory? Reading from memory is 100,000 faster than reading from disk β 100k miles is four times around the earth β 100,000 days is 273.9726 years β 100,000 seconds is 27.7777 hours So 100,000 is a lot! 10
- 11. Disks or disk like things β Move data to separate controller/disk, especially from logs β Do not log on slow devices β RAID to your favorite level (RAID 10 minimum) or go to the cloud β Going to the cloud moves some of the labor onto someone else (maybe) β Do not use consumer grade disks β Use disks designed for 7/24/365 operation, not by price point β CACHES - disk and controller β Write through or write back caches β Both lie β Make sure they do not auto tune during production hours β Faster devices are on their way!! 11
- 12. Network β Never expose your instances to the outside world β SCRUB rigorously all user data β Keep separate net for replication, and/or backups 12
- 13. Understand MySQL Authentication Process 1. Is the host allowed to talk to the MySQL server a. Is the ip address or domain whitelisted in the mysql.user table 2. Is the user allowed to talk to the MySQL server a. In the mysql.user table? b. Not restricted by some other constraint? 3. Password or User Authentication string valid? Designed to be overly enthusiastic - first match gets in β Caution: [email protected] may not equal boss@home β Set configuration to not do DNS lookup incase DNS gets overloaded β Zone updates can make your server crawl 13
- 14. Replication Servers MySQL async or semi-sync replication needs to have bigger, badder hardware slave servers than masters Do more work Use the MySQL Utilities to clone servers, set up replication Use a dedicated network to avoid network contentions 14
- 15. MySQL Group Replication MySQL Group Replication provides distributed state machine replication with strong coordination between servers. Servers coordinate themselves automatically when they are part of the same group. The group can operate in a single-primary mode with automatic primary election, where only one server accepts updates at a time. Alternatively, for more advanced users the group can be deployed in multi-primary mode, where all servers can accept updates, even if they are issued concurrently. 15
- 16. Software Run the latest, greatest version of MySQL that you can β Performance β Bug fixes β Features β Security Keep MySQL by itself β Databases do not play well with others β Contention for resources β Swapping β Maybe use a caching layer but watch memory user 16
- 17. Database Deltas BTW your corporate legal will probably ask you this question in the future 17 What do you do if you have to restore data from an old dataset with a schema that is no longer around?
- 18. Proxies 18 MySQL Router is part of InnoDB cluster, and is lightweight middleware that provides transparent routing between your application and back-end MySQL Servers. It can be used for a wide variety of use cases, such as providing high availability and scalability by effectively routing database traffic to appropriate back-end MySQL Servers.
- 19. Containers docker pull mysql/mysql-server:tag The tag is the label for the image version you want to pull (for example, 5.5, 5.6, 5.7, 8.0, or latest). If :tag is omitted, the latest label is used, and the image for the latest GA version of MySQL Community Server is downloaded 19
- 21. Thou shalt make backups Make sure you backup your data so frequently it verges on being painful and then look for ways to increase that! Know how to restore entire instances, entire databases, or a table. Save views, functions, stored procedures, and etc. And others on your staff need to be able to do this too (cross training)! Keep backups off site and test randomly Nobody has ever been fired for doing too many backups! -- Paranoia should be your friend 21
- 22. Replication 22
- 23. MySQL Replications is easy to set up Three types: 1. Async - slave grabs copy of changes from master and applies those changes to own set of data, master unaware of what slave is doing 2. Semi-sync - master waits for acknowledgement from at least one slave before proceeding a. Both async and sync have three forms -- Statement, Row, and mixed b. Single threaded before 5.6, multi threaded for different databases in 5.6, and multi threaded within a database with 5.7 3. Group replication 23
- 24. Async/Semi Sync Replication Filters β You do not need to replicate everything β Check churn of data, maybe 1x day backup (plus make sure to save binary logs) β Filter tables β You can change tables on the fly starting with MySQL 5.7 β βSomething is going on in manufacturing, can we get ALL of their data copied someplaces?β 24
- 25. Global Transactions IDs β Introduced in MySQL 5.6 β Each transaction has a unique GTID β Easy for slaves to get caught up to master β No longer have to look at file offsets on master and slave to get start position β Saves time, $, and sanity. Easy to automate β Storing replication data in InnoDB tables plus checksums make it crash safe β Row based replication can exploit only sending key and changed items, not entire row 25
- 26. Replication for Backup β Async/semi uses three threads on slave server β Feed data from master β To Log β Log to data β Shot down log to data thread, run backup, then restart log to data β Data from master will be stored but not applied during backup 26
- 27. In an Oracle Database Shop?? If you also have a big Oracle DB shop: β You can backup to big Oracle STB backup devices β MySQL works with Oracle Audit Vault, Database Firewall, and Key Manager β Need to have a support contract for these three 27
- 28. Tools to make life easier 28
- 29. There are lots of tools for DBA chores β Monitoring β Yes, you need to monitor β Administration β Command line versus GUI β Document instances β Backups 29
- 30. Monitoring β Active β Watched instances and sends alerts β MySQL Enterprise (support customers) β Percona Monitor β Nagios, Cacti, etc. β Your favorite (not mentioned) β Helps to be able to comb historical data β Semi-active β MySQL Workbench β Dashboard and SYS Schema β PHPMyAdmin β Your favorite (still not mentioned) 30
- 31. MySQL Workbench β Query tool β Visual explain to help in optimization β Admin tool β User setup, backups, imports, change settings β No more fat finger βUPDATE user SET SELECT_PRIV=βYβ, UPDATE_PRIV=βYβ,... β Dashboard β SYS Schema β Entity Relationship Matter β Migration tool β More β Free! 31
- 32. And more β Percona toolkit β Toad for MySQL from Dell β Your Favorite Tool (again not mentioned) 32
- 34. These are suggestions!!! β Turn off DNS lookups β Use skip-name-resolve β Save/Load statistics β Use innodb_stats_persistent β innodb_buffer_pool_dump=ON β innodb_buffer_pool_dump_at_shutdown=ON β innodb_buffer_pool_load_at_startup=ON β See Config Persistent Optimizer Statistics Parameters in the MySQL Manual 34
- 35. Suggestions continued β Use SET PERSIST (MySQL 8) 35
- 36. Suggestions continued β Tune log level (MySQL 5.7+) β Log_error_verbostiy - errors, errors & warnings, E&W + notes β Send to syslog β Turn off query cache (5.7 default, 8.0 deprecated) β Single threaded β Free up memory β Set innodb_buffer_pool_size = 75-80% physical ram 36
- 37. Big Hints! 37
- 38. Big Hint #1 BE DAMN STINGY with permissions and grants!! β Easier to say βnoβ than constantly be restoring β Use --safe-updates or --I-am-a-dummy in ~/.my.cnf β No more βopps, I forgot the where clauseβ 38
- 39. BIG Hint #2 Please use SYS Schema View, functions, and stored procedures on top of performance schema and information schema β Who is hogging resources β Indexes not being users β Problematic queries β Other routine PITAs 39
- 40. Big Hint #3 Security β Secure by default install (5.7+) β Forced root password β No more anonymous accounts, no test db β Password rotation β Password rules β Length, characters 40
- 41. Big Hint #4 Take Advantage of the new MySQL shell mysql-js> util.checkForServerUpgrade("root@localhost:3306"); 1) Usage of db objects with names conflicting with reserved keywords in 8.0 2) Usage of utf8mb3 charset Warning: The following objects use the utf8mb3 character set. It is recommended to convert them to use utf8mb4 instead, for improved Unicode support. simple_schema.city.name - column's default character set: utf8 3) Usage of use ZEROFILL/display length type attributes Notice: The following table columns specify a ZEROFILL/display length attributes. Please be aware that they will be ignored in MySQL 8.0 test.big_table.ORDINAL_POSITION - bigint(21) unsigned 4) Issues reported by 'check table x for upgrade' command No issues found 5) Table names in the mysql schema conflicting with new tables in 8.0 No issues found 6) Usage of old temporal type No issues found 7) Foreign key constraint names longer than 64 characters No issues found No fatal errors were found that would prevent a MySQL 8 upgrade, but some potential issues were detected. Please ensure that the reported issues are not significant before upgrading. 41