Practical my sql performance optimization
0 likes461 views
The document discusses various topics around optimizing MySQL performance including: - Types of indexes in MySQL and when each is used or not used - Factors that prevent indexes from being used and index hints to force usage - Abusing and optimizing the query cache - Optimizing queries for InnoDB and MyISAM storage engines - Controlling the MySQL optimizer through configuration parameters - General optimization techniques like avoiding full table scans
Practical my sql performance optimization
- 1. Practical MySQL Practical MySQL Performance Performance OptimizationOptimization Marian HackMan MarinovMarian HackMan Marinov <[email protected]><[email protected]> Chief System ArchitectChief System Architect SiteGroundSiteGround
- 2. Who am I?Who am I?Who am I?Who am I? ❖ Chief System Architect of Siteground.com ❖ Sysadmin since 1996 ❖ Organizer of OpenFest, BG Perl Workshops, LUG-BG and similar :) ❖ Teaching Network Security and Linux System Administration at Sofia University
- 3. ❖ MySQL 5.7 ❖ Or its equivalent from MariaDB or Percona Requirements...Requirements...
- 4. ❖ Why am I making this presentation? ❖ Where is the information coming from? MySQL Optimization Documentation ❖ I have skipped the SQL examples as they made the presentation too big ☹ DISCLAMERDISCLAMER
- 5. ❖ Manual monitoring ❖ Using Monyog MonitoringMonitoring
- 6. How MySQL uses indexesHow MySQL uses indexes ❖ What type of indexes are there?
- 7. How MySQL uses indexesHow MySQL uses indexes ❖ What type of indexes are there? B-Tree Hash FULLTEXT Spatial
- 8. ❖ What type of indexes are there? ❖ Which is used for what ? hash used for = or <=> can not be used in ORDER BY only whole keys can be used for search How MySQL uses indexesHow MySQL uses indexes
- 9. ❖ What type of indexes are there? ❖ Which is used for what ? hash How MySQL uses indexesHow MySQL uses indexes
- 10. ❖ What type of indexes are there? ❖ Which is used for what ? b-tree can be used for =, >, >=, <, <=, or BETWEEN comparisons can be used with LIKE, only if it doesn't start with % can not be used in the form column1 LIKE column2 can limit the size of the index (prefix) How MySQL uses indexesHow MySQL uses indexes
- 11. ❖ When is index NOT used How MySQL uses indexesHow MySQL uses indexes
- 12. ❖ When is index NOT used when the index does not cover all the columns in the where clause How MySQL uses indexesHow MySQL uses indexes
- 13. ❖ When is index NOT used when the index does not cover all the columns in the where clause when the types don't match: integer used for string index(column) How MySQL uses indexesHow MySQL uses indexes
- 14. ❖ When is index NOT used when the index does not cover all the columns in the where clause when the types don't match: integer used for string index(column) when != is used How MySQL uses indexesHow MySQL uses indexes
- 15. ❖ When is index NOT used when the index does not cover all the columns in the where clause when the types don't match: integer used for string index(column) when != is used when you have a multicolumn index but the column you are referencing is not the first one and you don't have a separate index How MySQL uses indexesHow MySQL uses indexes
- 16. ❖ When is index NOT used when you have a Btree index but you use distinct on the column How MySQL uses indexesHow MySQL uses indexes
- 17. ❖ When is index NOT used when you have a Btree index but you use distinct on the column when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN How MySQL uses indexesHow MySQL uses indexes
- 18. ❖ When is index NOT used when you have a Btree index but you use distinct on the column when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN when LIKE is used with % in the begining e.g. LIKE '%name' How MySQL uses indexesHow MySQL uses indexes
- 19. ❖ When is index NOT used when you have a Btree index but you use distinct on the column when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN when LIKE is used with % in the begining e.g. LIKE '%name' order of columns in indexes MATTERS How MySQL uses indexesHow MySQL uses indexes
- 20. 1410864 rows 1 row in set (10.46 sec) 1 row in set (8.33 sec) 1 row in set (6.66 sec) Index hintsIndex hints
- 21. ❖ IGNORE INDEX Index hintsIndex hints
- 22. ❖ IGNORE INDEX USE INDEX❖ Index hintsIndex hints
- 23. ❖ IGNORE INDEX ❖ USE INDEX ❖ FORCE INDEX Index hintsIndex hints
- 24. ❖ "Waiting for query cache lock" Abusing query cacheAbusing query cache
- 25. ❖ "Waiting for query cache lock" ❖ SELECT *,NOW() FROM TABLE Abusing query cacheAbusing query cache
- 26. ❖ "Waiting for query cache lock" ❖ SELECT *,NOW() FROM TABLE ❖ SQL_NO_CACHE Abusing query cacheAbusing query cache
- 27. ❖ "Waiting for query cache lock" ❖ SELECT *,NOW() FROM TABLE ❖ SQL_NO_CACHE ❖ SET SESSION query_cache_type = OFF; Abusing query cacheAbusing query cache
- 28. ❖ "Waiting for query cache lock" ❖ SELECT *,NOW() FROM TABLE ❖ SQL_NO_CACHE ❖ SET SESSION query_cache_type = OFF; ❖ Аs of MySQL 5.6.8 query cache is now disabled by default Abusing query cacheAbusing query cache
- 29. ❖ Large result sets hammer the network... Abusing query cacheAbusing query cache
- 30. ❖ Large result sets hammer the network... ❖ For caching large results move the cache, closer to the application by caching the results in ProxySQL Abusing query cacheAbusing query cache
- 31. ❖ InooDB uses only a single index per query Optimizing InnoDB queriesOptimizing InnoDB queries
- 32. ❖ InooDB uses only a single index per query ❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL Optimizing InnoDB queriesOptimizing InnoDB queries
- 33. ❖ InooDB uses only a single index per query ❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL ❖ START TRANSACTION READ ONLY Optimizing InnoDB queriesOptimizing InnoDB queries
- 34. ❖ use ANALYZE TABLE or myisamchk --analyze Optimizing MyISAM queriesOptimizing MyISAM queries
- 35. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 Optimizing MyISAM queriesOptimizing MyISAM queries
- 36. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently Optimizing MyISAM queriesOptimizing MyISAM queries
- 37. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently ❖ MyISAM supports concurrent inserts Optimizing MyISAM queriesOptimizing MyISAM queries
- 38. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently ❖ MyISAM supports concurrent inserts concurrent_insert = 0 (disabled) concurrent_insert = 1 (auto) concurrent_insert = 1 (always, even with deleted rows) Optimizing MyISAM queriesOptimizing MyISAM queries
- 39. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently ❖ MyISAM supports concurrent inserts ❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables Optimizing MyISAM queriesOptimizing MyISAM queries
- 40. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently ❖ MyISAM supports concurrent inserts ❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables ❖ Don't split large(columns) tables Optimizing MyISAM queriesOptimizing MyISAM queries
- 41. ❖ use ANALYZE TABLE or myisamchk --analyze ❖ myisamchk --sort-index --sort-records=1 ❖ avoid complex SELECT queries on MyISAM tables that are updated frequently ❖ MyISAM supports concurrent inserts ❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables ❖ Don't split large(columns) tables ❖ use DELAY_KEY_WRITE=1 Optimizing MyISAM queriesOptimizing MyISAM queries
- 42. mysql> SELECT @@optimizer_switchG *************************** 1. row *************************** @@optimizer_switch: index_merge=on,index_merge_union=on, index_merge_sort_union=on, index_merge_intersection=on, engine_condition_pushdown=on, index_condition_pushdown=on, mrr=on,mrr_cost_based=on, block_nested_loop=on,batched_key_access=off, materialization=on,semijoin=on,loosescan=on, firstmatch=on,duplicateweedout=on, subquery_materialization_cost_based=on, use_index_extensions=on, condition_fanout_filter=on,derived_merge=on Controlling the OptimizerControlling the Optimizer MySQL 5.7 only
- 43. ❖ Merging multiple index scans on a SINGLE table index_merge=off (default on) index_merge_union=on index_merge_sort_union=on index_merge_intersection=on low selectivity indexes are very slow with intersect Controlling the OptimizerControlling the Optimizer
- 44. ❖ Selecting different merge algorithms per table (only for 5.7) Batched Key Access join /*+ BKA(t1) */ Block Nested-Loop join /*+ BNL(t1, t2) */ Controlling the OptimizerControlling the Optimizer
- 45. ❖ Optimizing subqueries Controlling the OptimizerControlling the Optimizer
- 46. ❖ Optimizing subqueries Materialization strategy Controlling the OptimizerControlling the Optimizer
- 47. ❖ Optimizing subqueries Materialization strategy Exists strategy Controlling the OptimizerControlling the Optimizer
- 48. ❖ Optimizing subqueries Materialization strategy Exists strategy (usually preferred when the sub queries can produce NULL results) Controlling the OptimizerControlling the Optimizer
- 49. ❖ Optimizing subqueries Materialization strategy Exists strategy (usually preferred when the sub queries can produce NULL results) subquery_materialization_cost_based Controlling the OptimizerControlling the Optimizer
- 50. ❖ use SQL_SMALL_RESUL ❖ If all columns in the index are numeric MySQL will read only the INDEX ❖ Avoid large amounts of values in IN statements values in an IN() list count as a predicates combined with OR use eq_range_index_dive_limit to control the index dives in range comparisons(IN) 5.7 range optimization can't be used with NOT IN() /*+ NO_RANGE_OPTIMIZATION(t4 PRIMARY) */ ❖ Index Condition Pushdown 5.7 General optimizationGeneral optimization
- 51. ❖ Avoid full table scans --max-seeks-for-key=1000 SSD or NVMe General optimizationGeneral optimization
- 52. # Turn tracing on (it's off by default): SET optimizer_trace="enabled=on"; SELECT ...; # your query here SELECT * FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE; # possibly more queries... # When done with tracing, disable it: SET optimizer_trace="enabled=off"; Using the tracerUsing the tracer
- 53. "filesort_summary": { "rows": 100, "examined_rows": 100, "number_of_tmp_files": 0, "sort_buffer_size": 25192, "sort_mode": "<sort_key, packed_additional_fields>" } Using the tracerUsing the tracer
- 54. ❖ Consider using INSERT with multiple VALUES lists ❖ Updates on tables with many indexes may become slow ❖ Updates with selects in them may block because of query cache lock contention ❖ Deletes also suffer from the above problems Changing dataChanging data
- 55. ❖ Splitting less frequently used columns from large(column) tables using foreign keys Optimizing architectureOptimizing architecture
- 56. ACID RainACID Rain Concurrency-Related Attacks onConcurrency-Related Attacks on Database-Backed Web ApplicationsDatabase-Backed Web Applications ACID RainACID Rain Concurrency-Related Attacks onConcurrency-Related Attacks on Database-Backed Web ApplicationsDatabase-Backed Web Applications Marian HackMan Marinov <[email protected]>
- 57. THANK YOUTHANK YOUTHANK YOUTHANK YOU Marian HackMan Marinov <[email protected]>