Processes in Query Optimization in (ABMS) Advanced Database Management Systems
- 1. Topic: Processes in Query Optimization Advanced Database Management Systems
- 2. Topics to be covered • Why Query Optimizer ? • Cost-Based Optimization (CBO) • Not optimized code • Execution Plans • Query Block • Optimizer Components • QueryTransformer • Estimator • Plan Generator • AutomaticTuning optimizer
- 3. The query optimizer (called simply the optimizer) is built-in database software that determines the most efficient method for a SQL statement to access requested data. Introduction SQL Server can efficiently filter a data set using indexes via the WHERE clause or any combination of filters that are separated by an AND operator. By being exclusive, these operations take data and slice it into progressively smaller pieces, until only our result set remains. • Purpose of the Query Optimizer • Cost-Based Optimization • Execution Plans
- 4. WHY Query Optimizer ? • The optimizer choose the best plan with the lowest cost among all considered candidate plans. • The optimizer uses available statistics to calculate cost. • For a specific query in a given environment, the cost computation accounts for factors of query execution such as I/O, CPU, and communication.
- 5. Cost-Based Optimization (CBO) • The optimizer is free to merge, reorganize, and process in any order. • The optimizer determines the optimal plan for a SQL statement by examining multiple access methods, such as full table scan or index scans, different join methods such as nested loops and hash joins, different join orders, and possible transformations.
- 7. Cost-Based Optimization SELECT prod_category, AVG(amount_sold) FROM sales s, products p WHERE p.prod_id = s.prod_id GROUP BY prod_category
- 8. Example SELECT DISTINCT PRODUCT.ProductID, PRODUCT.Name FROM Production.Product PRODUCT INNER JOIN Sales.SalesOrderDetail DETAIL ON PRODUCT.ProductID = DETAIL.ProductID OR PRODUCT.rowguid = DETAIL.rowguid; Our expectation would be a table scan on Product and a table scan on SalesOrderDetail. Expensive. Product : 504 rows SalesOrderDetails : 121317 A singleOr can make performance worse
- 9. Example – Optimized Code SELECT PRODUCT.ProductID, PRODUCT.Name FROM Production.Product PRODUCT INNER JOIN Sales.SalesOrderDetail DETAIL ON PRODUCT.ProductID = DETAIL.ProductID UNION SELECT PRODUCT.ProductID, PRODUCT.Name FROM Production.Product PRODUCT INNER JOIN Sales.SalesOrderDetail DETAIL ON PRODUCT.rowguid = DETAIL.rowguid SELECT DISTINCT PRODUCT.ProductID, PRODUCT.Name FROM Production.Product PRODUCT INNER JOIN Sales.SalesOrderDetail DETAIL ON PRODUCT.ProductID = DETAIL.ProductID OR PRODUCT.rowguid = DETAIL.rowguid;
- 10. Execution Plans In the following graphic, the optimizer generates two possible execution plans for an input SQL statement, uses statistics to estimate their costs, compares their costs, and then chooses the plan with the lowest cost.
- 11. Execution Plans – Query Block The input to the optimizer is a parsed representation of a SQL statement. The following SQL statement consists of two query blocks. The subquery in parentheses is the inner query block. The outer query block, which is the rest of the SQL statement, retrieves names of employees in the departments whose IDs were supplied by the subquery. The query form determines how query blocks are interrelated. SELECT first_name, last_name FROM hr.employeesWHERE department_id IN (SELECT department_id FROM hr.departmentsWHERE location_id = 1800);
- 12. Query Subplans SELECT first_name, last_name FROM hr.employeesWHERE department_id IN (SELECT department_id FROM hr.departmentsWHERE location_id = 1800); • The database optimizes query blocks separately from the bottom up approach. • The database optimizes the innermost query block first and generates a subplan for it • The number of possible plans for a query block is proportional to the number of objects in the FROM clause. • This number rises exponentially with the number of objects. For example, the possible plans for a join of five tables are significantly higher than the possible plans for a join of two tables.
- 13. Optimizer Components Phase Operation Description 1 Query Transformer The optimizer determines whether it is helpful to change the form of the query so that the optimizer can generate a better execution plan. 2 Estimator The optimizer estimates the cost of each plan based on statistics in the data dictionary. 3 Plan Generator The optimizer compares the costs of plans and chooses the lowest-cost plan, known as the execution plan, to pass to the row source generator.
- 15. Step 1. QueryTransformer For some statements, the query transformer determines whether it is advantageous to rewrite the original SQL statement into a semantically equivalent SQL statement with a lower cost. When a viable alternative exists, the database calculates the cost of the alternatives separately and chooses the lowest-cost alternative.
- 17. Step 2. Estimator The estimator is the component of the optimizer that determines the overall cost of a given execution plan. The estimator uses three different measures to determine cost: 1. Selectivity The percentage of rows in the row set that the query selects, with 0 meaning no rows and 1 meaning all rows. e.g. WHERE last_name LIKE 'A%' Assume there are 150 distinct employee last names. For an equality predicate last_name = 'Smith', selectivity is the reciprocal of the number n of distinct values of last_name, which is suppose .006 because the query selects rows that contain 1 out of 150 distinct values.
- 18. Step 2. Estimator 2. Cardinality The cardinality is the number of rows returned by each operation in an execution plan. This input, which is crucial to obtaining an optimal plan, is common to all cost functions. For example, user hr queries the employees table as follows: SELECT first_name, last_name FROM employees WHERE salary='10200'; The employees table contains 107 rows. The current database statistics indicate that the number of distinct values in the salary column is 58. Therefore, the optimizer estimates the cardinality of the result set as 2, using the formula 107/58=1.84.
- 19. Step 2. Estimator 3. Cost This measure represents units of work or resource used. The query optimizer uses disk I/O, CPU usage, and memory usage as units of work. The cost is an internal numeric measure that represents the estimated resource usage for a plan. To estimate cost, the optimizer considers factors such as the following: • System resources, which includes estimated I/O, CPU, and memory • Estimated number of rows returned (cardinality) • Size of the initial data sets • Distribution of the data • Access structures
- 20. Step 3. Plan Generator
- 21. Step 3. Plan Generator The plan generator explores various plans for a query block by trying out different access paths, join methods, and join orders. Many plans are possible because of the various combinations that the database can use to produce the same result. The optimizer picks the plan with the lowest cost. The trace file shows the optimizer first trys table as the outer table in the join. The optimizer calculates the cost for three different join methods: nested loops join (NL), sort merge (SM), and hash join (HA). The optimizer picks the hash join as the most efficient method:
- 22. AutomaticTuning Optimizer The database provides the following types of optimization: • Normal optimization The optimizer compiles the SQL and generates an execution plan. The normal mode generates a reasonable plan for most SQL statements. Under normal mode, the optimizer operates with strict time constraints, usually a fraction of a second, during which it must find an optimal plan. • SQL Tuning Advisor optimization When SQL Tuning Advisor invokes the optimizer, the optimizer is known as Automatic Tuning Optimizer. In this case, the optimizer performs additional analysis to further improve the plan produced in normal mode. The optimizer output is not an execution plan, but a series of actions, along with their rationale and expected benefit for producing a significantly better plan.
- 23. Some words/phrases/sentences are taken from internet
- 24. ThankYou