Presentation top tips for getting optimal sql execution

<Insert Picture Here>
Top Tips for getting optimal SQL execution
Mohamed Zait
Maria Colgan

3
Agenda
•  Cardinality
–  How to combat common causes for incorrect cardinality
•  Access path
–  What causes the wrong access path to be selected
•  Join type
–  Common causes for why the wrong join type was selected
•  Join order
–  Common causes for why the wrong join order was selected

4
Cardinality
What is it?
•  Estimate of number rows that will be returned by each operation
How is it computed?
•  Cardinality for a single column equality predicate = total num of rows
num of distinct values
–  For example: A table has 100 rows, a column has 10 distinct values
=> cardinality=10 rows
•  More complicated predicates have more complicated cardinality
calculation
Why should you care?
•  Influences everything!
•  Access method, Join type, Join Order, etc

Cardinality displayed in an Execution Plan
Cardinality the estimated # of
rows returned
Determine correct cardinality using a SELECT COUNT(*) from each table applying any
WHERE Clause predicates belonging to that table

Check Cardinality Estimates
SELECT /*+ gather_plan_statistics */ p.prod_name, SUM(s.quantity_sold)
FROM sales s, products p
WHERE s.prod_id =p.prod_id GROUP By p.prod_name ;
SELECT * FROM table (
DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS LAST'));
Compare the estimated number of rows returned for each operation in the
plan to actual rows returned

Check Cardinality Estimates using SQL Monitor
SQL Monitor is the easiest way to compare the estimated number of rows returned for each
operation in a parallel plan to actual rows returned

Causes for incorrect Cardinality estimates
Cause
No statistics or Stale Statistics
Data Skew
Multiple single column predicates on a table
Multiple columns used in a join
Function wrapped column
Complicated expression

No Statistics
•  If no statistics are present Optimizer uses dynamic sampling
•  A small number of blocks are read from the table at compile
time and statistics are estimated based on the sample
•  Not a good replacement for statistics gathered by
DBMS_STATS

No Statistics
Detected by dynamic sampling appearing in note section of plan
Means no stats gathered
strong indicator this won’t
be best possible plan
Solution gather statistics

Stale Statistics
Detected by checking staleness or changing cardinality estimates
•  Statistics are considered stale when 10% or more of the
rows have changed
•  Changes include, inserts, updates, deletes etc
•  Query dictionary to check if statistics are stale
SELECT table_name, stale_stats
FROM user_tab_ statistics;
No means Stats are good
Yes means Stats are stale
Null means no Stats
Solution gather statistics
Table Name Stale_stats
Sales NO
Customers YES
Product --
Note if DML has occurred very recently use dbms_stats.flush_database_monitoring_info

Stale statistics cause “Out of Range” issues’
Column stats for TIME_ID say Min = 01-JAN-98 Max = 31-DEC-03
SELECT /*+ gather_plan_statistics */ p.prod_name, SUM(s.quantity_sold)
FROM sales3 s, products p
WHERE s.prod_id =p.prod_id
AND s.time_id=’31-JAN-04;
GROUP BY p.prod_name ;
Optimizer assumes there are no rows for
time_id=‘31-Jan_04’ because it is out size of
[MIN.MAX] range

Solution
•  Use dbms.stats.copy_table_stats() to copy statistics from
previous partition before beginning the data load
P 1 – Jan 2010
P8 – Aug 2010
P9 – Sept 2010
:
• Copies statistics of the source partition to the
destination partition
• Adjusts the minimum and maximum values
of the partitioning column
• Both partition level and global level stats
• Copies the statistics of the dependent
objects: columns, local (partitioned) indexes
etc.
• Does not update global indexes

Data Skew
CLERK7782CLARK
AD_VP102De Haan
AD_VPKOCHHAR
CLERK2021WARD
CLERK7499ALLEN
CLERK99SMITH
Job_idEm_idLast_name
HR Employee table
SELECT * FROM HR.Employee
WHERE Job_id = AD_VP;
101
NAME ENUM JOB
Kochhar 101 AD_VP
De Haan 102 AD_VP
= =
Optimizer assumed even distribution
cardinality estimate is NUM_ROWS 107 6
NDV 19

Solution
•  Regather statistics and set method_opt parameter to
gather a histogram for all columns that have a data skew
EXEC DBMS_STATS.GATHER_TABLE_STATS(
’HR’,’EMPLOYEES’,method_opt => ‘FOR ALL COLUMNS SIZE SKEWONLY’);
SELECT column_name, num_distinct, histogram
FROM user_tab_col_statistics
WHERE table_name = ’ EMPLOYEES';

Data Skew Issue Corrected by Histogram
CLERK7782CLARK
CLERK7788SCOTT
VPKING
CLERK7521WARD
CLERK7499ALLEN
CLERK6973SMITH
Job_idEm_idLast_name
Employee table
SELECT * FROM Employee
WHERE Job_id = VP;
8739
NAME ENUM JOB
Kochhar 101 AD_VP
De Haan 102 AD_VP
Histogram tells Optimizer there’s a skew
cardinality estimate is now
#rows X # buckets value is in 107 X 2 2
total # of buckets 107otal #
of buckets in histogram
= =
NOTE: If the ‘#buckets value is in’ is less than 2 we use density (selectivity used for non-popular value)

Multiple Single Column Predicates
Query: SELECT *
FROM vehicles
WHERE model = ‘530xi’
AND make = ‘BMW’;
•  Optimizer assumes each where clause predicate will filter
rows
•  Cardinality estimated as #ROWS * 1 * 1
NDV c1 NDV c2
•  Real data often shows correlations between column values
•  Car model influences make
•  City influences state
•  Snow shoes only sold in winter time

SELECT ……FROM..
AND color = 'RED’;
SLIVERC320MERC
REDSLKMERC
RED911PORSCHE
SILVER530xiBMW
BLACK530xiBMW
RED530xiBMW
ColorModelMake
Vehicles table
Example: Multiple non-correlated Columns
•  One record selected
•  Cardinality #ROWS * 1 * 1
NDV c1 NDV c2
•  Cardinality = 12 * 1 * 1
4 3
RED530xiBMW
ColorModelMake
=
= 1

SELECT ……FROM..
Example: Multiple correlated Columns
SLIVERC320MERC
REDSLKMERC
RED911PORSCHE
SILVER530xiBMW
BLACK530xiBMW
RED530xiBMW
ColorModelMake
Vehicles table•  Three records selected
•  Cardinality #ROWS * 1 * 1
NDV c1 NDV c2
•  Cardinality = 12 * 1 * 1
4 3
SILVER530xiBMW
BLACK530xiBMW
RED530xiBMW
ColorModelMake
= 1

Solution
•  Create extended statistics on the Model & Make columns
Select dbms_stats.create_extended_stats(
Null , ’ Vehicles’ , ’(model,make)’) From dual;
Exec dbms_stats.gather_table_stats( Null, ’ Vehicles’);
Select column_name, num_distinct, histogram
From user_tab_col_statistics
Where table_name = ’ Vehicles';
New Column
with system
generated
name
NOTE: Column Group statistics only works with equality predicates & in-lists

SELECT ……FROM..
Soultion: Multiple correlated columns
SLIVERC320MERC
BLACKSLKMERC
RED911PORSCHE
SILVER530xiBMW
BLACK530xiBMW
RED530xiBMW
ColorModelMake
Vehicles table
•  Three records selected.
•  Cardinality is correctly
estimated from multi-column
SILVER530xiBMW
BLACK530xiBMW
RED530xiBMW
ColorModelMake

Multiple Join Columns
Query: SELECT count(*)
FROM Sales s, Sales2 s2
WHERE s.cust_id = s2.cust_id
AND s.prod_id = s2.prod_id;
• Assumes each join condition filter rows independently
• That’s not always the case
• In this example Cust_id and Prod_id are correlated so estimate will
be in correct
Solution
• Create extended statistics on the join columns on each table
• Remember column group statistics only work with equality
predicates and in lists

Function wrapped Column
Query: SELECT *
FROM Customers
WHERE UPPER(CUST_LAST_NAME) = ‘SMITH’;
•  Optimizer doesn’t know how function affects values in the
column so it guesses the cardinality to be 1% of rows
SELECT count(*) FROM customers;
COUNT(*)
55500
Cardinality estimate is 1% of the rows 555 rows

Solution
exec dbms_stats.gather_table_stats(null,'customers',
method_opt =>'for all columns size skewonly for columns
(UPPER(CUST_LAST_NAME))');
Select column_name, num_distinct, histogram
From user_tab_col_statistics
Where table_name = 'CUSTOMERS';
New Column
with system
generated
name

Automatic Column Group Creation
1.  Start column group usage capture
SQL> exec dbms_stats.seed_col_usage(null,null,300);
Capture column group usage from queries executing in the next 5 mins
2.  Run your workload
3.  Check we have column usage information for our table
SQL> select dbms_stats.report_col_usage(user, 'customers')
from dual;
COLUMN USAGE REPORT FOR SH.CUSTOMERS
1. COUNTRY_ID : EQ
2. CUST_CITY : EQ
3. CUST_STATE_PROVINCE : EQ
4. (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID) : FILTER
5. (CUST_STATE_PROVINCE, COUNTRY_ID) : GROUP_BY
EQ means column was
used in single table
equality predicate
GROUP_BY used
in group by
expression
FILTER means column used in
single table filter predicates

Automatic Column Group Creation
4.  Create extended stats for customers based on usage
SQL> select dbms_stats.create_extended_stats(user, 'customers')
from dual;
EXTENSIONS FOR SH.CUSTOMERS
1.  (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID):
SYS_STUMZ$C3AIHLPBROI#SKA58H_N created
2.  (CUST_STATE_PROVINCE, COUNTRY_ID) :
SYS_STU#S#WF25Z#QAHIHE#MOFFMM_created
Column group statistics will now be automatically
maintained every time you gather statistics on this table

Complex Expressions
Query: SELECT count(*)
FROM sales
WHERE cust_id < 2222
AND prod_id > 5;
Optimizer won’t use the column group statistics in
this case because predicates are non-equalities
Estimate
is 10X off

Solution
•  Dynamic sampling is the only solution
Alter session set optimizer_dynamic_sampling = 4;

Dynamic Sampling automatic for Parallel execution
•  New in Oracle Database 11g Release 2
•  Optimizer automatically decide if dynamic sampling
will be useful & what level for parallel SQL statements
•  Decision is based on size of the tables in the statement
•  The complexity of the predicates
•  If OPTIMIZER_DYNAMIC_SAMPLING is explicitly set then
that specified value will be honored
•  You can tell if dynamic sampling kicks in by looking in
the note section of the execution plan

Solutions to incorrect Cardinality estimates
Cause Solution
Stale or missing statistics DBMS_STATS
Data Skew Re-gather statistics to get histogram*
Multiple single column predicates
on a table
Create a column group using
DBMS_STATS.CREATE_EXTENDED_STATS
Function wrapped column Create statistics on the funct wrapped
column using
DBMS_STATS.CREATE_EXTENDED_STATS
Multiple columns used in a join Create a column group on join columns using
DBMS_STATS.CREATE_EXTENDED_STAT
Complicated expression
containing columns from multiple
tables
Use dynamic sampling level 4 or higher
*Histograms have an interesting side effects on statements with binds prior to 11g use with caution

Agenda
•  Cardinality
•  How to combat common causes for incorrect cardinality
•  Access path
•  What causes the wrong access path to be selected
•  Join type
•  Common causes for why the wrong join type was selected
•  Join order
•  Common causes for why the wrong join order was selected

Access Paths – Getting the data
Access Path Explanation
Full table scan Reads all rows from table & filters out those that do not meet the where clause
predicates. Used when no index, DOP set etc
Table access by Rowid Rowid specifies the datafile & data block containing the row and the location of
the row in that block. Used if rowid supplied by index or in where clause
Index unique scan Only one row will be returned. Used when stmt contains a UNIQUE or a
PRIMARY KEY constraint that guarantees that only a single row is accessed.
Index range scan Accesses adjacent index entries returns ROWID values Used with equality on
non-unique indexes or range predicate on unique index (<.>, between etc)
Index skip scan Skips the leading edge of the index & uses the rest Advantageous if there are
few distinct values in the leading column and many distinct values in the non-
leading column
Full index scan Processes all leaf blocks of an index, but only enough branch blocks to find 1st
leaf block. Used when all necessary columns are in index & order by clause
matches index struct or if sort merge join is done
Fast full index scan Scans all blocks in index used to replace a FTS when all necessary columns
are in the index. Using multi-block IO & can going parallel
Index joins Hash join of several indexes that together contain all the table columns that are
referenced in the query. Wont eliminate a sort operation
Bitmap indexes uses a bitmap for key values and a mapping function that converts each bit
position to a rowid. Can efficiently merge indexes that correspond to several
conditions in a WHERE clause

Common Access Path Issues
Issue
Uses a table scan instead of index
Optimizer picks wrong index

Access Path Example 1
•  Table: MY_SALES
•  2,098,400 rows
•  5510 MB in size
•  Index PROD_BIG_CUST_IND on columns
•  prod_id, product_details, cust_id
•  Query: SELECT prod_id, cust_id
FROM my_sales
WHERE prod_id = 141
AND cust_id < 8938;
•  What access path do you think Optimizer should pick?

•  Optimizer picks an index range scan of prod_cust_big_ind
•  Elapse time is 4 seconds
•  Alternative is a full table scan
•  Elapse times is 22 seconds

•  A nightly batch job set the degree of parallelism on my_sales
to 100
•  Last night the batch job failed half way through and did not complete
•  What access path do you think the Optimizer picks now?
• Now the optimizer costs the table access 100 X cheaper
• But elapse times is 32 seconds

Solution
•  Don’t set the DOP on the object in a mixed workload
environment
•  Prevent batch jobs interfering with online users by using
•  An alter session command
•  Alter session force parallel query parallel 100;
•  A parallel hint
SELECT /*+ parallel(s,100) */ count(*)
FROM My_Sales s
WHERE …..;
•  Using AUTO DOP
•  PARALLEL_DEGREE_POLICY = AUTO or LIMITED

•  Table: MY_SALES
•  2,098,400 rows
•  5510 MB in size
•  Index PROD_CUST_BIG_IND on columns
•  prod_id, product_details, cust_id
•  Index PROD_CUST on columns
•  prod_id, cust_id
•  Query: SELECT product_details
FROM my_sales
WHERE prod_id = 141
AND cust_id < 8938;
•  What access path do you think the Optimizer picks?

•  Common misconception Optimizer will pick the index with
all the necessary columns in it
•  Optimizer picks the index based on the cost

Why didn’t the Optimizer select the index
prod_cust_big_ind ?
SELECT index_name, leaf_blocks, blevel
FROM user_indexes
WHERE table_name = ‘MY_SALES’;
•  Cost for prod_cust_ind: 2 + ( 2579 X 5468) 9
2098400
•  Cost for prod_big cust_ind: 9 +(0.013 X 699467) 9724
•  But plan says cost is 9728 why the difference?
•  CPU cost to apply the filter

Causes of Access Path issues
Issue Cause
Uses a table scan instead of index DOP on table but not index, value of MBRC,
or incorrect cardinality estimates
Picks wrong index Stale index statistics
Incorrect cardinality estimate
Bad Cardinality estimates See section one

Join Types
Join Type Explanation
Nested Loops joins For every row in the outer table, Oracle accesses all the rows in the inner table
Useful when joining small subsets of data and there is an efficient way to
access the second table (index look up)
Hash Joins The smaller of two tables is scan and resulting rows are used to build a hash
table on the join key in memory. The larger table is then scan, join column of
the resulting rows are hashed and the values used to probe the hash table to
find the matching rows. Useful for larger tables & if equality predicates
Sort Merge joins Consists of two steps:
1.  Both the inputs are sorted on the join key.
2.  The sorted lists are merged together.
Useful when the join condition between two tables is an inequality condition or
one of the tables is already sorted e.g. access path is an index
Cartesian Joins Joins every row from one data source with every row from the other data
source, creating the Cartesian Product of the two sets. Only good if tables
are very small. Only choice if there is no join condition specified in query

What causes the wrong Join Type to be selected
Issue
Optimizer selects a Nested Loop instead of a Hash Join
Optimizer selects a Hash join instead of a Nested Loop
Cartesian Joins is selected when the number of rows is large

Join Type Example 1
•  Query: SELECT count(*)
FROM sales s, customers c
WHERE s.cust_id = c.cust_id
AND substr(c.cust_state_province,1,2) = ‘CA’
AND c.country_id = c.country_id+0;
•  Execution Plan

Join Type Example 1
•  What would the cost and buffer gets be if we hinted for HJ
•  Query: SELECT /*+ use_hash(c s) */ count(*)
FROM sales s, customers c
WHERE s.cust_id = c.cust_id
AND substr(c.cust_state_province,1,2) = ‘CA’
AND c.country_id = c.country_id+0;
•  Execution Plan

Join Type Example 1
•  Why did the optimizer get it so wrong?
•  Underestimated cardinality on the left hand side of the join
•  Customers is left hand side of the join & predicates are
substr(c.cust_state_province,1,2) = ‘CA’
c.country_id = c.country_id+0;
Function wrapped
column
Additional predicated
doesn’t eliminate
rows
•  Solution:
•  Correct the cardinality estimate on the left hand side of join
•  Use extended statistics

What causes the wrong Join Type to be selected
Issue Cause
Nested loop
selected instead
of hash join
Cardinality estimate on the left side is under
estimated triggers Nested loop to be selected
Hash join selected
instead of nested
loop
In case of a hash join the Optimizer doesn’t taken
into consideration the benefit of caching. Rows on
the left come in a clustered fashion or (ordered)
so the probe in the right side is less expensive
Cartesian Joins Cardinality under estimated

Join Orders
The order in which the tables are join in a multi table stmt
•  Ideally start with the table that will eliminate the most rows
•  Strongly affected by the access paths available
Some basic rules
•  Joins guaranteed to produce at most one row always go first
•  Joins between two row sources that have at most one row each
•  When outer joins are used the table with the outer join
operator must come after the other table in the predicate
•  If view merging is not possible all tables in the view will be
joined before joining to the tables outside the view

What causes the wrong Join Order
Causes
Incorrect single table cardinality estimates
Incorrect join cardinality estimates

The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.

Presentation top tips for getting optimal sql execution

Recommended

More Related Content

What's hot (20)

Similar to Presentation top tips for getting optimal sql execution (20)

More from xKinAnx (20)

Recently uploaded (20)

Presentation top tips for getting optimal sql execution