Visualizing ORACLE performance data with R @ #C16LV

Visualizing ORACLE performance
with R
Maxym Kharchenko
Gluent.com

Whoami
■ Started as a database kernel developer
■ Then: ORACLE DBA for 15+ years
■ Now: Developer at Gluent (past: amazon.com)
■ OCM, ORACLE Ace Associate, AWS Certified Developer
■ Blog: http://intermediatesql.com
■ Twitter: @maxymkh

The cool things that we do at Gluent
Gluent
Oracle
Teradata
NoSQL
Big Data
Sources
MSSQL
App
X
App
Y
App
Z
We glue these
worlds together!

The cool things that we do at Gluent

Agenda
■ Why visualize with R
■ How to visualize with R
■ Pretty pictures !
■ Interesting use cases (more pretty pictures!)
■ Labs! Labs! Labs!

Why visualize ?
[1] 10.06 10.07 9.99 9.95 10.56 9.82 10.06 9.97 9.97 9.91
[11] 9.99 10.68 10.04 10.05 9.92 10.08 9.91 9.97 10.11 10.03
[21] 10.08 10.22 8.84 10.42 8.68 10.14 9.46 9.69 11.56 9.55
[31] 10.32 8.77 10.20 10.16 10.03 10.05 10.47 9.83 10.18 10.00
[41] 10.11 9.76 9.89 10.09 10.09 10.15 9.86 10.06 10.56 9.87
[51] 9.95 10.19 10.01 10.04 10.93 11.03 11.07 11.08 11.21 10.77
[61] 11.01 10.87 11.06 11.16 10.94 9.82 10.09 10.16 10.05 9.87
[71] 10.01 9.92 9.90 10.23 10.14 10.09 10.08 9.92 10.05 10.60
[81] 10.06 10.10 9.97 10.25 10.10 10.19 10.07 9.97 10.05 10.08
[91] 9.90 10.41 10.19 9.96 9.90 10.07 9.95 10.22 9.94 9.93

DBA 2.0: EM – Pretty … but not flexible

DBA 1.0: sqlplus – Flexible
SQL*Plus: Release 11.2.0.2.0 Production on Fri Feb 14
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0
With the Partitioning and Real Application Testing options
SQL> @event_histograms db%file%sequential

DBA 1.0: sqlplus – Flexible … but not pretty
EVENT Ms PCT GRAPH
----------------------- ---- ------- --------------------
db file sequential read 1 18.43 ########
2 4.09 #
4 23.52 ##########
8 43.04 ####################
16 10.05 ####
32 .72
64 .06
128 .09
256 .01

DBA 1.0: sqlplus – Flexible … but not pretty
EVENT Ms PCT GRAPH
----------------------- ---- ------- --------------------
db file sequential read 1 18.43 ********
2 4.09 *
4 23.52 **********
8 43.04 ********************
16 10.05 ****
32 .72
64 .06
128 .09
256 .01

Ok, sqlplus CAN be pretty
Tanel Poder’s fish.sql

Need a tool: both pretty AND flexible

http://www.r-project.org/
What R looks like

If you know how to program in
Perl/Python/Ruby etc
You know how to program in
R

#***********************************************************
# Prepare exploration: Define categories, set plot type etc
#***********************************************************
prepare_exploration <- function(
fill=NULL, y="N", x="TS", pct=FALSE,
to_ts=c("TS"), top_n=8, drop_others=FALSE, data=d
) {
if(is.null(fill)) {
data$CAT <- 1
} else {
data <- add_cat_top_n(fill, y, x, top_n,
drop_others, data)
if (pct) {
data <- add_pct(y, x, data)
}
}
data <- to_ts(to_ts, data)
return(data)
}

R: Appearances are important
A = B + C
A <- B + C

R: Everything is a VECTOR
A + B
Result:
[1] 4 6 8 10 12 14 16 18 20 22

R visualization workflow
Get data
into R
Transform Visualize

Get data into R
CSV, TXT:
d <- read.csv('http://…/file.csv')
ROracle, RJDBC:
odrv <- dbDriver("Oracle")
conn <- dbConnect(odrv, user, passwd, tns)
d <- dbGetQuery(conn, sql, binds)

Data in R - a “data frame”

Transform
If you know SQL
You know how to
transform data in R

Transform
d1 <- sqldf("
SELECT event, count(1) as n
FROM d
GROUP BY event
ORDER BY n DESC LIMIT 10
")

Visualize
ggplot(d, aes) + geom
+ “other stuff”

AES: Mapping data
Time Execs
10:15 100
10:20 150
10:25 180
10:30 120
10:35 220
aes(x=Time, y=Execs)

AES: Mapping data
Time Execs Type
10:15 90 READ
10:15 10 WRITE
10:20 120 READ
10:20 30 WRITE
10:25 100 READ
10:25 80 WRITE
10:30 20 READ
10:30 100 WRITE
10:35 120 READ
10:35 100 WRITE
aes(x=Time, y=Execs,
color=Type)

Geoms
Time Execs Type
10:15 90 READ
10:15 10 WRITE
10:20 120 READ
10:20 30 WRITE
10:25 100 READ
10:25 80 WRITE
10:30 20 READ
10:30 100 WRITE
10:35 120 READ
10:35 100 WRITE
fill=Type)
+ geom_bar()

Geoms
Time Execs Type
10:15 90 READ
10:15 10 WRITE
10:20 120 READ
10:20 30 WRITE
10:25 100 READ
10:25 80 WRITE
10:30 20 READ
10:30 100 WRITE
10:35 120 READ
10:35 100 WRITE
color=Type)
+ geom_point()

Geoms
Time Execs Type
10:15 90 READ
10:15 10 WRITE
10:20 120 READ
10:20 30 WRITE
10:25 100 READ
10:25 80 WRITE
10:30 20 READ
10:30 100 WRITE
10:35 120 READ
10:35 100 WRITE
color=Type, size=Execs)
+ geom_point()

Putting it all together
R> connect('db1')
R> exec('get_ash.sql',start_time='2016-02-01')
R> n_exec('db1, db2, db3', 'get_ash.sql',
start_time='2016-02-01')
R> d1 <- sqldf('select … from … where …')
R> explore_bar(fill='EVENT')
R> explore_area(fill='BLOCKING_SESSION')
R> explore_point(x='PARSES', y='EXECS',
color='MODULE', top=8)

Time series plots: v$active_session_history

Time series plots: dba_hist_seg_stat

Time series plots – ARC heat map

Not just for database metrics
■ cat listener.log | grep CONNECT_DATA |
perl -pale 's/^(S+ S+).*(PROGRAM=(.*?)).*$/$1 $2/'
>/tmp/s.txt

Summarized data: (sampled) v$sql
v$sql.elapsed_time

Summarized data: dba_hist_sqlstat

Scatter plots: dba_hist_sqlstat

Block flowers:
dba_hist_active_sess_history

ORA-00020: Max # of processes exceeded
When was the database really “down” ?

Logon trigger: Rejected connections
What was the exact effect on the system ?
ASH.in_connection_mgmt=‘Y’

Rolling partitions:
Can we archive data older than 30 days ?
dba_hist_seg_stat.
db_block_changes_delta
dba_hist_seg_stat.
logical_reads_delta

Query latency:
Can we trust the “average” elapsed time ?

Takeaways
■ R is a sqlplus with graphics
■ If you know how to script, R is easy to master
■ Did I mention that R is free ?

Visualizing ORACLE performance data with R @ #C16LV

More Related Content

What's hot (20)

Similar to Visualizing ORACLE performance data with R @ #C16LV (20)

Recently uploaded (20)

Visualizing ORACLE performance data with R @ #C16LV