An Interactive Introduction To R (Programming Language For Statistics)

An Interactive Introduction to RNovember 2009Michael E. Driscoll, Ph.D.med@dataspora.comhttp://www.dataspora.comDaniel Murphy FCAS, MAAAdmurphy@trinostics.com

R is a tool for…Data Manipulationconnecting to data sourcesslicing & dicing dataModeling & Computationstatistical modelingnumerical simulationData Visualizationvisualizing fit of modelscomposing statistical graphics

Let’s take a tour of some claim datain R

Let’s take a tour of some claim datain R## load in some Insurance Claim datalibrary(MASS)data(Insurance)Insurance <- edit(Insurance)head(Insurance)dim(Insurance)## plot it nicely using the ggplot2 packagelibrary(ggplot2)qplot(Group, Claims/Holders, data=Insurance,geom="bar", stat='identity', position="dodge", facets=District ~ ., fill=Age,ylab="Claim Propensity",xlab="Car Group")## hypothesize a relationship between Age ~ Claim Propensity## visualize this hypothesis with a boxplotx11()library(ggplot2)qplot(Age, Claims/Holders, data=Insurance,geom="boxplot", fill=Age)## quantify the hypothesis with linear modelm <- lm(Claims/Holders ~ Age + 0, data=Insurance)summary(m)

R is “an overgrown calculator”sum(rgamma(rpois(1,lambda=2),shape=49,scale=.2)))

R is “an overgrown calculator”simple math> 2+24storing results in variables> x <- 2+2 ## ‘<-’ is R syntax for ‘=’ or assignment> x^2 16vectorized math> weight <- c(110, 180, 240) ## three weights> height <- c(5.5, 6.1, 6.2) ## three heights> bmi <- (weight*4.88)/height^2 ## divides element-wise17.7 23.6 30.4

R is “an overgrown calculator”basic statisticsmean(weight) sd(weight) sqrt(var(weight))176.6 65.0 65.0 # same as sdset functionsunion intersect setdiffadvanced statistics > pbinom(40, 100, 0.5) ## P that a coin tossed 100 times 0.028## that comes up 40 heads is ‘fair’ > pshare <- pbirthday(23, 365, coincident=2) 0.530 ## probability that among 23 people, two share a birthday

Try It! #1 Overgrown Calculatorbasic calculations> 2 + 2 [Hit ENTER]> log(100) [Hit ENTER]calculate the value of $100 after 10 years at 5%> 100 * exp(0.05*10) [Hit ENTER]construct a vector & do a vectorized calculation> year <- (1,2,5,10,25) [Hit ENTER] this returns an error. why?> year <- c(1,2,5,10,25) [Hit ENTER]> 100 * exp(0.05*year) [Hit ENTER]

R is a numerical simulator built-in functions for classical probability distributions

let’s simulate 10,000 trials of 100 coin flips. what’s the distribution of heads?> heads <- rbinom(10^5,100,0.50)> hist(heads)

Functions for Probability Distributions> pnorm(0) 0.05 > qnorm(0.9) 1.28> rnorm(100) vector of length 100

Functions for Probability DistributionsHow to find the functions for lognormal distribution? 1) Use the double question mark ‘??’ to search> ??lognormal2) Then identify the package > ?Lognormal3) Discover the dist functions dlnorm, plnorm, qlnorm, rlnorm

Try It! #2 Numerical Simulationsimulate 1m policy holders from which we expect 4 claims> numclaims <- rpois(n, lambda)(hint: use ?rpoisto understand the parameters)verify the mean & variance are reasonable> mean(numclaims)> var(numclaims)visualize the distribution of claim counts> hist(numclaims)

Getting Data In - from Files> Insurance <- read.csv(“Insurance.csv”,header=TRUE) from Databases> con <- dbConnect(driver,user,password,host,dbname)> Insurance <- dbSendQuery(con, “SELECT * FROM claims”) from the Web> con <- url('http://labs.dataspora.com/test.txt')> Insurance <- read.csv(con, header=TRUE) from R objects> load(‘Insurance.RData’)

Getting Data Outto Fileswrite.csv(Insurance,file=“Insurance.csv”)to Databasescon <- dbConnect(dbdriver,user,password,host,dbname)dbWriteTable(con, “Insurance”, Insurance) to R Objectssave(Insurance, file=“Insurance.RData”)

Navigating within the R environmentlisting all variables> ls()examining a variable ‘x’> str(x)> head(x)> tail(x)> class(x)removing variables> rm(x)> rm(list=ls()) # remove everything

Try It! #3 Data Processing load data & view itlibrary(MASS)head(Insurance) ## the first 7 rowsdim(Insurance) ## number of rows & columnswrite it outwrite.csv(Insurance,file=“Insurance.csv”, row.names=FALSE)getwd() ## where am I?view it in Excel, make a change, save it remove the first districtload it back in to R & plot itInsurance <- read.csv(file=“Insurance.csv”)plot(Claims/Holders ~ Age, data=Insurance)

A Swiss-Army Knife for DataIndexingThree ways to index into a data framearray of integer indicesarray of character namesarray of logical BooleansExamples:df[1:3,]df[c(“New York”, “Chicago”),]df[c(TRUE,FALSE,TRUE,TRUE),]df[city == “New York”,]

A Swiss-Army Knife for Datasubset – extract subsets meeting some criteriasubset(Insurance, District==1)subset(Insurance, Claims < 20)transform – add or alter a column of a data frametransform(Insurance, Propensity=Claims/Holders)cut – cut a continuous value into groups cut(Insurance$Claims, breaks=c(-1,100,Inf), labels=c('lo','hi')) Put it all together: create a new, transformed data frametransform(subset(Insurance, District==1), ClaimLevel=cut(Claims, breaks=c(-1,100,Inf), labels=c(‘lo’,’hi’)))

A Statistical ModelerR’s has a powerful modeling syntaxModels are specified with formulae, like y ~ x growth ~ sun + watermodel relationships between continuous and categorical variables.Models are also guide the visualization of relationships in a graphical form

A Statistical ModelerLinear modelm <- lm(Claims/Holders ~ Age, data=Insurance)Examine it summary(m)Plot it plot(m)

A Statistical ModelerLogistic model m <- glm(Age ~ Claims/Holders, data=Insurance, family=binomial(“logit”)))Examine it summary(m)Plot it plot(m)

Try It! #4 Statistical Modelingfit a linear modelm <- lm(Claims/Holders ~ Age + 0, data=Insurance) examine it summary(m)plot itplot(m)

Visualization: Multivariate Barplotlibrary(ggplot2)qplot(Group, Claims/Holders, data=Insurance, geom="bar", stat='identity', position="dodge", facets=District ~ ., fill=Age)

Visualization: Boxplotslibrary(ggplot2)qplot(Age, Claims/Holders, data=Insurance, geom="boxplot“)library(lattice)bwplot(Claims/Holders ~ Age, data=Insurance)

Visualization: Histogramslibrary(ggplot2)qplot(Claims/Holders, data=Insurance, facets=Age ~ ., geom="density")library(lattice)densityplot(~ Claims/Holders | Age, data=Insurance, layout=c(4,1)

Try It! #5 Data Visualizationsimple line chart> x <- 1:10> y <- x^2> plot(y ~ x)box plot> library(lattice)> boxplot(Claims/Holders ~ Age, data=Insurance)visualize a linear fit> abline()

Getting Help with RHelp within R itself for a function> help(func)> ?funcFor a topic> help.search(topic)> ??topicsearch.r-project.orgGoogle Code Search www.google.com/codesearchStack Overflow http://stackoverflow.com/tags/RR-help list http://www.r-project.org/posting-guide.html

Six Indispensable Books on RLearning RData ManipulationVisualizationStatistical Modeling

Extending R with PackagesOver one thousand user-contributed packages are available on CRAN – the Comprehensive R Archive Networkhttp://cran.r-project.orgInstall a package from the command-line> install.packages(‘actuar’)Install a package from the GUI menu“Packages”--> “Install packages(s)”

Final Try It!Simulate a TweedieSimulate the number of claims from a Poisson distribution with λ=2 (NB: mean poisson = λ, variance poisson = λ)

For as many claims as were randomly simulated, simulate a severity from a gamma distribution with shape α=49 and scale θ=0.2 (NB: mean gamma = αθ, variance gamma = αθ2)

Is the total simulated claim amount close to expected?

Calculate usual parameterization (μ,p,φ)of this Tweedie distribution

Does your histogram look like Glenn Meyers’?http://www.casact.org/newsletter/index.cfm?fa=viewart&id=5756Final Try It!Simulate a Tweedie- ANSWERSSimulate the number of claims from a Poisson distribution with λ=2 (NB: mean poisson = λ, variance poisson = λ)rpois(1,lambda=2)For as many claims as were randomly simulated, simulate a severity from a gamma distribution with shape α=49 and scale θ=0.2rgamma(rpois(1,lambda=2),shape=49,scale=.2)Is the total simulated claim amount close to expected? sum(rgamma(rpois(1,lambda=2),shape=49,scale=.2)) Repeat the above 10000 times replicate(10000, sum(rgamma(rpois(1,lambda=2),shape=49,scale=.2))) Visualize the distributionhist(replicate(10000, sum(rgamma(rpois(1,lambda=2),shape=49,scale=.2))), breaks=200, freq=FALSE)

P&C Actuarial ModelsDesign • Construction Collaboration • Education Valuable • Transparent Daniel Murphy, FCAS, MAAAdmurphy@trinostics.com925.381.9869From Data to DecisionBig Data • Analytics • Visualizationwww.dataspora.comMichael E. Driscoll, Ph.D.med@dataspora.com415.860.434737Contact Us

AppendicesR as a Programming LanguageAdvanced VisualizationEmbedding R in a Server Environment

R as a Programming Languagefibonacci <- function(n) { fib <- numeric(n) fib [1:2] <- 1 for (i in 3:n) { fib[i] <- fib[i-1] + fib[i-2] } return(fib[n])}Image from cover of Abelson & Sussman’stextThe Structure and Interpretation of Computer Languages

Assignmentx <- c(1,2,6)x a variable x<- R’s assignment operator, equivalent to ‘=‘ c( a function c which combines its arguments into a vectory <- c(‘apples’,’oranges’)z <- c(TRUE,FALSE) c(TRUE,FALSE) -> zThese are also valid assignment statements.

Function CallsThere are ~ 1100 built-in commands in the R “base” package, which can be executed on the command-line. The basic structure of a call is thus:output <- function(arg1, arg2, …)Arithmetic Operations + - * / ^R functions are typically vectorized x <- x/3 works whether x is a one or many-valued vector

Data Structures in Rnumericx <- c(0,2:4)vectorsCharactery <- c(“alpha”, “b”, “c3”, “4”)logicalz <- c(1, 0, TRUE, FALSE)> class(x)[1] "numeric"> x2 <- as.logical(x)> class(x2)[1] “logical”

Data Structures in Rlistslst <- list(x,y,z)objectsmatricesM <- matrix(rep(x,3),ncol=3)data frames*df <- data.frame(x,y,z)> class(df)[1] “data.frame"

Summary of Data Structures?matricesvectorsdata frames*lists

Advanced Visualizationlattice, ggplot2, and colorspace

qplot(log(carat), log(price), data = diamonds, alpha=I(1/20)) + facet_grid(. ~ color)Achieving small multiples with “facets”

lattice = trellis(source: http://lmdvr.r-forge.r-project.org )

list of latticefunctionsdensityplot(~ speed | type, data=pitch)

visualizing six dimensionsof MLB pitches with lattice

xyplot(x ~ y | type, data=pitch,fill.color = pitch$color,panel = function(x,y, fill.color, …, subscripts) { fill <- fill.color[subscripts]panel.xyplot(x, y, fill= fill, …) })

Beautiful Colors with Colorspacelibrary(“Colorspace”)red <- LAB(50,64,64)blue <- LAB(50,-48,-48)mixcolor(10, red, blue)

efficient plotting with hexbinplothexbinplot(log(price)~log(carat),data=diamonds,xbins=40)

An Interactive Introduction To R (Programming Language For Statistics)

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to An Interactive Introduction To R (Programming Language For Statistics) (20)

Recently uploaded (20)

An Interactive Introduction To R (Programming Language For Statistics)

Editor's Notes