Next Generation Programming in R
12 likes1,775 views
The document discusses recent developments in the R programming environment for data analysis, including packages like magrittr, readr, tidyr, and dplyr that enable data wrangling workflows. It provides an overview of the key functions in these packages that allow users to load, reshape, manipulate, model, visualize, and report on data in a pipeline using the %>% operator.
![Data Wrangling
with dplyr and tidyr
Cheat Sheet
RStudio® is a trademark of RStudio, Inc. • CC BY RStudio • info@rstudio.com • 844-448-1212 • rstudio.com
Syntax - Helpful conventions for wrangling
dplyr::tbl_df(iris)
Converts data to tbl class. tbl’s are easier to examine than
data frames. R displays only the data that fits onscreen:
dplyr::glimpse(iris)
Information dense summary of tbl data.
utils::View(iris)
View data set in spreadsheet-like display (note capital V).
Source: local data frame [150 x 5]
Sepal.Length Sepal.Width Petal.Length
1 5.1 3.5 1.4
2 4.9 3.0 1.4
3 4.7 3.2 1.3
4 4.6 3.1 1.5
5 5.0 3.6 1.4
.. ... ... ...
Variables not shown: Petal.Width (dbl),
Species (fctr)
dplyr::%>%
Passes object on left hand side as first argument (or .
argument) of function on righthand side.
"Piping" with %>% makes code more readable, e.g.
iris %>%
group_by(Species) %>%
summarise(avg = mean(Sepal.Width)) %>%
arrange(avg)
x %>% f(y) is the same as f(x, y)
y %>% f(x, ., z) is the same as f(x, y, z )
Reshaping Data - Change the layout of a data set
Subset Observations (Rows) Subset Variables (Columns)
F M A
Each variable is saved
in its own column
F M A
Each observation is
saved in its own row
In a tidy
data set: &
Tidy Data - A foundation for wrangling in R
Tidy data complements R’s vectorized
operations. R will automatically preserve
observations as you manipulate variables.
No other format works as intuitively with R.
FAM
M * A
*
tidyr::gather(cases, "year", "n", 2:4)
Gather columns into rows.
tidyr::unite(data, col, ..., sep)
Unite several columns into one.
dplyr::data_frame(a = 1:3, b = 4:6)
Combine vectors into data frame
(optimized).
dplyr::arrange(mtcars, mpg)
Order rows by values of a column
(low to high).
dplyr::arrange(mtcars, desc(mpg))
Order rows by values of a column
(high to low).
dplyr::rename(tb, y = year)
Rename the columns of a data
frame.
tidyr::spread(pollution, size, amount)
Spread rows into columns.
tidyr::separate(storms, date, c("y", "m", "d"))
Separate one column into several.
wwwwwwA1005A1013A1010A1010
wwp110110100745451009
wwp110110100745451009 wwp110110100745451009wwp110110100745451009
wppw11010071007110451009100945
wwwww110110110110110 wwww
dplyr::filter(iris, Sepal.Length > 7)
Extract rows that meet logical criteria.
dplyr::distinct(iris)
Remove duplicate rows.
dplyr::sample_frac(iris, 0.5, replace = TRUE)
Randomly select fraction of rows.
dplyr::sample_n(iris, 10, replace = TRUE)
Randomly select n rows.
dplyr::slice(iris, 10:15)
Select rows by position.
dplyr::top_n(storms, 2, date)
Select and order top n entries (by group if grouped data).
< Less than != Not equal to
> Greater than %in% Group membership
== Equal to is.na Is NA
<= Less than or equal to !is.na Is not NA
>= Greater than or equal to &,|,!,xor,any,all Boolean operators
Logic in R - ?Comparison, ?base::Logic
dplyr::select(iris, Sepal.Width, Petal.Length, Species)
Select columns by name or helper function.
Helper functions for select - ?select
select(iris, contains("."))
Select columns whose name contains a character string.
select(iris, ends_with("Length"))
Select columns whose name ends with a character string.
select(iris, everything())
Select every column.
select(iris, matches(".t."))
Select columns whose name matches a regular expression.
select(iris, num_range("x", 1:5))
Select columns named x1, x2, x3, x4, x5.
select(iris, one_of(c("Species", "Genus")))
Select columns whose names are in a group of names.
select(iris, starts_with("Sepal"))
Select columns whose name starts with a character string.
select(iris, Sepal.Length:Petal.Width)
Select all columns between Sepal.Length and Petal.Width (inclusive).
select(iris, -Species)
Select all columns except Species.
Learn more with browseVignettes(package = c("dplyr", "tidyr")) • dplyr 0.4.0• tidyr 0.2.0 • Updated: 1/15
wwwwwwA1005A1013A1010A1010
devtools::install_github("rstudio/EDAWR") for data sets
rstudio.com/resources/cheatsheets/](https://image.slidesharecdn.com/2016-03-01-nextgenerationprogramminginr-160301150812/85/Next-Generation-Programming-in-R-53-320.jpg)
Next Generation Programming in R
- 1. Next generation programming in R Florian Uhlitz [email protected] uhlitz.github.io %>%
- 2. magrittr readr tidyr dplyr %>% load data reshape data manipulate data Stefan Milton Bache, University of Southern Denmark Hadley Wickham, Rice University, RStudio Recent developments in the R environment
- 3. magrittr readr tidyr dplyr %>% load reshape manipulate%>% %>% Toolbox for data wrangling in R data wrangling adapted from H. Wickham
- 4. magrittr readr tidyr dplyr %>% load reshape manipulate%>% %>% Toolbox for data wrangling in R data wrangling model visualise adapted from H. Wickham
- 5. report magrittr readr tidyr dplyr %>% load reshape manipulate%>% %>% Toolbox for data wrangling in R data wrangling model visualise adapted from H. Wickham
- 6. report magrittr readr tidyr dplyr %>% load reshape manipulate%>% %>% Toolbox for data wrangling in R data wrangling model visualise base ggplot2 rmarkdown broom adapted from H. Wickham
- 7. data analysis report magrittr readr tidyr dplyr %>% load reshape manipulate%>% %>% Toolbox for data wrangling in R data wrangling model visualise base ggplot2 rmarkdown broom adapted from H. Wickham
- 8. magrittr In a pipe, the result of the left hand statement is handed over to the function on the right hand side: …similar to Unix pipe operator | f(x, y) x %>% f(y) f(x, y, z) x %>% f(y, z) f2(f1(x), y) f1(x) %>% f2(y)
- 12. tidyr gather() spread() Reshaping adapted from rstudio.com/resources/cheatsheets/
- 13. tidyr gather() spread() separate() unite() Reshaping adapted from rstudio.com/resources/cheatsheets/
- 14. dplyr filter(x > 1) select(B, C, E) A B C D E B C Ex 1 2 3 1 x 2 3 Subsetting adapted from rstudio.com/resources/cheatsheets/
- 15. dplyr Transforming Summarising 1 2 3 x 4 5 6 y 1 2 3 x 4 5 6 y 5 7 9 z mutate(z = x + y) summarise(A = sum(x), B = sum(y)) 1 2 3 x 4 5 6 y 6 A 15 B adapted from rstudio.com/resources/cheatsheets/
- 16. dplyr Transforming Summarising 1 2 3 x 4 5 6 y 1 2 3 x 4 5 6 y 5 7 9 z mutate(z = x + y) summarise(A = sum(x), B = sum(y)) 1 2 3 x 4 5 6 y 6 A 15 B group_by() %>% mutate() group_by() %>% summarise() adapted from rstudio.com/resources/cheatsheets/
- 18. »Happy families are all alike; every unhappy family is unhappy in its own way.« Leo Tolstoy Anna Karenina principle
- 19. »Tidy data sets are all alike; every messy data set is messy in its own way.« Hadley Wickham Tidy data principle
- 20. Tidy data definition Wickham, H. (2014). Tidy Data. Journal of Statistical Software
- 27. read_excel(“untidy_data.xlsx”) %>% set_colnames(mynames) %>% slice(1:36) %>% fill(group, condition) %>% separate(group, into = c(“Gene”, “Mutation”, “clone”), sep = “_”) %>% write_tsv(“tidy_data.tsv”)
- 28. read_excel(“untidy_data.xlsx”) %>% set_colnames(mynames) %>% slice(1:36) %>% fill(group, condition) %>% separate(group, into = c(“Gene”, “Mutation”, “clone”), sep = “_”) %>% write_tsv(“tidy_data.tsv”)
- 29. read_excel
- 31. read_excel %>% set_colnames %>% tail
- 33. read_excel %>% set_colnames %>% slice
- 34. read_excel %>% set_colnames %>% slice %>% fill
- 35. read_excel %>% set_colnames %>% slice %>% fill %>% select
- 36. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct
- 37. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct %>% separate
- 38. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct %>% separate Caution! readr, tidy & dplyr do “clever” stuff. (heuristics like predicting a column class by looking at the first 1000 entries)
- 39. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct separate
- 40. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct separate %>% unite
- 41. read_excel %>% set_colnames %>% slice %>% fill %>% select %>% distinct separate %>% unite
- 43. Tidy data definition Wickham, H. (2014). Tidy Data. Journal of Statistical Software
- 44. read_tsv
- 45. read_tsv %>% gather(key, value, -variable)
- 46. read_tsv %>% gather %>% spread(key, value)
- 48. read_tsv %>% gather %>% filter
- 49. read_tsv %>% gather %>% filter %>% group_by
- 50. read_tsv %>% gather %>% filter %>% group_by %>% summarise %>% arrange
- 51. read_tsv %>% gather %>% filter %>% group_by %>% summarise %>% arrange
- 52. read_tsv %>% gather %>% filter %>% group_by %>% summarise %>% arrange
- 53. Data Wrangling with dplyr and tidyr Cheat Sheet RStudio® is a trademark of RStudio, Inc. • CC BY RStudio • [email protected] • 844-448-1212 • rstudio.com Syntax - Helpful conventions for wrangling dplyr::tbl_df(iris) Converts data to tbl class. tbl’s are easier to examine than data frames. R displays only the data that fits onscreen: dplyr::glimpse(iris) Information dense summary of tbl data. utils::View(iris) View data set in spreadsheet-like display (note capital V). Source: local data frame [150 x 5] Sepal.Length Sepal.Width Petal.Length 1 5.1 3.5 1.4 2 4.9 3.0 1.4 3 4.7 3.2 1.3 4 4.6 3.1 1.5 5 5.0 3.6 1.4 .. ... ... ... Variables not shown: Petal.Width (dbl), Species (fctr) dplyr::%>% Passes object on left hand side as first argument (or . argument) of function on righthand side. "Piping" with %>% makes code more readable, e.g. iris %>% group_by(Species) %>% summarise(avg = mean(Sepal.Width)) %>% arrange(avg) x %>% f(y) is the same as f(x, y) y %>% f(x, ., z) is the same as f(x, y, z ) Reshaping Data - Change the layout of a data set Subset Observations (Rows) Subset Variables (Columns) F M A Each variable is saved in its own column F M A Each observation is saved in its own row In a tidy data set: & Tidy Data - A foundation for wrangling in R Tidy data complements R’s vectorized operations. R will automatically preserve observations as you manipulate variables. No other format works as intuitively with R. FAM M * A * tidyr::gather(cases, "year", "n", 2:4) Gather columns into rows. tidyr::unite(data, col, ..., sep) Unite several columns into one. dplyr::data_frame(a = 1:3, b = 4:6) Combine vectors into data frame (optimized). dplyr::arrange(mtcars, mpg) Order rows by values of a column (low to high). dplyr::arrange(mtcars, desc(mpg)) Order rows by values of a column (high to low). dplyr::rename(tb, y = year) Rename the columns of a data frame. tidyr::spread(pollution, size, amount) Spread rows into columns. tidyr::separate(storms, date, c("y", "m", "d")) Separate one column into several. wwwwwwA1005A1013A1010A1010 wwp110110100745451009 wwp110110100745451009 wwp110110100745451009wwp110110100745451009 wppw11010071007110451009100945 wwwww110110110110110 wwww dplyr::filter(iris, Sepal.Length > 7) Extract rows that meet logical criteria. dplyr::distinct(iris) Remove duplicate rows. dplyr::sample_frac(iris, 0.5, replace = TRUE) Randomly select fraction of rows. dplyr::sample_n(iris, 10, replace = TRUE) Randomly select n rows. dplyr::slice(iris, 10:15) Select rows by position. dplyr::top_n(storms, 2, date) Select and order top n entries (by group if grouped data). < Less than != Not equal to > Greater than %in% Group membership == Equal to is.na Is NA <= Less than or equal to !is.na Is not NA >= Greater than or equal to &,|,!,xor,any,all Boolean operators Logic in R - ?Comparison, ?base::Logic dplyr::select(iris, Sepal.Width, Petal.Length, Species) Select columns by name or helper function. Helper functions for select - ?select select(iris, contains(".")) Select columns whose name contains a character string. select(iris, ends_with("Length")) Select columns whose name ends with a character string. select(iris, everything()) Select every column. select(iris, matches(".t.")) Select columns whose name matches a regular expression. select(iris, num_range("x", 1:5)) Select columns named x1, x2, x3, x4, x5. select(iris, one_of(c("Species", "Genus"))) Select columns whose names are in a group of names. select(iris, starts_with("Sepal")) Select columns whose name starts with a character string. select(iris, Sepal.Length:Petal.Width) Select all columns between Sepal.Length and Petal.Width (inclusive). select(iris, -Species) Select all columns except Species. Learn more with browseVignettes(package = c("dplyr", "tidyr")) • dplyr 0.4.0• tidyr 0.2.0 • Updated: 1/15 wwwwwwA1005A1013A1010A1010 devtools::install_github("rstudio/EDAWR") for data sets rstudio.com/resources/cheatsheets/