2 + 3[1] 5
10 / 4[1] 2.5
2 * (3 + 1)[1] 8
MY202 · Lab 1 · Self-paced guide
This page walks you through Lab 1 one small step at a time. You don’t need any experience with R, coding, or statistics. If a word is new to you, it is explained the first time it appears, and there is a glossary at the end.
How it works. You keep this page open next to RStudio. The page explains each step and shows you exactly what you should see. You type the code into the lab notebook, a file that mixes instructions with boxes for code, and run it, which means asking R to carry it out. Both are explained step by step below. Take your time: there is no timer.
Your turn Something to do in RStudio. Follow the numbered instructions.
What you should see The real result R gives back. Compare it with yours.
Big idea A concept worth remembering beyond today.
Check yourself A quick question. Click an answer to see if you were right.
At the end of each step there is a Mark as done button. Your ticks are saved in this browser, so you can close the page and pick up where you left off. The bar at the top shows how far you’ve got.
You don’t have to do it all in one go. Parts 1 and 2 are the most important. Part 3 is harder, and it’s fine to leave it for another day.
R is a programming language for working with data. RStudio is the program you use to write and run R code. Think of R as the engine and RStudio as the car’s dashboard: you only ever touch the dashboard.
.qmd (the notebook) and a folder called data. Keep them together in the same folder..qmd file whose name contains student.A zip file is a folder squashed into one file to make downloading easier. Unpack it first: on Windows, right-click it and choose Extract All…; on a Mac, double-click it. Then use the normal folder that appears, not the zip file.
Windows often hides the last part of file names (the file extension, which tells the computer what kind of file it is). Look for the file whose name contains “student” and whose type is shown as “Quarto” or has a blue RStudio-style icon. Opening it through File → Open File… in RStudio also works.
When the notebook is open, RStudio looks roughly like this. Each area is called a pane.
That’s fine. The layout can differ between computers. When the notebook is open, the Source pane appears; before that, the Console takes up the whole left side.
The notebook mixes two things:
Here is what a chunk looks like, and what happens when you run it:
There are two ways to run code:
| To run… | Do this |
|---|---|
| the whole chunk | Click the green ▶ at the top right of the chunk, or press Ctrl + Shift + Enter |
| one line | Click anywhere on that line and press Ctrl + Enter |
Lines starting with # are comments. R ignores them: they are notes for humans. That’s why the empty chunks in the notebook say # Code here. You can type your code underneath that line.
The lines with ```{r} and ``` mark where a chunk starts and ends. Leave them alone and type your code between them.
R comes with a set of basic tools. Other people have written extra tools and shared them as packages. Before you can use a package, you load it with library(). You do this once each time you open RStudio.
The notebook loads four packages:
| Package | What it gives you |
|---|---|
tidyverse |
The main toolkit for today. The tidyverse is really a family of packages that are designed to work together and loads them all at once, including dplyr (summarising tables, Part 1), tidyr (reshaping tables, Part 2), readr (reading files, Part 2), purrr (repeating steps, Part 3) and tibble (a neater kind of table). |
psymetadata |
The research data we use in Parts 1 and 3. |
gt |
Turns a table into a nicely formatted one for reports (Part 2). |
ggsci |
Colour schemes for graphs. Not needed today. |
You don’t need to remember which function comes from which package. When this page says “from dplyr”, it just tells you where a tool lives.
library(.You may see some text in red, like this (your version numbers may differ). That’s normal. Red text is not always an error (R refusing to run something; there’s more on errors below).
── Attaching core tidyverse packages ──────────────── tidyverse 2.0.0 ──
✔ dplyr 1.2.1 ✔ readr 2.2.0
✔ forcats 1.0.1 ✔ stringr 1.6.0
✔ ggplot2 4.0.3 ✔ tibble 3.3.1
✔ lubridate 1.9.5 ✔ tidyr 1.3.2
✔ purrr 1.2.2
── Conflicts ───────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package to force all conflicts to become errors
The “Conflicts” part just says that two packages have a tool with the same name (for example filter()), and the one loaded last wins (“masks” the other). You can ignore it.
The package isn’t installed on this computer yet. Installing is a one-off step, like installing an app. Click in the Console (bottom left), type the line below with the missing package’s name, press Enter, wait until the > prompt comes back, then run the setup chunk again.
install.packages("psymetadata")Do not put install.packages() inside the notebook: you only need it once, not every time.
Before we touch any data, let’s learn the handful of words R uses. Try each example yourself: click into the Console, type the code, and press Enter. Mistakes here cost nothing.
Type each line into the Console and press Enter after each one.
2 + 3
10 / 4
2 * (3 + 1)2 + 3[1] 5
10 / 4[1] 2.5
2 * (3 + 1)[1] 8
The [1] at the start of each answer just means “this is the first value”. You’ll see it before almost every result, and you can ignore it for now.
Usually you want R to remember a result so you can use it later. You do this by giving it a name. A named thing in R is called an object.
x <- 5
_ and ., but no spaces.< followed by a -. Read it as “gets”: “x gets 5”. Shortcut: Alt + -.x <- 5
x
x * 2x <- 5
x[1] 5
x * 2[1] 10
Notice: the first line printed (showed) nothing. When you assign with <-, R stores the value silently. Look at the Environment pane (top right): x now appears there. Typing the name on its own (x) asks R to show you what’s inside.
If you don’t assign it, R forgets it. x * 2 showed 10, but x is still 5. To keep a result, you must save it with <-, either under a new name or by overwriting the old one.
R doesn’t know a thing called x yet. Either the line x <- 5 hasn’t been run, or the name is spelled differently. Capitals matter: X and x are two different names.
A function is a tool that takes something in, does something with it, and gives something back. You always recognise a function by its round brackets ( ).
round(3.14159, digits = 2)
round rounds a number.name = value. Arguments are separated by commas.round(3.14159, digits = 2)[1] 3.14
Every function has a help page that lists its arguments. To open it, put a ? in front of the name and run it, for example ?round. The help appears in the Help tab, bottom right. Help pages are written for experienced users, so don’t worry if they look dense: the Description at the top and the Examples at the bottom are the useful parts.
R treats different kinds of values differently. The kind is called the type.
| Type | Example | What it is |
|---|---|---|
| numeric | 57.1 |
a number |
| character | "Niger" |
text. Always in quotes. |
| logical | TRUE, FALSE |
yes or no; the answer to a question |
| missing | NA |
“not available”: the value is missing |
The quotes matter. Without them, R thinks a word is the name of an object and goes looking for it:
"Niger"[1] "Niger"
NigerError:
! object 'Niger' not found
And "1970" in quotes is text, not a number: you can’t do maths with it. We’ll fix exactly this problem in Part 2.
A vector is a sequence of values of the same type. You make one with the function c(), short for “combine”.
years <- c(1970, 1990, 2010)
years[1] 1970 1990 2010
years + 1[1] 1971 1991 2011
Most of R works on whole vectors at once: years + 1 added 1 to every value. A column in a table is a vector too.
== and %in%You can ask R yes/no questions. The answer is TRUE or FALSE, once for every value.
| Question | Write | Example |
|---|---|---|
| is equal to? | == |
year == 1990 |
| is not equal to? | != |
design != "Experiment" |
| greater / less than? | > < |
n > 100 |
| is one of? | %in% |
year %in% c(1970, 1990) |
years == 1990[1] FALSE TRUE FALSE
years %in% c(1970, 2010)[1] TRUE FALSE TRUE
filter(year = 1990)filter(year == 1990)filter() is a function that keeps only some rows of a table; you’ll use it in Part 2. One = sets an argument, like digits = 2. Two == ask “is this equal?”. If you mix them up, R tells you: “We detected a named input. This usually means that you’ve used = instead of ==.”
You have n <- 50. What does n > 100 give back?
50
> asks a question, so the answer is TRUE or FALSE, not a number.
FALSE
50 is not greater than 100, so the answer is FALSE.
An error
This is valid code: n exists and > compares two numbers.
You will see a lot of errors today. Everyone does, including people who have used R for twenty years. An error doesn’t break anything: R just refuses to run that line and tells you why. Read the message, fix the line, run it again.
| The message says… | It usually means… |
|---|---|
object 'xyz' not found |
A typo in a name, or you haven’t run the line that creates it yet |
could not find function "xyz" |
A typo in the function name, or the setup chunk hasn’t been run |
unexpected symbol / unexpected ')' |
A missing comma, bracket or quote. Count your brackets: every ( needs a ) |
there is no package called 'xyz' |
The package needs installing (see the setup step) |
+ trap
If the Console shows + instead of > at the start of the line, R is waiting for you to finish something, usually a missing ) or ". Press Esc to cancel and start the line again.
About 45–60 minutes. In this part you’ll explore a real research dataset and answer a question with it: do different kinds of studies find different results? You’ll learn the tools count(), mutate(), group_by() and summarise(), from a package called dplyr.
Scroll down in the notebook to Part 1. Each step below matches a numbered question there. The notebook skips number 5 (it goes 4, 6, 7…). That’s a typo in the notebook, not something you missed.
Researchers often want to know what all the studies on a topic found, not just one. A meta-analysis collects the results of many studies and combines them.
Our dataset, agadullina2018, is from a meta-analysis on prejudice. It asks: are people more prejudiced towards groups they see as tight-knit and uniform (psychologists call this entitativity)? Each row is one result from one published study.
Each result is summarised as an effect size: a single number for how strong the relationship was. Bigger numbers mean a stronger link between seeing a group as uniform and being prejudiced against it. In this data the effect size is in the column yi.
The data comes with the psymetadata package, so it is already loaded: you don’t need to open a file.
Before using any dataset, find out what it contains. Datasets that come in packages have a help page (also called documentation), just like functions.
?agadullina2018Studies on Out-Group Entitativity and Prejudice
Description:
Results from 21 studies, including 85 effect sizes
(fisher-z), on the effect of out-Group entitativity and
prejudice (Agadullina and Lovakov 2018).
Usage:
data("agadullina2018")
Format:
A data frame with 85 rows and 9 variables:
• 'es_id': effect size id
• 'study_id': study id
• 'author': study author
• 'pub_year': year of publication
• 'n': sample size
• 'design': within or between subjects design
• 'ent_alpha': reliability of the entitativity measure
• 'yi': effect size (fisher-z)
• 'vi': sampling variance (SE^2)
Details:
Further details can be found at https://osf.io/8dw5y/.
References:
Agadullina ER, Lovakov AV (2018). "Are people more
prejudiced towards groups that are perceived as coherent?
A meta-analysis of the relationship between out-group
entitativity and prejudice." _British Journal of Social
Psychology_, *57*(4), 703-731.
The most useful part is Format: it lists each column (also called a variable) and what it means. Note three of them:
design: how the study was done. Cross-sectional studies measure people’s views at one point in time; experiments change something on purpose and see what happens.yi: the effect size.vi: how uncertain that effect size is (smaller is more precise). We won’t use it today.The psymetadata package isn’t loaded yet. Run the setup chunk at the top of the notebook first, then try again.
Everything in R is an object, and objects come in different kinds, called classes. The function class() tells you which kind you have.
In the chunk under question 2, type and run:
class(agadullina2018)[1] "data.frame"
A data frame is R’s word for a table: rows and columns, like a spreadsheet. Each column is a vector, and all columns have the same length.
Typing the name of an object on its own and running it prints it: R shows you what’s inside.
In the chunk under question 3, type and run:
agadullina2018In the notebook, a table appears under the chunk with arrows to page through it. Scroll and click around: there are 85 rows (one per result) and 9 columns.
This table is small. Real datasets often have millions of rows, and printing them all would flood your screen. The notebook warns you not to print whole data frames in the exam. The next step shows a better way.
A tibble is a modern kind of data frame. It’s the same table, but it prints more helpfully: only the first few rows, plus the size of the table and the type of each column. All the tidyverse tools work with tibbles.
We’ll make a tibble version of the data and give it a new name, so we can use it in the rest of Part 1.
agadullina2018_tbl <- as_tibble(agadullina2018)
_tbl at the end is a reminder that it’s a tibble. Any name works, but you must use the same one later.as_tibble() converts a data frame to a tibble.In the chunk under question 4, type both lines and run the chunk:
agadullina2018_tbl <- as_tibble(agadullina2018)
agadullina2018_tblThe first line creates the tibble (silently). The second line prints it. Check the Environment pane: agadullina2018_tbl should now be listed.
How to read this table:
<chr> and <dbl>, are the column’s type: <chr> is character (text), <dbl> is a number with decimals (“double”), <int> a whole number.If you type agadullina2018_tbl in the Console instead of the notebook, the tibble prints as text. The first line tells you its size: # A tibble: 85 × 9 means 85 rows and 9 columns.
# A tibble: 85 × 9
es_id study_id authors pub_year n design ent_alpha yi vi
<int> <int> <chr> <chr> <int> <chr> <dbl> <dbl> <dbl>
1 1 1 Agadull… 2017 193 Cross… 0.897 0.814 0.00526
2 2 1 Agadull… 2017 193 Cross… 0.897 0.228 0.00526
3 3 1 Agadull… 2017 193 Cross… 0.897 0.521 0.00526
4 4 1 Agadull… 2017 193 Cross… 0.897 0.657 0.00526
5 5 2 Agadull… 2015 88 Cross… 0.839 -0.383 0.0118
# ℹ 80 more rows
glimpse(agadullina2018_tbl) turns the table on its side: one line per column, with the first few values. It’s handy when a table has too many columns to fit on the screen.
Rows: 85
Columns: 9
$ es_id <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 1…
$ study_id <int> 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3…
$ authors <chr> "Agadullina & Lovakov", "Agadullina & Lovakov", "Aga…
$ pub_year <chr> "2017", "2017", "2017", "2017", "2015", "2015", "201…
$ n <int> 193, 193, 193, 193, 88, 88, 298, 587, 587, 559, 559,…
$ design <chr> "Cross-sectional", "Cross-sectional", "Cross-section…
$ ent_alpha <dbl> 0.897, 0.897, 0.897, 0.897, 0.839, 0.839, 0.765, 0.8…
$ yi <dbl> 0.81439746, 0.22761291, 0.52077996, 0.65650427, -0.3…
$ vi <dbl> 0.005263158, 0.005263158, 0.005263158, 0.005263158, …
You printed the tibble before creating it. Run the line with <- first. If you run the whole chunk with ▶, both lines run in order.
Our question is whether different kinds of studies find different results. First: how many results come from each kind of study? The function count() counts how often each value appears in a column.
count(agadullina2018_tbl, design)
count(agadullina2018_tbl, design)The result is a new, small table: each design, and in the column n, how many rows have it. So 71 results come from cross-sectional studies and 14 from experiments.
|>The pipe |> is the most useful symbol you’ll learn today. It takes whatever is on its left and passes it into the function on its right, as the first input. Read it as “and then”.
Without a pipe
count(agadullina2018_tbl, design)With a pipe
agadullina2018_tbl |>
count(design)Both lines do exactly the same thing. The second reads: “take agadullina2018_tbl, and then count design”.
| then >. The | key is usually above Enter or next to Shift.The same result as Q6. With one step the pipe makes no difference. Its benefit shows when you chain several steps, as you’ll do next.
%>% is an older version of the pipe and works the same way for today. To switch: Tools → Global Options → Code, tick Use native pipe operator, |>, and click OK.
There’s a space between | and >. They must be typed right next to each other: |>.
mutate()Counts are useful, but percentages are easier to compare. mutate() adds a new column (or changes an existing one), calculated from the columns you already have.
mutate(pct = (n / sum(n)) * 100)
pct is short for percentage.=: here it means “set the new column to…”.n, divided by the total of all counts sum(n), times 100.Now chain it onto the counting step with another pipe. Click Start below to go through the code one line at a time.
agadullina2018_tbl |>
count(design) |>
mutate(pct = (n / sum(n)) * 100)Start with the data: one row per result, 85 rows.
and then count the designs. The pipe passes the table into count(). Now we have a small table with 2 rows.
and then add a column. mutate() receives the small count table, and calculates pct for each row: 71 out of 85 is 83.5%, 14 out of 85 is 16.5%.
Type the three lines into the chunk under question 8 and run it. Notice that every line except the last ends with a pipe: that’s what tells R the next line continues the same instruction.
n only exists after count() has made it. Check that count(design) comes before mutate(), and that the line with count ends with |>.
summarise()Now the actual research question. What is the average effect size? summarise() collapses a whole table into a single row of summary numbers. Inside it, mean() calculates an average.
summarise(agadullina2018_tbl, yi = mean(yi))
yi, but you could call it mean_effect or anything else.yi.summarise(agadullina2018_tbl, yi = mean(yi))The notebook says “per study”, but this calculates one average over all 85 results. Splitting it up comes in the next step.
One row, one number: on average, the effect size is about 0.41.
group_by()But do cross-sectional studies and experiments find the same thing? To get one average per design, put group_by(design) before summarise().
group_by() on its own changes nothing you can see. It puts an invisible label on each row, saying which group it belongs to. It changes how the next step behaves: summarise() then makes one row per group instead of one row in total.
agadullina2018_tbl |>
group_by(design) |>
summarise(yi = mean(yi))Start with the data: 85 rows.
group_by(design) sorts the rows into two groups. The table looks exactly the same. Only when it’s printed as text do you see a hint: the line # Groups: design [2] says there are 2 groups.
# A tibble: 85 × 9
# Groups: design [2]
es_id study_id authors pub_year n design ent_alpha yi vi
<int> <int> <chr> <chr> <int> <chr> <dbl> <dbl> <dbl>
1 1 1 Agadull… 2017 193 Cross… 0.897 0.814 0.00526
2 2 1 Agadull… 2017 193 Cross… 0.897 0.228 0.00526
3 3 1 Agadull… 2017 193 Cross… 0.897 0.521 0.00526
4 4 1 Agadull… 2017 193 Cross… 0.897 0.657 0.00526
5 5 2 Agadull… 2015 88 Cross… 0.839 -0.383 0.0118
# ℹ 80 more rows
summarise() now works within each group: one row per design, each with its own average. Cross-sectional studies: 0.468. Experiments: 0.111.
Type the three lines into the chunk under question 10 and run it. Then try removing the group_by line and running it again: you’re back to one row.
group_by(design) is missing, or it comes after summarise(). The order matters: group first, then summarise.
Time to put the results into words. This is the point of all the code: answering a question.
Which statement matches our results?
Experiments find larger effects than cross-sectional studies.
Look again at the Q10 result: experiments average 0.111 and cross-sectional studies 0.468.
Most results come from cross-sectional studies, and they find much larger effects than experiments.
84% of the results are cross-sectional (Q8), and their average effect is about four times as large as for experiments (Q10).
Both kinds of study find about the same effect.
0.468 and 0.111 are quite different. The kind of study seems to matter.
A simple average treats every result as equally trustworthy. But some come from big, precise studies and some from small ones (that’s what vi measures). Real meta-analyses give more weight to more precise results. You’ll learn how later in the course.
Part 1 done. You’ve used four of the tools that cover most everyday data work. A good moment for a break.
About 45–60 minutes. Here you’ll load a real file from the World Bank, discover that its layout makes it hard to analyse, and reshape it. You’ll learn read_csv(), rename(), pull(), pivot_longer(), filter(), pivot_wider() and gt(), from the readr, dplyr, tidyr and gt packages. Each is explained when you get to it.
The same information can be laid out in a table in many ways. Most R tools expect one particular layout, called tidy data:
Illustration: Julia Lowndes and Allison Horst, Tidy Data for reproducibility, efficiency, and collaboration, Openscapes, 2020. CC BY 4.0.
Our World Bank file shows life expectancy (how many years a newborn baby can expect to live) for every country, for every year from 1960 to 2024. It’s laid out like this:
Wide: how the file looks 263 rows × 67 columns
| Country Name | 1970 | 1990 | 2010 | … |
| Burkina Faso | 39.5 | 49.3 | 56.3 | … |
| Honduras | 54.4 | 65.2 | 69.8 | … |
| Niger | 35.9 | 41.7 | 57.1 | … |
Easy for a human to read. But year is a variable, and here it is hidden in the column names. There’s no column called “year” that you could filter or group by.
Long (tidy): what we want 3 columns
| country_name | year | lifeexp |
| Burkina Faso | 1970 | 39.5 |
| Burkina Faso | 1990 | 49.3 |
| Burkina Faso | 2010 | 56.3 |
| Honduras | 1970 | 54.4 |
| … | … | … |
One row per country and year. Every variable has its own column, so the tools from Part 1 work.
Wide is good for showing a table to people. Long is good for analysing it. Most data cleaning is about getting data into the long, tidy shape.
So far the data came inside a package. Now we load a file from your computer. The file is a CSV (“comma-separated values”): a plain-text table where commas separate the columns. Excel can open it too.
read_csv() reads a CSV file and turns it into a tibble. You tell it where the file is with a path:
read_csv("data/wb-life-exp.csv")
data, the file wb-life-exp.csv”. It’s text, so it goes in quotes. The / separates a folder from what’s inside it.The path starts from the folder the notebook is in. So this only works if the data folder sits next to the notebook file. You can check in the Files tab, bottom right.
The first column is called Country Name, with a space and capitals. Spaces in names are awkward in code, so we rename it to country_name, using rename(new_name = old_column). Instead of typing the old name, you can give its position: 1 means “the first column”.
lifeexp <- read_csv("data/wb-life-exp.csv") |>
rename(country_name = 1)
lifeexpread_csv() reads the file. Look at the first column’s name, Country Name, and the year columns: 1960, 1961, … R also prints a short message about the column types it guessed.
and then rename() changes the first column’s name to country_name. The whole result is saved under the name lifeexp (short for life expectancy), thanks to the <- at the start.
The arrow saves the result of the whole pipeline, not just the first line.
Print it to check. The first column is now called country_name.
lifeexp appears in the Environment pane, with 263 obs. (observations, or rows) of 67 variables.R can’t find the file at that path. The working directory is the folder R starts looking in; for a notebook, that’s the folder the notebook is in. Check, in this order:
data, with wb-life-exp.csv inside?data folder), drop the folder from the path: read_csv("wb-life-exp.csv").wb-life-exp (1).csv? Rename the file, or use its exact name in the path.pull()pull() takes one column out of a table and gives it to you as a plain vector: just the values, in a row. It’s a quick way to look through everything in a column.
pull(lifeexp, country_name)Scroll through the output. Is every name a country?
[1] "Aruba" "Africa Eastern and Southern"
[3] "Afghanistan" "Africa Western and Central"
[5] "Angola" "Albania"
[7] "Andorra" "Arab World"
[9] "United Arab Emirates" "Argentina"
[11] "Armenia" "American Samoa"
[13] "Antigua and Barbuda" "Australia"
[15] "Austria" "Azerbaijan"
[17] "Burundi" "Belgium"
[19] "Benin" "Burkina Faso"
[21] "Bangladesh" "Bulgaria"
[23] "Bahrain" "Bahamas, The"
[25] "Bosnia and Herzegovina" "Belarus"
[27] "Belize" "Bermuda"
[29] "Bolivia" "Brazil"
[31] "Barbados" "Brunei Darussalam"
[33] "Bhutan" "Botswana"
[35] "Central African Republic" "Canada"
[37] "Central Europe and the Baltics" "Switzerland"
[39] "Channel Islands" "Chile"
[41] "China" "Cote d'Ivoire"
[43] "Cameroon" "Congo, Dem. Rep."
[45] "Congo, Rep." "Colombia"
[47] "Comoros" "Cabo Verde"
[49] "Costa Rica" "Caribbean small states"
[51] "Cuba" "Curacao"
[53] "Cayman Islands" "Cyprus"
[55] "Czechia" "Germany"
[57] "Djibouti" "Dominica"
[59] "Denmark" "Dominican Republic"
Not every row is a country. About 46 rows are groups of countries: “Arab World”, “Euro area”, “High income”, “World”, and so on. If you averaged across all rows, you’d count many countries twice. Always look at your data before you analyse it.
pivot_longer()Now the main event. pivot_longer() turns a wide table into a long one. It needs answers to three questions:
pivot_longer(-country_name, names_to = "year", values_to = "lifeexp")
country_name”: the minus sign means “not this one”.year. Its name is new, so it’s in quotes.lifeexp.Before one row per country
| country_name | 1970 | 1990 |
| Niger | 35.9 | 41.7 |
After one row per country and year
| country_name | year | lifeexp |
| Niger | "1970" | 35.9 |
| Niger | "1990" | 41.7 |
lifeexp |>
pivot_longer(-country_name, names_to = "year", values_to = "lifeexp")You can split a long line after any comma to make it easier to read. RStudio keeps it as one instruction until the closing bracket.
The table now has 3 columns and 17,358 rows: 263 rows × 66 year columns. Look at the type under year: <chr>, text. The years came from column names, and column names are always text. We fix that next.
NA means a value is missing. Some countries have no data for early years, and the file’s last column, 2025, is completely empty because that year isn’t over yet. You could remove those rows by adding values_drop_na = TRUE inside pivot_longer().
New column names in names_to and values_to need quotes: "year", "lifeexp". And check the commas between the three parts.
parse_number()parse_number() takes text and pulls the number out of it: "1970" becomes 1970. We use it inside mutate(), which you know from Part 1. This time, mutate() replaces the year column with a new version of itself.
mutate(year = parse_number(year))
year.Copy your code from Q3 into the chunk under Q4, add a pipe at the end of the last line, and add the mutate line:
lifeexp |>
pivot_longer(-country_name, names_to = "year", values_to = "lifeexp") |>
mutate(year = parse_number(year))The type under year is now <dbl>: a number. Now we can ask number questions like “is the year 1990?”.
Each question adds one line to the end of the previous answer. You never rewrite from scratch: copy the last working version, add |>, add the next step, run. If it breaks, the problem is almost always in the line you just added.
filter()filter() keeps the rows for which a question is answered TRUE, and drops the rest. Remember %in% from the basics: it asks “is this value one of these?”.
filter(country_name %in% c("Niger", "Honduras"), year %in% c(1970, 1990, 2010))
TRUE.filter() line to your pipeline. Here’s an example with four countries; swap in your own:lifeexp |>
pivot_longer(-country_name, names_to = "year", values_to = "lifeexp") |>
mutate(year = parse_number(year)) |>
filter(country_name %in% c("Burkina Faso", "Honduras", "Niger", "Suriname"),
year %in% c(1970, 1990, 2010))4 countries × 3 years = 12 rows.
pull() output. For example, the data says “Egypt, Arab Rep.”, not “Egypt”.1970, not "1970".== instead of %in%. == compares values one by one in order, and quietly gives the wrong rows. To check against several values, always use %in%.pivot_wider()The long format was good for filtering. But to show the result to someone, one row per country with a column per year is easier to read. pivot_wider() is the reverse of pivot_longer().
pivot_wider(names_from = year, values_from = lifeexp)
year. The column exists already, so no quotes.lifeexp.Add one more pipe and the pivot_wider() line to your pipeline from Q5:
lifeexp |>
pivot_longer(-country_name, names_to = "year", values_to = "lifeexp") |>
mutate(year = parse_number(year)) |>
filter(country_name %in% c("Burkina Faso", "Honduras", "Niger", "Suriname"),
year %in% c(1970, 1990, 2010)) |>
pivot_wider(names_from = year, values_from = lifeexp)To keep one decimal place, add lifeexp = round(lifeexp, 1) to the mutate(): mutate(year = parse_number(year), lifeexp = round(lifeexp, 1)). The completed notebook does this, and also shows five years (1970, 1980, 1990, 2000, 2010).
gt()gt() (from the gt package, short for “grammar of tables”) turns a tibble into a formatted table that you could put in a report.
Add |> gt() to the end of your pipeline. Here is the whole thing. Step through it below to see what each line does.
This is the full pipeline from Part 2. Press Start to replay it one step at a time.
lifeexp |>
pivot_longer(-country_name,
names_to = "year",
values_to = "lifeexp") |>
mutate(year = parse_number(year),
lifeexp = round(lifeexp, 1)) |>
filter(country_name %in% c("Burkina Faso",
"Honduras", "Niger", "Suriname"),
year %in% c(1970, 1990, 2010)) |>
pivot_wider(names_from = year,
values_from = lifeexp) |>
gt()The wide table we loaded: 263 rows, one per country or region; one column per year.
pivot_longer() stacks the year columns: one row per country and year, 17,358 rows.
mutate() turns year into a number and rounds life expectancy to one decimal.
filter() keeps four countries in three years: 12 rows.
pivot_wider() spreads the years back out into columns: one row per country.
gt() formats the table for a report.
| country_name | 1970 | 1990 | 2010 |
|---|---|---|---|
| Burkina Faso | 39.5 | 49.3 | 56.3 |
| Honduras | 54.4 | 65.2 | 69.8 |
| Niger | 35.9 | 41.7 | 57.1 |
| Suriname | 59.5 | 62.1 | 69.6 |
Look at your table. Which of your countries gained the most years of life expectancy between 1970 and 2010? Did any of them go down for a while?
In the example table, which country gained the most between 1970 and 2010?
Suriname
Suriname went from 59.5 to 69.6: about +10 years, the smallest gain of the four.
Niger
Niger went from 35.9 to 57.1: more than 21 extra years. Most of that gain came after 1990.
Honduras
Honduras gained about 15 years (54.4 to 69.8): a lot, but less than Niger.
read → rename → reshape → fix types → filter → reshape back → present. You’ve just done a complete small analysis. Most data projects follow this same shape, just with more steps.
About 30–45 minutes. This part is harder, so go slowly, and it’s fine to stop here and come back another day. You’ll learn map() and bind_rows(), from the purrr and dplyr packages.
Sometimes data doesn’t arrive as one table but as several: one file per year, one spreadsheet tab per region. In R, you can keep several tables together in a list.
A list is a container that can hold anything, including whole tables. Think of it as a box with several labelled compartments.
The data for this part is another meta-analysis, maldonado2020: 1,268 results on how ageing affects executive functions, the mental skills you use to plan, focus and switch between tasks. The notebook splits it into a list of four tables, one per skill:
split_data: a list of 4 tables
Find question 1 in Part 3. The chunk already contains code that builds split_data. You don’t need to understand it: just run it with the green ▶. Check that split_data appears in the Environment pane as a “List of 4”.
maldonado2020 comes with the psymetadata package. Run the setup chunk at the top of the notebook, then this chunk again.
map()Say you want to see the first 5 rows of each table. You could write the same line four times, once per table. But that’s tedious, and every copy is a chance for a typo. map() does it for you: it takes a list and a function, and applies the function to every piece.
map(list, f)To say what to do to each piece, you write a small recipe with a ~ (tilde) and .x:
map(split_data, ~ head(.x, 5))
~ means “here comes a recipe”. On most keyboards it’s Shift + the key next to 1 or next to Enter.head() shows the first rows of a table..x is a placeholder for “the current piece”. map() swaps in each table in turn: first inhibition, then processing speed, and so on.map(split_data, ~ head(.x, 5))$inhibition
author yi vi task
12 Bugg, 2014A 1.957228 272.179381 Stroop
13 Bugg, 2014B 3.391032 164.365623 Stroop
14 Bugg, 2014C 3.409401 163.539770 Stroop
17 Circelli, 2013 1.077655 1.261782 Anti-saccade Task
18 Cona, 2013 1.787737 28.994392 Go/No-Go Task
$`processing speed`
author yi vi n1 n2 mean_age1 mean_age2
3 Balouch, 2013 1.5349718 0.9803923 28 29 24.64 67.86
8 Bernard, 2013 0.6710382 412.2254792 10 16 21.7 61.9
19 Gillis, 2013 -0.3854916 0.8525118 25 34 20.52 68.12
25 Kelly, 2013 2.2941662 0.1349244 48 48 18-24 60-79
26 Kelly, 2013 2.5073828 0.2895045 48 48 18-24 60-79
$shifting
author yi vi task
2 Balouch, 2013 -0.2578337 7.189572e-02 Trail Making Test
5 Barulli, 2013 2.7258066 3.450196e+04 Shifting Task
6 Bernard, 2013 1.2530391 1.939528e+00 Trail Making Test
10 Brand, 2013 0.7824283 5.386653e-02 Modified WCST
20 Gillis, 2013 -0.1181795 7.892916e-01 Trail Making Test
$updating
author yi vi n1 n2 mean_age1 mean_age2
1 Aguirre, 2014 0.8708623 2.2852193 48 48 20.5 68.31
4 Balouch, 2013 0.3963551 0.1139885 28 29 24.64 67.86
7 Bernard, 2013 2.0009349 82.3702214 10 16 21.7 61.9
9 Brand, 2013 0.4282817 0.1627862 251 46 24.55 70.35
11 Brand, 2013 0.7656311 2.4470096 251 46 24.55 70.35
The result is again a list of four. Each piece is labelled with $ and its name, like $inhibition. Notice that the pieces have different columns: only author, yi and vi appear in all four.
You forgot the ~ before the recipe. Without it, R tries to run head(.x, 5) straight away, before map() has filled in .x.
map()If the function needs nothing but the piece itself, you can just give its name, without ~ or .x. For example, colnames() lists a table’s column names:
Try these, one at a time. Run each and look at what comes back.
map(split_data, colnames)
map(split_data, as_tibble)
map(split_data, ~ select(.x, author, yi, vi))$inhibition
[1] "author" "yi" "vi" "task"
$`processing speed`
[1] "author" "yi" "vi" "n1" "n2"
[6] "mean_age1" "mean_age2"
$shifting
[1] "author" "yi" "vi" "task"
$updating
[1] "author" "yi" "vi" "n1" "n2"
[6] "mean_age1" "mean_age2"
select() is new: it keeps only the columns you name. Where filter() picks rows, select() picks columns. We keep the three columns that every piece has.
Now combine the last two into one recipe. Inside the recipe, you can use a pipe, just like before:
map(
split_data,
~ .x |>
as_tibble() |>
select(author, yi, vi)
)map() is going to go through split_data, piece by piece. Spreading the code over several lines makes it easier to read; the brackets show where map( starts and ends.
For each piece: start with that piece, .x, and then…
…turn it into a tibble, and then…
…keep the three columns author, yi and vi. The ) on the last line closes map(. The result: a list of four tibbles, all with the same three columns. (Shortened to 3 rows each here.)
$inhibition
# A tibble: 289 × 3
author yi vi
<chr> <dbl> <dbl>
1 Bugg, 2014A 1.96 272.
2 Bugg, 2014B 3.39 164.
3 Bugg, 2014C 3.41 164.
# ℹ 286 more rows
$`processing speed`
# A tibble: 270 × 3
author yi vi
<chr> <dbl> <dbl>
1 Balouch, 2013 1.53 0.980
2 Bernard, 2013 0.671 412.
3 Gillis, 2013 -0.385 0.853
# ℹ 267 more rows
$shifting
# A tibble: 208 × 3
author yi vi
<chr> <dbl> <dbl>
1 Balouch, 2013 -0.258 0.0719
2 Barulli, 2013 2.73 34502.
3 Bernard, 2013 1.25 1.94
# ℹ 205 more rows
$updating
# A tibble: 501 × 3
author yi vi
<chr> <dbl> <dbl>
1 Aguirre, 2014 0.871 2.29
2 Balouch, 2013 0.396 0.114
3 Bernard, 2013 2.00 82.4
# ℹ 498 more rows
task doesn’t exist”
You selected a column that only some pieces have. map() applies the same recipe to every piece, so it must work for all of them. Check with map(split_data, colnames) and pick columns that appear everywhere.
bind_rows()Now that every piece has the same columns, we can stack them on top of each other into one big table. bind_rows() does exactly that.
But if we just stack them, we lose track of which piece each row came from. .id = "domain" adds a new column, domain, that records the name of the piece.
map(
split_data,
~ .x |>
as_tibble() |>
select(author, yi, vi)
) |>
bind_rows(.id = "domain")The list of four tibbles from Q3, each with the same three columns.
$inhibition
# A tibble: 289 × 3
author yi vi
<chr> <dbl> <dbl>
1 Bugg, 2014A 1.96 272.
2 Bugg, 2014B 3.39 164.
3 Bugg, 2014C 3.41 164.
# ℹ 286 more rows
$`processing speed`
# A tibble: 270 × 3
author yi vi
<chr> <dbl> <dbl>
1 Balouch, 2013 1.53 0.980
2 Bernard, 2013 0.671 412.
3 Gillis, 2013 -0.385 0.853
# ℹ 267 more rows
$shifting
# A tibble: 208 × 3
author yi vi
<chr> <dbl> <dbl>
1 Balouch, 2013 -0.258 0.0719
2 Barulli, 2013 2.73 34502.
3 Bernard, 2013 1.25 1.94
# ℹ 205 more rows
$updating
# A tibble: 501 × 3
author yi vi
<chr> <dbl> <dbl>
1 Aguirre, 2014 0.871 2.29
2 Balouch, 2013 0.396 0.114
3 Bernard, 2013 2.00 82.4
# ℹ 498 more rows
and then bind_rows() stacks them into one table of 1,268 rows. The new first column, domain, says which piece each row came from.
Type the code into the chunk under question 4 and run it. Check: the result should have 1,268 rows and 4 columns.
map_dfr()map_dfr() does map() and bind_rows() in one go. The dfr stands for “data frame, by rows”. It gives the same table as Q4.
map_dfr(
split_data,
~ .x |>
as_tibble() |>
select(author, yi, vi),
.id = "domain"
)That means there’s a newer way to do the same thing, map(...) |> list_rbind(), but the old one still works. You’ll see map_dfr() in a lot of code online.
Part 1 · dplyr count, mutate, group_by and summarise, chained with |>, cover most everyday summaries.
Part 2 · tidyr Tidy data has one variable per column and one observation per row. pivot_longer() and pivot_wider() switch between shapes.
Part 3 · purrr map() applies one recipe to every piece of a list. bind_rows() stacks the results.
One question to take away: think of the data you might use for your own project. Is it tidy? If not, which rule does it break?
Press Ctrl + S in RStudio to save the notebook. Your code is saved in the .qmd file, so next time you can open it and run everything again.
Further reading
Next week: making graphs with ggplot.
Underlined words on this page are explained in this list. Click or tap one anywhere on the page to see what it means and where it was first explained.
| Word | Meaning |
|---|---|
| argument | An input to a function, written inside its brackets: round(3.14, digits = 1) |
assign, <- |
Store a value under a name, so R remembers it |
| chunk | A grey box of R code in a notebook |
| class | The kind of object, e.g. "data.frame". Check with class() |
| comment | A line starting with #. R ignores it: it’s a note for humans |
| console | The pane where you can type R code line by line |
| CSV | A plain-text table file where commas separate the columns |
| data frame | R’s word for a table with rows and columns |
| documentation | The help page for a function or dataset. Open it with ?, e.g. ?round |
| effect size | A number for how strong a relationship is in a study |
| file extension | The end of a file name after the dot, e.g. .csv or .qmd. It tells the computer what kind of file it is |
| function | A tool that takes inputs and gives back a result. Always has brackets: mean() |
| list | A container that can hold several objects, even whole tables |
| meta-analysis | A study that combines the results of many other studies |
| NA | A missing value |
| object | Anything R has stored under a name |
| observation | One case you measured, e.g. one country in one year; in tidy data, a row |
| package | A bundle of extra functions written by others. Install once, load with library() every session |
| pane | One of the four areas of the RStudio window |
| path | Where a file is, written as folders and a file name: "data/wb-life-exp.csv" |
pipe, |> |
“And then”: passes the result on the left into the function on the right |
| pipeline | Several steps chained together with |>, read top to bottom |
| tibble | A modern data frame that prints more neatly |
| tidy data | One variable per column, one observation per row, one value per cell |
| tidyverse | A family of packages that work together, loaded with library("tidyverse"). Includes dplyr, tidyr, readr, purrr and tibble |
| type | The kind of value: number, text (character), TRUE/FALSE (logical) |
| variable | Something measured; in tidy data, a column |
| vector | Several values of the same type in a row, made with c() |
| working directory | The folder R looks in when you give it a file name. For a notebook, the folder the notebook is in |
| zip file | A folder squashed into one file for downloading. Unpack (extract) it before using what’s inside |