[1] 25368 20
MY202 · Lab 4
Jonas Weinert
Department of Methodology | London School of Economics and Political Science
29 October 2026
| Part | Topic | Functions | Mode | Time |
|---|---|---|---|---|
| Setup and two warm-up questions | together | 6 min | ||
| I | Exploring the data | dim(), across(), as.factor(), ggplot |
your turn · together · your turn | 24 min |
| II | Logistic regression | initial_split(strata =), logistic_reg(), summary() |
concept, then your turn | 23 min |
| III | Evaluation | augment(), conf_mat(), metric_set(), map_dfr() |
together, then your turn | 31 min |
| Wrap-up (+ 2 min spare) | 6 min |
Together: we type the code into the notebook, the slide explains each piece, and the line number says where. Your turn: a timer, tasks and an annotated syntax sheet, then the solutions step by step.
Warm-up
Two questions · predicting categories instead of numbers
Last week, why did we score the model on a test set instead of on the rows it was fitted on?
1 in 100 people has a rare disease. A model predicts “healthy” for everyone. What is its accuracy (the share of predictions that are right)?
Part I
25,368 people · 19 health indicators · one yes/no outcome
Notebook lines 31–56 the variable list, then read_csv()run together
diabetes, the outcome: 1 = yes, 0 = no. The classes are imbalanced.Health conditions high_bp · high_chol · chol_check · bmi · stroke · heart_diseaseor_attack · diff_walk · gen_hlth
Lifestyle smoker · phys_activity · fruits · veggies · hvy_alcohol_consump
Who they are sex · age · education · income · any_healthcare · no_docbc_cost
Everything is stored as a number. Most columns are 0/1. age is a code for 5-year bands (1 = 18–24, 13 = 80 or older); gen_hlth runs from 1 (excellent) to 5 (poor); income 1–8 and education 1–6 are ordered levels. bmi is the only real measurement.
Part I · Q1 and Step 1 · lines 58–74
Syntax you need
# rows and columns
dim(diabetes_data)
# count the NAs in every column:
# across(everything(), ...) applies the same
# function to each column; .x is "this column"
diabetes_data |>
summarise(across(everything(), ~ sum(is.na(.x))))
# one line per column, with its type
glimpse(diabetes_data)Stuck?
read_csv() chunk at line 55 firstis.na() gives TRUE/FALSE; sum() counts the TRUEsdim(diabetes_data)
diabetes_data |>
summarise(across(everything(), ~ sum(is.na(.x)))) |>
glimpse()
glimpse(diabetes_data)Size, missing values, column types. Each click shows the next result.
25,368 rows (people) and 20 columns: the outcome and 19 features.
[1] 25368 20
summarise() collapses the table to one row; across(everything(), …) puts the NA count of each column in it. glimpse() prints that row sideways. All zeros: no missing values.
Rows: 1
Columns: 8
$ diabetes <int> 0
$ high_bp <int> 0
$ high_chol <int> 0
$ chol_check <int> 0
$ bmi <int> 0
$ smoker <int> 0
$ stroke <int> 0
$ heart_diseaseor_attack <int> 0
Every column is <dbl>, a decimal number, even yes/no ones like high_bp. R can’t tell a category from a measurement. That’s the next slide.
Rows: 25,368
Columns: 8
$ diabetes <dbl> 0, 0, 0…
$ high_bp <dbl> 0, 1, 0…
$ high_chol <dbl> 0, 1, 0…
$ chol_check <dbl> 1, 1, 1…
$ bmi <dbl> 21, 28,…
$ smoker <dbl> 0, 0, 0…
$ stroke <dbl> 0, 0, 0…
$ heart_diseaseor_attack <dbl> 0, 0, 0…
Steps 2 and 3 show the first 8 of 20 columns to fit the slide.
Notebook line 86 "Think about it": write your answer in the notebookdiscuss
bmi, gen_hlth (1 = excellent … 5 = poor) and high_bp (0/1) are all stored as numbers. Which of them really are numbers?
Continuous → numeric Any value in a range: bmi, and age (the notebook treats the band code as continuous).
Ordered levels → integer Whole-number scales where the order means something: gen_hlth, education, income.
Categories → factor Yes/no columns: high_bp, smoker, sex… and the outcome diabetes. As a number, sex = 1 is “one more than 0”. It isn’t.
For classification the outcome must be a factor. That’s how tidymodels knows the job.
Notebook lines 92–104 Step 3 at line 93, Step 4 at line 103code together
continuous_vars <- c("age", "bmi") discrete_vars <- c("gen_hlth", "education", "income") categorical_vars <- c("diabetes", "high_bp", "high_chol", …, "sex") diabetes_data <- diabetes_data |> mutate( across(all_of(continuous_vars), ~ as.numeric(.x)), across(all_of(discrete_vars), ~ as.integer(.x)), across(all_of(categorical_vars), ~ as.factor(.x))) map_chr(diabetes_data, class)
~ makes a mini-function, .x the current column.class() per column.Part I · EDA challenge · line 145
diabetes: what share has it?ageSyntax you need
# share with and without diabetes
diabetes_data |>
count(diabetes) |> # one row per class
mutate(pct = n / sum(n) * 100) # turn counts into %
# prevalence by age group
diabetes_data |>
group_by(age) |>
summarise(prev = mean(diabetes == "1")) |> # share of 1s
ggplot(aes(x = age, y = prev)) +
geom_line() + geom_point()
# a number against the outcome
diabetes_data |>
ggplot(aes(x = diabetes, y = bmi)) +
geom_boxplot()Stuck?
diabetes == "1" is TRUE/FALSE; mean() of TRUE/FALSE is the share of TRUEsdiabetes is a factor now; compare it with "1", don’t average itAge From 1% at 18–24 to over 20% in the 70s. Only 13.8% overall: few positives.
Conditions and lifestyle Four times the prevalence with high blood pressure. Half with physical activity. Plots suggest; a model can hold the others fixed. Part II.
Part II
From a line to an S-curve · probabilities · thresholds · odds
A made-up chess club: games played so far, and whether the next one was won (1) or lost (0).
What’s the chance a player wins? Let’s fit a line, like last week.
A line doesn’t know when to stop A probability lives between 0 and 1. A line runs off to ±∞.
Logistic regression fits an S-shaped curve, the sigmoid: flat near 0 for beginners, flat near 1 for veterans, steep in between.
What comes out A probability: the chance that the outcome is 1. In R this column is called .pred_1.
0.5 is a choice, not a law Who says it should be 0.5? When the 1s are rare, as with diabetes, it often shouldn’t be. Part III moves it.
How much it matters A steeper S: the chance changes faster as the feature rises. Flat: the feature makes no difference.
Which way Rising: more of the feature, more risk. Falling: more of the feature, less risk.
Where it starts The baseline risk before the feature does anything. These dials are the coefficients summary() prints, on the log-odds scale; exp() turns each into an odds ratio.
Odds ratio (OR) = exp(estimate) Odds: a 1-in-10 chance is odds of 1 to 9. The OR multiplies the odds. High blood pressure: 2.29.
Read it like this Above 1: higher odds; below 1: lower; 1: no difference. BMI 1.06: 6% higher odds per point. Heavy drinker 0.37: 63% lower.
What OR 2.29 means for a person A 10% chance becomes about 20% with high blood pressure, all else equal: roughly double, when the risk is small.
Part II · Q1–Q4 · lines 176–208
strata = diabetestrainsummary()Lab 3 code, changed for today
set.seed(123) # same split every time
split_grid <- initial_split(diabetes_data, prop = 0.75,
strata = diabetes) # NEW: same share
train <- training(split_grid) # of 1s in both sets
test <- testing(split_grid)
model <-
logistic_reg() |> # was: linear_reg()
fit(diabetes ~ ., data = train) # outcome ~ every other column
model |>
extract_fit_engine() |> # pull out R's glm object
summary() # so summary() can print itStuck?
summary(model) prints little: add extract_fit_engine() in between, as in Lab 3glm: no set_engine() neededset.seed(123)
split_grid <- initial_split(diabetes_data, prop = 0.75,
strata = diabetes)
train <- training(split_grid)
test <- testing(split_grid)
model <-
logistic_reg() |>
fit(diabetes ~ ., data = train)
model |>
extract_fit_engine() |>
summary()Split with the outcome as strata, fit the logistic regression on the training rows, print the coefficients.
19,025 training rows, 6,343 test rows. Thanks to strata, both have the same share with diabetes, 13.8%.
# A tibble: 4 × 4
# Groups: set [2]
set diabetes n share
<chr> <fct> <int> <dbl>
1 train 0 16405 0.862
2 train 1 2620 0.138
3 test 0 5469 0.862
4 test 1 874 0.138
fit() sets the dials: one coefficient per feature plus the intercept. For now, read only the sign: positive means more risk.
# A tibble: 20 × 2
term estimate
<chr> <dbl>
1 (Intercept) -7.53
2 high_bp1 0.828
3 high_chol1 0.658
4 chol_check1 1.03
5 bmi 0.06
6 smoker1 -0.078
7 stroke1 0.061
8 heart_diseaseor_attack1 0.274
# ℹ 12 more rows
The coefficient table from summary(): estimate, z value and p-value (the standard error is left out to fit), as in Lab 3 with z instead of t. Sorted by p-value here: general health, BMI, blood pressure, cholesterol, age.
Estimate z p
(Intercept) -7.534 -22.5 3.5e-112
gen_hlth 0.507 18.5 1.0e-76
bmi 0.060 18.3 1.9e-74
high_bp1 0.828 15.0 3.7e-51
high_chol1 0.658 12.9 2.5e-38
age 0.125 12.3 6.6e-35
hvy_alcohol…1 -0.984 -6.4 1.2e-10
sex1 0.231 4.6 3.3e-06
Step 3 shows the 8 rows with the smallest p-values; summary() prints all 20 in column order, plus the deviance and AIC below.
Notebook lines 210–235 class discussion: write the answers into the four boxesdiscuss
Estimate The change in the log-odds per unit of the feature; its sign says which way. exp(estimate) is the odds ratio: gen_hlth OR 1.66 per step towards “poor” health.
p-value As last week: if the true effect were 0, how surprising is this estimate? high_bp, bmi, gen_hlth, age: extremely. smoker, phys_activity: not, once the rest is held fixed.
Careful Heavy drinkers: odds × 0.37, “63% lower”. Does alcohol protect? More likely people with diabetes are told to stop drinking. Association, not cause.
Box 4: which metrics? Not accuracy: only 14% have diabetes, so “no” for everyone is already 86% right. Part III gives the alternatives.
p-values say what’s associated. They don’t say whether the model predicts well.
Part III
Is the model any good? · confusion matrix · precision · recall · thresholds
Notebook lines 241–251 Q1 at line 242, Q2 at line 250code together
train_preds <- model |> augment(new_data = train) test_preds <- model |> augment(new_data = test) pull(test_preds, .pred_1)[1:10]
.pred_class (0 or 1), .pred_0 and .pred_1 (the two probabilities).Notebook lines 257–259 Q3: threshold 0.5, both setscode together
test_preds <- test_preds |> mutate(.pred_class = as.factor( if_else(.pred_1 > 0.5, 1, 0)))
augment()’s own .pred_class, which used 0.5 too; now the rule is in plain sight.Look where the mass is Only 3.8% of people get a probability above 0.5. Most of the red bars, people who do have diabetes, sit left of the line.
Notebook lines 263–269 Q4: one chunk for the training set, one for the test setcode together
test_preds |> conf_mat(truth = diabetes, estimate = .pred_class) |> autoplot(type = "heatmap"): truth is the real column, estimate our class; autoplot() draws the table as a heatmap. R puts the 0 class first, so its top-left cell is the true negatives.
Read it Of 874 people with diabetes in the test set, the model flags 136 and misses 738.
Notebook line 273 how would you evaluate whether this model is good or bad?discuss
The accuracy trap, again Predict “no diabetes” for everyone: accuracy 0.862. Our model: 0.867. Barely better than doing nothing.
What our model really does Very good at leaving the healthy alone (0.981). When it flags someone it’s right about half the time (0.562). It finds only 16% of people with diabetes.
Notebook lines 275–280 discuss with your table partnersdiscuss
The model decides who gets invited for a free diabetes blood test. Which is worse, a false positive or a false negative?
Screening: miss nobody A false negative is a person with diabetes who is never tested. A false positive costs one blood test. We want high recall, and accept lower precision.
Treatment: be sure If a flag meant starting medication straight away, a false positive treats a healthy person. Then precision matters more.
“Good” depends on what the predictions are used for. The threshold is how we choose.
Notebook line 288 what's wrong with 0.8? what happens if α goes up, or down?discuss
The trade-off Lower α: more people flagged, recall up, precision down. Raise it: the reverse.
And 0.8? Hardly anyone gets a probability that high: recall close to 0.
Notebook line 293 Q7: precision, recall and F1 on the training set for 7 thresholdscode together
alpha_list <- c(0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7) class_metrics <- metric_set(precision, recall, f_meas) class_metrics <- partial(class_metrics, event_level = "second") ev_metrics <- map_dfr( alpha_list, ~ augment(model, new_data = train) |> mutate(.pred = as.factor(if_else(.pred_1 > .x, "1", "0"))) |> class_metrics(truth = diabetes, estimate = .pred) |> mutate(alpha = .x), .id = NULL)
metric_set(rsq, rmse) last week. f_meas is F1.partial() sets it once.alpha_list; stack the results..x.Part III · Q8–Q10 · lines 300–320
Syntax you need
# Q8: ev_metrics is long: one row per alpha and metric
ev_metrics |>
ggplot(aes(alpha, .estimate, colour = .metric)) +
geom_line() + geom_point()
# Q9: one column per metric, so each can be an axis
ev_metrics |>
pivot_wider(id_cols = alpha, names_from = .metric,
values_from = .estimate) |>
ggplot(aes(x = recall, y = precision)) + geom_point()
# Q10: the alpha in the row with the largest F1
best_alpha <- ev_metrics |>
filter(.metric == "f_meas") |>
slice_max(.estimate) |> # keep the top row
pull(alpha) # take out the number
# then Q7's mutate() with best_alpha on test,
# and f_meas(..., event_level = "second")Stuck?
pivot_wider(): precision, recall, f_measevent_level = "second" is missingev_metrics
ev_metrics |>
mutate(.metric = if_else(.metric == "f_meas", "F1-Score",
str_to_title(.metric))) |>
ggplot(aes(alpha, .estimate, colour = .metric)) +
geom_line(linetype = "dashed") +
geom_point() +
scale_colour_uchicago() +
labs(x = "Probability threshold",
y = "Evaluation metric estimate",
colour = "Evaluation metric")Q7’s table, then the Q8 plot of it.
Q7’s result: one row per threshold and metric, 21 in all. Shown side by side here: as α rises, precision rises and recall falls.
# A tibble: 7 × 4
alpha precision recall f_meas
<dbl> <dbl> <dbl> <dbl>
1 0.1 0.276 0.85 0.416
2 0.2 0.363 0.644 0.464
3 0.3 0.43 0.445 0.438
4 0.4 0.493 0.282 0.359
5 0.5 0.532 0.155 0.24
6 0.6 0.577 0.076 0.134
7 0.7 0.555 0.025 0.048
mutate() only tidies the legend labels (f_meas → F1-Score, precision → Precision). Recall falls off a cliff as α rises; F1 peaks at α = 0.2.
Step 1 is shown with pivot_wider() to fit the slide.
ev_metrics |>
filter(.metric %in% c("precision", "recall")) |>
pivot_wider(id_cols = alpha, names_from = .metric,
values_from = .estimate) |>
ggplot(aes(x = recall, y = precision)) +
geom_point() +
geom_text(aes(label = paste0("α = ", alpha)),
position = position_nudge(x = -0.045)) +
geom_line(linetype = "dashed")
best_alpha <- ev_metrics |>
filter(.metric == "f_meas") |>
slice_max(.estimate) |>
pull(alpha)
model |>
augment(new_data = test) |>
mutate(prediction = if_else(.pred_1 > best_alpha, 1, 0),
prediction = as.factor(prediction)) |>
f_meas(truth = diabetes, estimate = prediction,
event_level = "second")Q9 reshapes and plots; Q10 picks the best α on the training set and tests it.
Keep two metrics, then pivot_wider(): one row per α, one column per metric.
# A tibble: 7 × 3
alpha precision recall
<dbl> <dbl> <dbl>
1 0.1 0.276 0.85
2 0.2 0.363 0.644
3 0.3 0.43 0.445
4 0.4 0.493 0.282
5 0.5 0.532 0.155
6 0.6 0.577 0.076
7 0.7 0.555 0.025
Each point is a threshold. Lower α moves right: more of the sick found (recall), less sure about each flag (precision).
filter() keeps the F1 rows, slice_max() the best one, pull() takes out its α.
[1] 0.2
Same cut-off rule as Q3, now at α = 0.2, on the test set: F1 0.462, against 0.464 on the training set. No overfitting.
# A tibble: 1 × 3
.metric .estimator .estimate
<chr> <chr> <dbl>
1 f_meas binary 0.462
F1 one number for precision and recall
Same model, better use Nothing was refitted. Moving α from 0.5 to 0.2 almost doubles F1 and raises recall from 0.15 to 0.64 (training set).
No overfitting Training and test F1 are almost identical. A model with 20 coefficients on 19,000 rows has little room to memorise.
Still not great F1 under 0.5: the features overlap a lot between the two groups. A better threshold can’t fix that; better features or models might.
exp() turns it into an odds ratio. For classification the outcome must be a factorQuestion: you’re designing the screening programme. Which threshold, and how many false alarms per person found would you accept?
How was today’s lab? Scan the code, or open forms.gle/i3uobCAaGmkzf1ns7. It takes 30 seconds and changes what next week looks like.
MY202 · Lab 4 · Classification