Fewer dimensions: PCA and MCA

MY202 · Lab 7

Jonas Weinert

Department of Methodology  |  London School of Economics and Political Science

12 November 2026

Lab overview

Part Topic Notebook Mode Time
Setup, and why fewer dimensions lines 25–53 together 7 min
I PCA: the school report, then 24 ability tests lines 55–214 concept → run together → your turn, twice 40 min
II MCA: what do you watch, then 19 moral questions lines 216–291 concept → run together → your turn 26 min
Wrap-up: PCA or MCA? 7 min

First unsupervised lab: no outcome to predict, so no accuracy to chase. All the code is given. Your job today is to run it and read the pictures: every “your turn” is a set of reflection questions from the notebook.

Why fewer dimensions?

WhyI · PCAII · MCAWrap-up

Notebook lines 38–53 the callout "The curse of dimensionality"concept

Data become sparse Same 30 customers, one more variable, and the neighbourhood empties.

Distance stops meaning much With 2 variables the farthest customer is 32× as far as the nearest; with 1,000, 1.1×. Bad news for kNN.

And models overfit and slow down So: squeeze many variables into a few that keep most of the structure.

PCA: continuous data

Part I

301 pupils · 24 ability tests · how many things do they really measure?

The school report: two marks, one number

WhyI · PCAII · MCAWrap-up

You may write one number per pupil on the report. Which?

PC1: where the cloud is longest The line along which pupils differ most. PC2 is at a right angle to it.

A score is a shadow Drop each pupil onto the line. Roughly “maths + physics”: an overall mark.

Nothing was lost yet, only turned PC1 holds 94% of the spread. Drop PC2 and you lose 6%: two columns became one.

Four subjects: reading the loadings

WhyI · PCAII · MCAWrap-up

A loading is the weight of a subject in a component blue = positive, red = negative; this is the plot you will read in the lab

PC1 weights all four subjects about equally. Give it a name.

PC1 · a level · 62% Same sign, similar size: a high score means good marks across the board.

PC2 · a contrast · 31% Opposite signs: stronger in sciences than in humanities.

How to read any component Which bars are long? Same sign or opposite? What do the long ones share? Then you name it.

Class discussion: two tests that agree

WhyI · PCAII · MCAWrap-up

Notebook line 66 the class discussion before the data30 seconds with your neighbour

Two IQ tests are strongly correlated. What would you expect PCA to do with them?

ADelete one of the two tests, since it adds nothing
BGive both a similar weight on the same component
CPut each on its own component, so they stay apart
DNothing: PCA only works on uncorrelated variables
B. Maths and physics again: tests that move together share a direction, so one component describes both. PCA keeps every variable and combines them; it never drops a column. Without correlation there would be nothing to compress.

Meet the data: 24 ability tests

WhyI · PCAII · MCAWrap-up

Notebook lines 68–80 "Load and prepare data": select the tests, then splitrun together

301pupils aged 11–16 in two Chicago schools, tested in 1939
24tests, t01 to t24: the columns we compress

What is kept case (an id), school (Pasteur or Grant-White) and the 24 tests. Age, grade and sex are dropped, and two repeat tests (t25, t26).

Tests What they ask Examples
t01–t04 Spatial cubes, form board, lozenges
t05–t09 Verbal word meaning, sentence completion
t10–t13 Speed addition, counting dots, code
t14–t19 Memory word, number and figure recognition
t20–t24 Reasoning and maths deduction, series, arithmetic

The split 240 pupils to train, 61 to test, as in Labs 3–5. PCA is fitted on the training set only. The test set is not used again today.

Explore: do the tests overlap?

WhyI · PCAII · MCAWrap-up

Notebook lines 82–97 "Explore relationships": the correlation heatmap and its reflectionrun together, then one minute in pairs

What can you say about this heatmap? What should PC1 look like?

Almost no blue Nearly every pair is positively correlated (0.26 on average): good at one test, good at most. Expect PC1 to be an overall level, like the school report.

Darker blocks The verbal tests t05–t09 stand out (0.68 among themselves). The speed tests t10–t13 form a weaker block. Expect the next components to be contrasts between blocks.

Apply PCA: read the recipe

WhyI · PCAII · MCAWrap-up

Notebook lines 99–118 "Apply PCA": the recipe, then bake()run together

pca_recipe <- recipe(school ~ ., data = train_data) |> update_role(case, new_role = "ID") |> step_zv(all_numeric_predictors()) |> step_normalize(all_numeric_predictors()) |> step_pca(all_numeric_predictors(), num_comp = 5, keep_original_cols = TRUE) |> prep()

  1. Same units first. Otherwise the test with the biggest numbers would be PC1.
  2. The rotation. Finds the 24 directions, longest first.
  3. Keep five as new columns PC1–PC5. We check this choice next.
  4. Do it now, on the training set. bake() then adds the scores.
# A tibble: 240 × 4
   case    PC1   PC2    PC3
  <int>  <dbl> <dbl>  <dbl>
1   201 -3.11  1.40  -0.414
2   202  0.384 0.186  0.377
3   203 -0.202 2.63   1.45 
4   204 -0.822 0.523 -1.62 
# ℹ 236 more rows

Read it Pupil 201 has PC1 = -3.1: their shadow on the longest direction, well below the average of 0.

school ~ . is not a model Nothing is predicted. It only says which columns count as “predictors”.

Your turn A: how many components?

Part I · lines 120–174 · in pairs

  1. Lines 122–151: run both chunks. One bar per component: the share of variance, then the running total.
  2. Lines 155–159: why might num_comp = 5 be reasonable? How much do 2, 3 and 5 components explain together? Why is plotting only 2 a risk?
  3. Lines 163–168: run the scatter of PC1 against PC2.
  4. Lines 172–174: clusters, outliers, a trend? What is the correlation between PC1 and PC2, and why? What might the plot hide?

How to read the output

eigenvalues            # one row per component
#  PC  percent  cumulative
#   1    0.31        0.31   <- PC1 alone
#   2    0.10        0.40   <- PC1 + PC2 together

cor(pca_data$PC1, pca_data$PC2)   # try it
  • percent: this component’s share of the total spread
  • cumulative: what you keep if you stop here
  • Look for where the bars flatten: later components add little each

Stuck?

  • pca_fit not found: run lines 101–118 first
  • School report: after turning the page, was any tilt left in the cloud?

Answers A: variance explained

Why five? PC1 stands out with 31%. PC2–PC5 add 10%, 8%, 7%, 4%; after that each adds under 4%. No sharp elbow: a judgement call.

2, 3 and 5 components 41%, 48% and 60% of the variance. 24 columns became 5 and kept three fifths of the spread.

Why 2 is risky Two fit on a screen, and show 41%. The other 59% is invisible. Check this number before reading any 2-D plot.

Answers A: the first two components

One cloud No clusters and no clear outliers: pupils differ by degree, not by type.

The correlation is exactly 0 By construction: the page was turned until no tilt was left. The missing trend says nothing about the pupils.

What it hides 59% of the variance. Structure in PC3–PC5 cannot be seen here.

And one thing it shows Colour by school: Grant-White sits to the upper right. PCA never saw the school.

Your turn B: what is each component?

Part I · lines 176–214 · in pairs

  1. Lines 178–209: run the chunk. Three panels, one per component; one bar per test; longest bars at the top.
  2. Line 213: on PC1, what sign do the loadings have, and how similar are their sizes?
  3. Line 214: which tests sit at opposite ends of PC2 and PC3? What do the tests at each end have in common?
  4. Give each component a name of two or three words.

The school-report recipe

  • 1 · Which bars are long? Short bars barely count
  • 2 · Same sign or opposite? One colour: a level. Two colours: a contrast, blue against red
  • 3 · What do the long ones share? Use the table
t01–t04 Spatial: shapes, cubes, lozenges
t05–t09 Verbal: words, sentences, general information
t10–t13 Speed: addition, code, counting dots
t14–t19 Memory: recognising words, numbers, figures
t20–t24 Reasoning and maths

Stuck?

  • Blue or red on its own means nothing: flip every sign of a component and it is the same component

Answers B: naming the components

PC1 · overall ability · 31% All 24 positive, from 0.09 to 0.28. The heatmap’s “no blue”. Psychologists call it g.

PC2 · words against memory, speed · 10% Blue: sentence completion, general information, word meaning. Red: object–number, addition, number recognition.

PC3 · shapes against speed · 8% Blue: number recognition, lozenges, cubes. Red: addition, counting dots, code. The names are ours; the data only give weights.

MCA: categorical data

Part II

94,278 people · 19 moral questions · yes or no

Why not PCA?

WhyI · PCAII · MCAWrap-up

Notebook lines 230–233 the class discussion at the start of Part IIone minute with your neighbour

Why can’t you run PCA on categorical variables? What might “distance” mean instead?

PCA needs arithmetic Means, variances, a longest direction. A test score of 60 is 10 above 50. What is TV − newspaper? Or the mean of “yes” and “no”?

Count instead of measure With categories we can still count who gave which answers together. That table of co-occurrences is all MCA uses.

Distance = chosen by different people Two answers are close when mostly the same people give them. Close means “go together”, not “similar in value”. The notebook calls it chi-square distance.

What do you watch? A map of answers

WhyI · PCAII · MCAWrap-up
Football F1 Bake Off Strictly
A y y n n
B y y n n
C y y n y
D y n n n
E n n y y
F n n y y
G n y y y
H n n y n
I y y y y
J n n n n
K y y n n
L n n y y

Twelve people, four yes/no questions 8 possible answers

Which answers tend to come from the same people?

Close = same people football_y sits by f1_y: the sports fans.

Opposite = rarely together Each _y faces its own _n. Dim 1: sport against shows.

Load the data: every answer becomes y or n

WhyI · PCAII · MCAWrap-up

Notebook lines 235–252 "Load and prepare ethical norms data"run together

norms <- read_csv("data/wvs-wave-7-ethical-norms.csv") |> mutate(across(everything(), ~ as.factor(if_else(.x > median(.x), "y", "n")))) colnames(norms) <- values

  1. Every column: 19 World Values Survey questions, “is this ever justifiable?”, from 1 (never) to 10 (always).
  2. Above that question’s median: “y”. Otherwise “n”. A 10-point scale becomes two categories.
  3. Readable names instead of Q177–Q195.

Share answering "y" after the median split

premarital sex49%
divorce41%
stealing property26%
terrorism23%

“y” is not “supports” For 12 of the 19 questions the median is 1, “never”. There, y means “anything but never”: terrorism_y is “did not rule it out”.

Apply MCA: one line

WhyI · PCAII · MCAWrap-up

Notebook lines 254–263 "Apply MCA" and the reflection on the supplementary variablerun together

mca <- MCA(norms, quali.sup = 1, graph = FALSE)

  1. From FactoMineR, not tidymodels: no recipe, no prep().
  2. A table of factors: 19 columns of y and n.
  3. Column 1 is supplementary: cheating_benefits is left out when the axes are built.
  4. No automatic plot. We draw our own next.

Why mark a variable as supplementary?

It is placed on the map afterwards The other 18 questions build the axes. cheating_benefits is then dropped onto the finished map without shaping it.

A test of the map If the map means something, a question it has never seen should land where we expect. A hold-out for a method with no test score.

Your turn C: read the map

Part II · lines 265–291 · in pairs

  1. Lines 267–273: run the biplot. One triangle per answer; green is the supplementary question.
  2. Line 277: which answers sit at the far ends of Dim 1 and of Dim 2? What do they have in common?
  3. Line 278: where does cheating_benefits land?
  4. Lines 280–287: run the contribution plot.
  5. Line 291: which answers contribute most to Dim 1? What do they share?

The TV-map rules

  • Close together: answers given by the same people
  • Opposite sides: each question’s _y faces its _n: read one side of an axis and you know the other
  • Name an axis by what its two ends have in common
  • Far from the middle: a rare answer
fviz_contrib(mca, choice = "var", axes = 1, top = 10)
#   choice = "var"   the answers, not the people
#   axes = 1         for Dim 1 (try axes = 2)
# red dashed line:   an equal share for every answer

Stuck?

  • Start with the right-hand side of Dim 1: are the labels _y or _n?
  • Then compare top right with bottom right

Answers C: the biplot

Dim 1 · 37% · how much is justifiable Every _y on the right, every _n on the left. Like PC1: a level.

Dim 2 · 16% · which kind Top right: violence, theft, bribes. Bottom right: divorce, premarital sex, abortion. Like PC2: a contrast, harm to others against private life.

The supplementary question fits cheating_benefits_y lands right and above the middle, by fare dodging. It never shaped the axes.

Answers C: who builds Dim 1?

The top eight are all “y” answers Stealing, political violence, social violence, terrorism, bribes, suicide, tax cheating, violence against a spouse. All above the red line (2.8%, an equal share).

What they share Harm and dishonesty, and rarity: few people give these answers, so they separate respondents most.

Contributions are MCA’s loadings PCA: which variables have long bars on a component. MCA: which answers contribute most to a dimension. Same question, same reading.

PCA or MCA?

WhyI · PCAII · MCAWrap-up
PCA MCA
Data Numbers with meaningful spacing: test scores, incomes Categories: yes/no, news source, age group
Built from Correlations between variables Counts of answers given together
Close means Similar values Chosen by the same people
What you read Loadings: one bar per variable The map and contributions: one point per answer
Today step_pca() in a recipe: 24 tests → 5 components MCA() from FactoMineR: 19 questions → 2 dimensions

The same story twice First axis: a level (overall ability; how much is justifiable). Second: a contrast (verbal against speed; harm against private life).

Both are linear summaries Good for seeing structure and for fewer columns in a later model. The naming is always yours.

Summary

WhyI · PCAII · MCAWrap-up
1Why reduce: with many variables, data are sparse, distances stop telling rows apart, and models overfit. Real variables overlap, so a few new axes can keep most of the structure
2PCA is the school report: turn the cloud so the first axis runs along its longest direction. Each component has a share of variance (how much it keeps) and loadings (what it is made of). Same sign: a level. Opposite signs: a contrast. In code: step_normalize(), step_pca(), tidy(matrix = "eigenvalues"), tidy(matrix = "loadings")
3MCA is the TV map: one point per answer, close when the same people give them, each y opposite its n. Read the ends of each axis and the contributions. In code: MCA(), fviz_mca_biplot(), fviz_contrib()

Question: two components show 41% of the test data. Before you trust a pattern in the PC1–PC2 scatter, what would you check?

Further reading

30 seconds of feedback

QR code linking to the feedback survey

How was today’s lab? Scan the code, or open forms.gle/i3uobCAaGmkzf1ns7. It takes 30 seconds and changes what next week looks like.