Assignment 1 · Problems 1(c)–2 · submitted 11 Sep 2023

Wheat seeds PCA explorer

Seventy X-ray-scanned kernels from each of three wheat varieties (Kama, Rosa and Canadian), each described by seven shape measurements. Principal components turn seven correlated measurements into a few uncorrelated summaries, and here two of them are enough to see the three varieties pull apart.

kernels: 210; kernels
measurements: 7; measurements
varieties: 3; varieties

The data

Seven measurements, one label

The eighth column is the variety code 1–3. It is a category, not a measurement, so it is left out of the PCA and only used to colour the plots (Problem 2(a)).

X1Areamm²
X2Perimetermm
X3Compactness4πA/P²
X4Kernel lengthmm
X5Kernel widthmm
X6Asymmetrycoef.
X7Groove lengthmm
X8Variety (1 Kama, 2 Rosa, 3 Canadian)

Problem 2

Principal components, three ways

PCA finds orthogonal directions $\gamma_1,\gamma_2,\dots$ of decreasing variance $\lambda_1\ge\lambda_2\ge\cdots$ . The cumulative proportion $\psi_k=\sum_{j\le k}\lambda_j/\sum_j\lambda_j$ tells how much of the total variance the first k components keep. Switch the variant to see what changes when the analysis is done on the covariance matrix (as asked), on the correlation matrix, or in the mixed way it was submitted.

Which analysis?

Every chart below is recomputed in your browser (Jacobi eigen-decomposition, 210 × 7).

What the question asked: PCA of S

On the raw scale, area (X1) and perimeter (X2) have by far the largest variances, so PC1 alone explains 82.9% of the total variance and is essentially a size component. This is the usual argument for standardising first, but it is a different analysis from the one requested.

Scree plot and cumulative variance ψ

Eigenvalues of S · bars: share of variance, line: ψ_k

PC	1	2	3	4	5	6	7
λ	10.8	2.13	0.0736	0.0129	0.00275	0.00157	3.0e-5
share	82.9%	16.4%	0.6%	0.1%	0.0%	0.0%	0.0%
ψ	82.9%	99.3%	99.9%	100.0%	100.0%	100.0%	100.0%

PC1 vs PC2 scores by variety

centred X × eigenvectors of S

Kama (variety 1)
Rosa (variety 2)
Canadian (variety 3)

x axisy axisBiplot arrows

Correlation circle: PC1 and PC2

ρ(Xᵢ, PCⱼ) = γᵢⱼ √λⱼ / √sᵢᵢ

Loadings γ (weights of each variable)

first three principal components; bar length = |γᵢⱼ|

Variable	γ·1	γ·2	γ·3
X1 Area	0.884	0.101	-0.265
X2 Perimeter	0.395	0.056	0.283
X3 Compactness	0.004	-0.003	-0.059
X4 Kernel length	0.129	0.031	0.400
X5 Kernel width	0.111	0.002	-0.319
X6 Asymmetry	-0.128	0.989	-0.064
X7 Groove length	0.129	0.082	0.762

The coefficients do not depend on the kernel i; the scores PCᵢ = γᵀ(Xᵢ − μ) do. Signs of eigenvectors are arbitrary (here the largest entry is made positive; the as-submitted view uses the 2023 signs).

Problem 1(d)

$S=\Gamma\Lambda\Gamma^{\top}$ for the wheat data

The covariance-PCA eigenvalues are exactly the entries of $\Lambda$ below. Computed here with a Jacobi eigen-solver in TypeScript and checked against R's eigen(cov(X)) to 10 decimal places in the test suite.

Sample covariance S (7 × 7)

Cell shade ∝ |sᵢⱼ|. Area (X1) has variance 8.47 while compactness (X3) has 5.6e-4: four orders of magnitude apart.

	X1	X2	X3	X4	X5	X6	X7
X1	8.466	3.778	0.042	1.225	1.067	-1.004	1.235
X2	3.778	1.706	0.016	0.563	0.466	-0.427	0.572
X3	0.042	0.016	5.6e-4	0.004	0.007	-0.012	0.003
X4	1.225	0.563	0.004	0.196	0.144	-0.114	0.203
X5	1.067	0.466	0.007	0.144	0.143	-0.147	0.139
X6	-1.004	-0.427	-0.012	-0.114	-0.147	2.261	-0.008
X7	1.235	0.572	0.003	0.203	0.139	-0.008	0.242

Λ: eigenvalues of S

λ110.7933
λ22.1295
λ30.0736
λ40.0129
λ52.748e-3
λ61.570e-3
λ72.966e-5

Γ: orthonormal eigenvectors (columns)

	γ1	γ2	γ3	γ4	γ5	γ6	γ7
X1	0.884	0.101	-0.265	0.199	0.137	-0.281	-0.025
X2	0.395	0.056	0.283	-0.579	-0.575	0.302	0.066
X3	0.004	-0.003	-0.059	0.058	0.053	0.045	0.994
X4	0.129	0.031	0.400	-0.436	0.787	0.113	0.001
X5	0.111	0.002	-0.319	0.234	0.145	0.896	-0.082
X6	-0.128	0.989	-0.064	-0.025	0.002	-0.003	0.001
X7	0.129	0.082	0.762	0.613	-0.088	0.110	0.009

Check: max |ΓΛΓᵀ − S| = 1.8e-14 (R's round(S - S.sample) in the 2023 answer printed a matrix of zeros).

Reading the results

With the covariance matrix the first component is dominated by area and perimeter, the two variables with the largest raw variances. After standardising, three components keep about 98.7% of the variance (ψ₃), matching the submitted “keep 3 PCs” conclusion. Along PC1 (size) Rosa kernels are the largest and Canadian the smallest, with Kama in between; PC2 sets Canadian further apart because its kernels are the least compact and the most asymmetric.