Type I, II, and III Sums of Squares

On an unbalanced design, the same two-way ANOVA gives three different answers. The only question is which one your software computed — and whether it told you.

Follow along: the diamonds dataset from ggplot2 — 53,940 diamonds with their price, cut, and color. It is a public download, byte-identical to the file the diamond-paradox flagship uses: diamonds.csv. We chose it because cut × color is severely unbalanced — and that is exactly what makes the three conventions disagree.
Requires Stratum 1.3, where the Two-Way ANOVA gains the Sums of Squares control.

Run a two-way ANOVA on price by cut and color, and a colleague running the same model in a different program can hand you a different F-statistic for the same effect. Nobody made a mistake. They simply picked a different sum-of-squares convention — and on unbalanced data those conventions genuinely disagree. The number that ranges from twenty-six billion down to nine, depending only on which type you asked for, is not a rounding artifact. It is the whole story.

Why the imbalance is the whole story

Build the analysis once: Analyze ▸ ANOVA ▸ Two-Way, Response = price, Factor A = cut, Factor B = color, Interaction on. Every figure below is a state of this single analysis; only the Sums of Squares control changes.

Turn on Cell Means and look at the N column. The cell counts run from 119 diamonds (Fair cut, color J) up to 4,884 (Ideal cut, color G) — a 41:1 spread. That imbalance is the source of everything that follows. When every cell holds the same number of observations, the factors are statistically independent and the sum-of-squares question never arises. When cell counts differ this much, cut and color become entangled, and there is no longer one obvious way to split the variance between them.

Cell and marginal means table in Stratum, with the N column running from 119 to 4,884 across the cut by color cells — The unbalanced design, made visible: the N column swings from 119 (Fair · J) to 4,884 (Ideal · G). No imbalance, no disagreement.

Type I: sequential, and order-dependent

Set the control to Type I (sequential). Type I adds the effects to the model one at a time and credits each with whatever variance is left after the ones before it. The first factor entered gets everything it can claim; the second gets only what the first did not already absorb; the interaction gets the remainder.

That means the answer depends on the order you list the factors. Enter cut first and it claims SS 11.04 B, leaving color with 25.51 B (F 279.4). Enter color first instead and the two trade: color claims 26.85 B and cut falls to 9.70 B. Same data, same model, same diamonds — a different F-test purely because you changed which factor was entered first. Type I is the right tool when the order is meaningful, and a trap when it is arbitrary.

Type I sums of squares in Stratum, cut entered first: cut 11.04 billion, color 25.51 billion — Type I with cut entered first: cut claims 11.04 B, color is left with 25.51 B (F 279.4).

Type I sums of squares in Stratum, color entered first: color 26.85 billion, cut 9.70 billion — Swap the order — color entered first: color rises to 26.85 B and cut falls to 9.70 B. The same effect, a different number, purely from which factor went in first.

Type III: every effect adjusted for all the others

Switch back to Type III (partial) — Stratum's default, and the default in SPSS, JMP, and car::Anova. Type III asks a different question of each effect: how much variance does this factor explain that no other term in the model, including the interaction, can account for? Every effect is adjusted for every other, so no factor gets a head start from being listed first. Re-swap the factors and the numbers do not move — order-independence is the point.

The header band on the report reads “Type III (partial) sums of squares,” and the effect tests come back:

Effect	df	F	p-value
cut	4	144.4	< .0001
color	6	103.6	< .0001
cut × color	24	4.53	< .0001

Type III partial sums of squares effect-tests table in Stratum, with a Partial eta-squared column and the section header naming the convention — Type III (partial) effect tests, with a Partial η² column and a header that names the convention in force — so you always know which sum of squares you are reading.

The headline swing

Here is where the conventions part company. Under Type I, color carries a sum of squares of 26.85 B or 25.51 B depending on order; under Type II it is fixed at 25.51 B. Under Type III it drops to 9.46 B — barely a third. Why? color is heavily entangled with the cut × color interaction, and Type III makes every main effect earn its variance after the interaction is already in the model. cut, the less-confounded factor, moves far less: 11.04 → 9.70 → 8.79 B across Type I-first, Type II, and Type III, with F falling gently from 181.4 to 159.4 to 144.4. And the interaction row never moves at all — it is always tested last and fully adjusted, so every convention agrees on it: 1.65 B, F 4.53, p ≈ 1×10⁻¹².

Type II: the middle ground

Type II sits between the two. It adjusts each main effect for the other main effect but not for the interaction. For a two-factor model that produces a neat identity: Type II equals “Type I entered last.” That is why color's Type II sum of squares (25.51 B) matches its Type I value when color is listed second, and why cut's Type II figure (9.70 B) matches its Type I-last value too. Type II is often the better test when there is no meaningful interaction; Type III is the safer default when there might be one.

Stratum computes all three. Switch the Sums of Squares control to Type II (partial) and the report re-runs with the header renamed: color lands at 25.51 B and cut at 9.70 B — the middle ground, exactly where the “entered-last” identity predicts.

Type II partial sums of squares in Stratum: color 25.51 billion, cut 9.70 billion, matching the Type I entered-last values — Type II (partial): each main effect adjusted for the other but not for the interaction — color 25.51 B, cut 9.70 B, matching “Type I entered last.” Stratum names the convention in the header, as it does for every type.

Balance makes the question moot

To prove the disagreement is a property of the imbalance and nothing else, draw an equal n = 119 from every cut × color cell — 4,165 rows — and refit. Now all three types return the same number for each effect: cut SS = 6.89×10⁸, color SS = 1.93×10⁹, identical under Type I (either order), Type II, and Type III. Balance is exactly what collapses them. The entire Type I/II/III question is a symptom of unequal cell counts, not of the factors themselves.

The honest lesson

One caveat worth stating plainly, because a sophisticated reader will check: with N = 53,940 every effect is significant at p < .0001 under all three conventions. The split shows up in the sums of squares and the F-statistics, not in the reject-or-keep verdict. On a strongly powered set like this, the conventions disagree about effect magnitude; on a modestly sized sample, those same gaps are what decide significance. So the real takeaway is not “the p-value flips” — here it doesn't — but that unbalanced data does not have one sum of squares. It has three. Know which one you picked. Stratum names its convention in the report header for exactly that reason.

Download Stratum