Effect Size

What it is

A hypothesis test tells you whether an effect is probably real. It says nothing about whether that effect is large enough to care about. Effect size fills the gap: it's a standardized number for the magnitude of a difference or relationship, and — unlike a p-value — it doesn't grow just because you collected more data.

That independence from sample size is the whole point. Two studies can both report "p < 0.001" while one found a difference you'd never notice and the other found one that changes everything. The effect size is what separates them.

The formula

For the difference between two group means, the standard measure is Cohen's d — the gap between the means expressed in pooled standard-deviation units:

d = ( x̄₁ − x̄₂ ) / s_pooled

where s_pooled is the pooled standard deviation of the two groups (a sample-size-weighted blend of each group's spread). A d of 1.0 means the two means sit a full standard deviation apart.

Effect size isn't only for mean differences. For the strength of an association, the correlation coefficient r and its square R² (the share of variance explained) are themselves effect sizes. For an ANOVA across several groups, η² (eta-squared) plays the same role — the proportion of total variance attributable to group membership.

These cutoffs are conventions, not laws — Cohen offered them as a fallback when no field-specific benchmark exists. In a high-stakes domain a "small" d can be hugely important; in a noisy one a "large" d may be routine. Context decides.

How to read it

The key insight is that a p-value conflates effect size and sample size. The same true difference produces a smaller p-value as n rises, so with enough data even a microscopic effect crosses the significance threshold. That makes "significant" a poor proxy for "important."

Effect size breaks them apart. Because it's measured in standard-deviation (or variance-explained) units rather than tail probabilities, it stays put no matter how many rows you add. So the discipline is simple: always report the effect size alongside the p-value. The p-value tells you whether the effect is probably non-zero; the effect size tells you whether anyone should care.

What effect size is not

• Not a significance test. It measures magnitude, not whether the magnitude is distinguishable from zero — that's the p-value's job.
• Not inflated by sample size. Adding rows tightens its confidence interval but doesn't push the estimate up.
• Not automatically "important." The small/medium/large labels are defaults; importance depends on the field and the stakes.
• Not a single number. d, r, R², and η² each fit a different design — pick the one that matches your test.

A p-value is half the story — it says an effect is probably there. The effect size is the other half: how big it is. Wrap that in a confidence interval for precision and you've told the whole story, not just the significant-or-not headline.