What is a p-value in simple terms?

A p-value is the probability of seeing a result at least as extreme as the one you got, assuming there is really no effect (the null hypothesis is true). A small p-value means your data would be surprising if nothing were going on, which is evidence that something is.

What does p < 0.05 mean?

By the common 0.05 convention, p < 0.05 means the result is called statistically significant: data this extreme would occur less than 5% of the time if the null hypothesis were true. The 0.05 cutoff is a convention, not a law of nature.

Does a small p-value prove the hypothesis is true?

No. A p-value measures how compatible the data are with the null hypothesis, not the probability that any hypothesis is true. It also says nothing about how large or important the effect is — for that you need an effect size and a confidence interval.

What Is a P-Value? Definition, Formula & How to Read It

Q: What does p < 0.05 mean?

By the common 0.05 convention, p < 0.05 means the result is called statistically significant: data this extreme would occur less than 5% of the time if the null hypothesis were true. The 0.05 cutoff is a convention, not a law of nature.

Q: Does a small p-value prove the hypothesis is true?

No. A p-value measures how compatible the data are with the null hypothesis, not the probability that any hypothesis is true. It also says nothing about how large or important the effect is — for that you need an effect size and a confidence interval.

The p-value is the probability of seeing data at least as extreme as yours if the null hypothesis were true. A small p-value means your result would be surprising under "nothing is going on" — which is evidence that something is.

What it is

Every hypothesis test starts with a null hypothesis — a baseline claim of "no effect": two groups have the same mean, two variables are unrelated, a coin is fair. The test computes a statistic from your data, then asks a single question: if the null were true, how often would random sampling alone produce a result this extreme or more? That probability is the p-value.

So a p-value of 0.03 says: "Data this extreme would show up about 3% of the time if the null hypothesis were true." Rare-under-the-null is the whole idea — the rarer your data would be in a world with no effect, the more that world looks wrong.

The formula

There's no single formula — the p-value is a tail probability of whatever test statistic you computed, read off that statistic's reference distribution. For a test statistic T with observed value t:

two-sided:  p = P( |T| ≥ |t| )   under the null
one-sided:  p = P(  T ≥  t  )   under the null

For a t-test, T follows a Student's t distribution; for a chi-square test, a χ² distribution; for a z-test, the standard normal. The mechanics differ, but every one ends in the same place: the area in the tail beyond your observed statistic.

How to read it

P-value	Conventional reading
`p < 0.01`	Strong evidence against the null
`p < 0.05`	"Statistically significant" by the usual convention
`0.05 ≤ p < 0.10`	Marginal — suggestive, not conclusive
`p ≥ 0.10`	Little evidence against the null

The 0.05 threshold (the significance level, α) is a convention chosen before you see the data — not a law of nature. A p-value of 0.049 and one of 0.051 are practically identical pieces of evidence; the cliff between them is an artifact of rounding human decisions into a binary.

A worked example. You compare a treatment and control group and get a two-sample t-test statistic of t = 2.4. Looking up the t-distribution tail, that gives p ≈ 0.02. Reading: if the treatment truly did nothing, you'd see a gap this large or larger only about 2% of the time. That's unlikely enough that most analysts would reject "the treatment does nothing." It does not tell you the treatment is large, useful, or worth the cost — only that it's probably not zero.

Four things a p-value is not

Not the probability the null is true. It's computed assuming the null is true; it can't also be the chance that assumption holds.
Not the size of the effect. With a big enough sample, a trivial difference produces a tiny p-value. Pair it with an effect size.
Not a measure of importance. "Significant" is a statistical word, not a practical one.
Not a license to stop thinking. A p-value assumes the test's conditions hold — independence, the right distribution, no p-hacking across many tests.

How Stratum reports it. Stratum computes exact p-values for its t-tests, ANOVA, correlation, chi-square, and regression coefficients, and displays them by one shared rule: any value below 0.0001 shows as <.0001 rather than a row of leading zeros. See the conventions page for the full formatting rule.

P-Value

What it is

The formula

How to read it

Four things a p-value is not

See also