P-Value Interpreter

What does a p-value actually mean?

A p-value answers one very specific question: if there were truly no effect in the population, how likely would it be to see data as extreme as mine, just from random sampling? Formally, it is the probability of obtaining results at least as extreme as the ones you observed, assuming the null hypothesis is true. A p-value of .03 means that, in a world with no real effect, only about 3 in 100 random samples would produce a result as large as yours.

That is all it tells you. It is a statement about your data under an assumption, not a statement about your hypothesis. This distinction is where most misinterpretations begin. A p-value of .03 does not mean there is a 97% chance your hypothesis is true, and it does not mean there is a 3% chance the null hypothesis is true. Those conclusions would require Bayesian methods and prior probabilities that a standard p-value simply does not contain.

Why the 0.05 threshold?

The 0.05 cutoff is a convention, not a law of nature. R.A. Fisher proposed it in the 1920s as a convenient benchmark for flagging results worth a closer look, and it stuck. In practice this means a result with p = .048 and one with p = .052 provide almost identical evidence, even though only one gets the "significant" label. The American Statistical Association has formally warned against this kind of binary thinking — treat the p-value as a continuous measure of evidence, not a pass/fail grade.

Significant is not the same as important

With a large enough sample, even a trivially small effect will produce p < .001. With a small sample, a genuinely large effect can produce p = .20. This is why journals and thesis committees increasingly require an effect size (Cohen's d, η², r, Cramér's V) alongside every p-value: the p-value tells you whether an effect is detectable, the effect size tells you whether it matters.

Likewise, a non-significant result is not evidence that "there is no effect." It often just means your sample was too small to detect the effect that exists — a question of statistical power, which depends on sample size, effect size, and alpha level.

If you want the full story — including the five most common misinterpretations and exactly how to phrase p-values in your results chapter — read our in-depth guide: How to Interpret P-Values Without Getting It Wrong.