A p-value is the probability of getting results at least as extreme as yours if the null hypothesis were true. A small p-value means your data would be surprising in a world where nothing is going on. That is all it means, and most misreadings of research come from asking it to mean more.
A coin example
You flip a coin 100 times and get 60 heads. Is the coin biased?
Start by assuming it is fair (H₀: p = 0.5). A fair coin gives 50 heads on average, with a standard deviation of √(100 × 0.5 × 0.5) = 5. Sixty heads is 10 above the average, or 2 standard deviations, so z = 2.0.
The chance that a fair coin lands 2 or more standard deviations away from 50, in either direction, is about 0.046. That is the p-value. A fair coin would give a result this lopsided about 1 time in 22.
Run the coin example in the one-proportion z-test calculator
The calculator also shows the exact binomial p-value, 0.057. The normal approximation and the exact answer fall on opposite sides of 0.05. Borderline results like this are a good reason not to treat 0.05 as a magic line.
What a p-value is not
| Wrong reading | Why it’s wrong |
|---|---|
| “There is a 4.6% chance the coin is fair.” | The p-value assumes the coin is fair and asks how surprising the data are. It can’t turn that around and give you the probability the assumption is true. |
| “There is a 95.4% chance the coin is biased.” | Same mistake, other direction. |
| “p = 0.001 means a bigger effect than p = 0.04.” | Not necessarily. A tiny effect in a huge sample can give a tiny p-value. Look at the effect size. |
| “p = 0.20 proves there is no effect.” | It only says the data are not unusual under H₀. A small study can easily miss a real effect. |
| “p < 0.05 means the result matters.” | Statistical significance and practical importance are different things. A diet that cuts 0.2 kg in a year can be significant with enough people. |
Where does 0.05 come from?
Mostly from habit. Ronald Fisher suggested it in the 1920s as a convenient point for deciding whether a result was worth a second look, not as a rule for what is true. Some fields use 0.01. Particle physics uses about 0.0000003 (five standard deviations) before claiming a discovery. Pick α before you look at your data, and report the exact p-value rather than just “p < 0.05”.
One-tailed and two-tailed p-values
The coin p-value of 0.046 is two-tailed: it counts lopsided results in both directions, too many heads or too many tails. If you had set out to test only whether the coin favours heads, the one-tailed p-value would be half that, 0.023. Decide which you need before you collect the data.
Getting a p-value from a test statistic
If you already have a z, t, χ² or F value from a textbook or a paper, you can turn it into a p-value directly:
Or bracket it by hand using the t table or chi-square table.
What to report alongside p
A p-value on its own says little about what you found. Report the effect itself (a difference, a correlation, an odds ratio), a confidence interval for it and the sample size. “The new layout raised sign-ups from 4.1% to 5.0% (95% CI for the difference 0.2 to 1.6 points, p = 0.01)” tells the reader far more than “p = 0.01”.
Check yourself
A study reports p = 0.03. A colleague says “so there’s a 97% chance the treatment works”. What would you tell them?
That’s not what p means. It means that if the treatment did nothing, results this strong would appear about 3% of the time. The chance the treatment works depends on other things too, such as how plausible it was before the study.
Two studies test the same drug. One finds p = 0.04, the other p = 0.06. Do they disagree?
Not really. Their results are almost the same, and both sit near the 0.05 line. Compare their effect sizes and confidence intervals, which probably overlap a lot.