How to Choose a Statistical Test

The right test depends on three things: what kind of outcome you measured, how many groups or variables you are comparing, and whether the same people appear more than once. Answer the questions below to get a recommendation, or use the tables further down.

Not sure which test?

Answer a few questions and we’ll point you to the right test.

Step 1: what kind of outcome do you have?

  • A measurement on a scale, such as weight, time, score or income. Means and standard deviations make sense.
  • A category, such as yes/no, pass/fail or blood type. You count how many fall into each group.
  • A rank or rating on a short scale, such as a 1 to 5 satisfaction score. These are often treated as measurements if there are enough points on the scale, but rank-based tests are safer when the scale is short or the data are lopsided.

Step 2: find your situation in the table

Comparing groups on a measurement

SituationUsual testIf data are skewed or have outliers
One group against a known valueOne-sample t-testWilcoxon signed-rank
Two separate groupsIndependent t-test (Welch)Mann-Whitney U
Same people measured twicePaired t-testWilcoxon signed-rank
Three or more separate groupsOne-way ANOVAKruskal-Wallis
Two grouping factorsTwo-way ANOVAConsider transforming the data

Counts and categories

SituationTest
One proportion against a known valueOne-proportion z-test
Two proportions, such as an A/B testTwo-proportion z-test or A/B test calculator
Two categorical variables in a tableChi-square test of independence
A 2 × 2 table with small countsFisher’s exact test
Counts in categories against expected sharesChi-square goodness of fit

Relationships between variables

SituationTest
Two measurements, straight-line relationshipPearson correlation
Two measurements or ranks, any steady upward or downward trendSpearman correlation
Predict a measurement from one variableSimple linear regression
Predict a measurement from several variablesMultiple regression
Predict a yes/no outcomeLogistic regression

Step 3: paired or independent?

This is the question people get wrong most often. If the same people or items appear in more than one group, or each value in one group is matched to one value in the other, the data are paired. Using an independent test on paired data throws away information, and the result can flip from significant to not significant. The paired t-test guide has an example where exactly that happens.

A note on normality

“Use a non-parametric test if the data aren’t normal” is often over-applied. The t-test and ANOVA hold up well with moderate skew once you have around 30 per group. Worry more about clear outliers and very small samples. With 8 values per group and one wild value, a rank-based test is the safer choice.

Check yourself

You record whether 200 website visitors clicked a button, split between two page designs. Which test?

The outcome is yes/no and there are two independent groups, so a two-proportion z-test, or the chi-square test on the 2 × 2 table. They give the same p-value.

Twelve patients rate their pain from 0 to 10 before and after a treatment. Which test?

Same patients twice, so a paired test. With a short rating scale and only 12 people, the Wilcoxon signed-rank test is a sensible choice. A paired t-test is also commonly used.