Chi-Square Test

The chi-square test works on counts, not measurements. It checks whether the counts you observed are close to the counts you would expect if nothing interesting were going on. There are two common versions: the test of independence and the goodness-of-fit test.

Test of independence: a worked example

A clinic tracks 400 people through one flu season. Half were vaccinated.

Got fluNo fluTotal
Vaccinated12188200
Not vaccinated30170200
Total42358400

The question: is getting flu related to vaccination, or could a split like 12 against 30 happen by chance?

Step 1: expected counts. If vaccination made no difference, the 42 flu cases would be shared evenly between the two equal groups: 21 each. In general, expected count = row total × column total ÷ grand total. For the top-left cell, 200 × 42 ÷ 400 = 21.

Step 2: compare observed with expected. For each cell, work out (observed − expected)² ÷ expected, then add them up:

(12 − 21)²/21 + (188 − 179)²/179 + (30 − 21)²/21 + (170 − 179)²/179 = 8.62

Step 3: find the p-value. df = (rows − 1) × (columns − 1) = 1. With χ² = 8.62 and df = 1, p = 0.003. The critical value at α = 0.05 is 3.84 (see the chi-square table), and 8.62 is well past it.

Run the flu example in the chi-square calculator

Flu was less common among vaccinated people: 6% against 15%. The chi-square test says that difference is unlikely to be chance.

What the test doesn’t tell you

χ² tells you whether there is an association, not how strong it is. For that, look at the percentages, the risk ratio (6% ÷ 15% = 0.4, so 60% fewer cases) or an effect size such as phi or Cramér’s V. Here φ = 0.15, which looks small, yet the practical effect is large. Effect sizes for tables are hard to judge without the percentages next to them.

It also doesn’t show cause. This was an observational study. If the vaccinated people were also more careful in other ways, that could explain part of the gap.

Goodness of fit

The goodness-of-fit version compares one set of counts with a set of expected proportions. Is a die fair? Do customers choose four menu items equally often? Does the mix of blood types in a sample match the national figures? The df is the number of categories − 1. The goodness-of-fit calculator handles any expected proportions.

When the chi-square test isn’t safe

  • Small expected counts. The usual rule: all expected counts at least 1, and no more than 20% of them below 5. For a 2 × 2 table with small counts, use Fisher’s exact test, which the chi-square calculator runs automatically for 2 × 2 tables.
  • Percentages instead of counts. The test must be run on raw counts. 30% of 20 and 30% of 2,000 carry very different amounts of evidence.
  • The same person counted twice. Each person or item must fall into exactly one cell.

Check yourself

A 3 × 4 table has a grand total of 600. Row 1 totals 200 and column 2 totals 150. What is the expected count in row 1, column 2?

200 × 150 ÷ 600 = 50.

What is the df for a 3 × 4 table?

(3 − 1) × (4 − 1) = 6.