Problem 20.3 Easy Free

Simpson's Paradox

Teams AA and BB play two independent soccer matches. In match 1, team AA’s average shooting accuracy exceeds team BB’s. In match 2, team AA’s average accuracy again exceeds team BB’s. Does this guarantee that team AA’s average accuracy over the two matches combined exceeds team BB’s? If so, prove it; if not, give a counter-example with explicit shot counts and accuracies.

Work it out, then compare with the answer.

SolutionShow the worked solution

No. Interpret “average accuracy” as goals divided by shots. A team’s combined accuracy is then a weighted average of its two match accuracies. Each team’s weights are the fractions of its own shots taken in each match. If team AA takes almost all its shots in the match where everyone’s accuracy is low, and team BB in the match where it is high, the combined comparison can reverse.

Write pX,ip_{X,i} for team XX’s accuracy in match ii, and wXw_X for the fraction of team XX’s total shots taken in match 11. Then

pX=wX pX,1+(1−wX) pX,2.p_X = w_X\, p_{X,1} + (1-w_X)\, p_{X,2}.

The inequalities pA,1>pB,1p_{A,1}>p_{B,1} and pA,2>pB,2p_{A,2}>p_{B,2} say nothing about pAp_A versus pBp_B once wA≠wBw_A\neq w_B.

Counter-example.

Match 1Match 2Combined
Team AA1/100=1%1/100 = 1\%1/1=100%1/1 = 100\%2/101≈1.98%2/101 \approx 1.98\%
Team BB0/1=0%0/1 = 0\%99/100=99%99/100 = 99\%99/101≈98.0%99/101 \approx 98.0\%

Team AA is more accurate in match 1 (1%>0%1\% > 0\%) and in match 2 (100%>99%100\% > 99\%). Yet over both matches AA scores 22 of 101101 shots and BB scores 9999 of 101101. So BB’s combined accuracy is far higher.

The reason is the weights: wA=100/101w_A = 100/101 while wB=1/101w_B = 1/101. Almost all of AA’s shots come in the low-accuracy match, and almost all of BB’s come in the high-accuracy match. This reversal is Simpson’s paradox.

No: e.g. A ⁣:1/100, 1/1;B ⁣:0/1, 99/100  ⇒  2101<99101.\boxed{\text{No: e.g. } A\colon 1/100,\ 1/1;\quad B\colon 0/1,\ 99/100 \;\Rightarrow\; \tfrac{2}{101} < \tfrac{99}{101}.}

Generalisation. The conclusion does hold if both teams split their shots between the matches in the same proportion, i.e. wA=wB=ww_A=w_B=w. In that case

pA−pB=w (pA,1−pB,1)+(1−w) (pA,2−pB,2)>0.p_A - p_B = w\,(p_{A,1}-p_{B,1}) + (1-w)\,(p_{A,2}-p_{B,2}) > 0.

Key idea. Pooled rates are weighted averages with group-specific weights, so a comparison that holds within every group can reverse in the aggregate.