When both groups improve and the total gets worse
Tonight's idea
A success rate is a fraction: successes over attempts. Combine two groups and the overall rate is not the average of the two rates but the weighted average, with the bigger group pulling harder. In the language of the mediant, adding tops and bottoms gives \(\frac{a + c}{b + d}\), between \(\frac{a}{b}\) and \(\frac{c}{d}\) and nearer the one with the larger bottom.
That makes something strange possible. Hospital A can beat Hospital B on survival for serious patients and for mild patients, yet lose overall. A's total is dragged towards its serious-patient rate because it treats mostly serious cases; B's towards its mild-patient rate because it treats mostly mild ones. Both totals are correct, but comparing them compares the patients who walked in, not the hospitals. This is Simpson's paradox.
A reversal needs a lurking variable that is tied to both the group you are comparing and the outcome. Here it is how sick the patient is.
The famous case is Berkeley's 1973 graduate admissions. Men were admitted at a higher rate overall, but department by department the gap mostly vanished or reversed: women had applied more to the departments that admitted few of anyone.
Watch one: two hospitals
Five hundred patients each. Rates are survivors over patients.
| Serious | Mild | All patients | |
|---|---|---|---|
| Hospital A | 320/400 = 80% | 99/100 = 99% | 419/500 = 83.8% |
| Hospital B | 35/50 = 70% | 432/450 = 96% | 467/500 = 93.4% |
A wins both columns and loses the total. Check with weights: A's overall rate is \(\tfrac{400}{500} \times 80\% + \tfrac{100}{500} \times 99\% = 64\% + 19.8\% = 83.8\%\). B's is \(\tfrac{50}{500} \times 70\% + \tfrac{450}{500} \times 96\% = 7\% + 86.4\% = 93.4\%\).
Your turn
- Two batters. Season 1: Ana 60 hits from 200, Ben 10 from 40. Season 2: Ana 20 from 40, Ben 90 from 200. Find all four season rates and both overall rates. Who had the better rate each season? Who had the better overall rate?
- Hospital B is asked a fair question: what would its overall rate be if it had treated A's mix of patients, 400 serious and 100 mild, at its own rates of 70% and 96%? Compute it and compare with A's 83.8%.
- The mediant fact says a hospital's overall rate must lie between its two group rates. Suppose A had rates 90% and 99% and B had 50% and 70%. Could any mix of patients make B's total beat A's? Explain in one sentence. What must be true of the two hospitals' rate ranges for the paradox to be possible?
- A friend says: "I do not know whether my case is serious or mild, so B's 93.4% is the number that matters to me." Is she right? Say precisely which number she should look at, and why the 93.4% is not about her.
Stretch: the smallest paradox
Build Simpson's paradox with tiny numbers. Find two fractions for A and two for B, every top and bottom a whole number from 1 to 5, with A ahead in both groups but behind once tops and bottoms are added. Why can the total not reverse if the two groups are the same size in both hospitals?
Answers
- Season 1: Ana \(60/200 = 30\%\), Ben \(10/40 = 25\%\). Season 2: Ana \(20/40 = 50\%\), Ben \(90/200 = 45\%\). Ana wins both seasons. Overall: Ana \(80/240 = 33.3\%\), Ben \(100/240 = 41.7\%\). Ben wins overall, because most of his attempts were in the easy season and most of Ana's in the hard one.
- \(0.70 \times 400 + 0.96 \times 100 = 280 + 96 = 376\) survivors of 500: 75.2%. On the same patients, B's 75.2% loses to A's 83.8%. This is called standardising, and it is how hospitals are actually compared.
- No. A's total must be at least 90% and B's at most 70%, whatever the mix. For a reversal the ranges must overlap: the weaker hospital's best group rate must exceed the stronger hospital's worst.
- She is wrong. She is a serious case or a mild case, even if she does not yet know which, and A has the better rate for both kinds. The 93.4% is high because of who walks into B, not because of what B does to them. The numbers for her are the group rates (A ahead in each) or the standardised rate from question 2.
Stretch. One answer: A scores \(1/1\) and \(1/3\); B scores \(4/5\) and \(1/4\). A leads both groups (100% against 80%, 33% against 25%), but overall A has \(2/4 = 50\%\) and B has \(5/9 \approx 56\%\). (Allowing a group with no successes, \(1/1, 1/3\) against \(3/4, 0/1\) works with nothing above 4.) If both hospitals had the same group sizes, the overall rates would be weighted averages with the same weights, and a hospital ahead in each group would be ahead in the average too. The paradox needs different mixes.