The central limit theorem

Choose a parent distribution and slide n. The small picture is the parent: what one draw looks like. The big picture is what happens when you take n draws, average them, and repeat 10,000 times. Each average is one count in the histogram. The dashed orange curve is the normal distribution with the parent's mean and standard deviation σ/√n. Start at n = 1 with the two-point parent, which could not look less like a bell, and slide up. Then untick "zoom to the bell" to watch the spread shrink.

Mission. Take the skewed parent. Read the observed SD at n = 4. Predict the SD at n = 16 and at n = 64, then test. Then find the smallest n at which you would be happy to call the skewed histogram "normal". Is it the same n as for the uniform parent?

Answer

Halving the SD needs four times the n: the SD at n = 16 is half the SD at n = 4, and at n = 64 it is a quarter. The readout's σ/√n predicts all three. The skewed parent needs a larger n than the uniform one before the bell fits well: its long right tail leaves the histogram lopsided at n = 4 and still visibly so at n = 10, while the uniform parent is close to normal by n = 3 or 4. The theorem says what happens as n → ∞. How fast depends on the parent.

parent n draws per average 1 zoom to the bell (±4 SE)

What's going on

Take any distribution with a mean μ and a finite standard deviation σ. Draw n values independently and average them. The average has mean μ, because averaging does not move the centre, and standard deviation σ/√n, because the variances of independent draws add: n of them give variance nσ², the average divides by n², and the square root of σ²/n is σ/√n. That is the standard error. Quadruple the sample and you halve the noise.

The surprise is the shape. The central limit theorem says the histogram of averages approaches the normal curve as n grows, whatever the parent looked like. The two-point parent is the extreme case: an average of n of those is just a count of ones divided by n, so the histogram is a binomial, and the binomial becomes a bell. The exponential parent is skewed, and the skew fades like 1/√n, which is why it takes longer.

What the theorem does not say: it does not make the parent normal (the small picture never changes), and it does not help if σ is infinite, as it is for some heavy-tailed distributions. For some of those, averages never settle down at all.

Sheet: Why averages are normal. Related demo: the Galton board, where the parent is a coin flip.