Fit a line

Click anywhere to add a point, drag a point to move it. The line is the one that makes the squared vertical gaps (the residuals) as small as possible. Watch r, the correlation: near 1 or −1 the points hug the line, near 0 they do not, and its sign says which way the line tilts.

residuals squares mean lines

What's going on

For a line ŷ = a + bx, each point has a residual y − ŷ, the vertical distance to the line. Least squares picks a and b to minimise the sum of the squared residuals. The answer has a clean form: the slope is b = S_xy / S_xx, where S_xy = Σ(x − x̄)(y − ȳ) and S_xx = Σ(x − x̄)², and the line always passes through the point of means (x̄, ȳ). Tick "mean lines" to see it.

The correlation coefficient r = S_xy / √(S_xx S_yy) is always between −1 and 1. It measures how close the points are to some straight line, not how steep that line is: stretch the y-axis and r does not change. r² is the fraction of the variation in y that the line accounts for.

Now press "Add an outlier". One point far from the others can swing the slope and wreck r, because squaring makes big residuals count enormously. That is a feature when the outlier is real and a bug when it is a typo, which is why the first thing a statistician does is look at the picture. Remember too that a strong r does not say x causes y: ice-cream sales and drownings are correlated because both follow the weather.

Sheets: Standard deviation and the bell curve and Elo ratings and regression to the mean.