Least squares regression line
The least squares line y-hat = b0 + b1 x is the straight line that makes the sum of squared vertical gaps to the points as small as possible. The slope is b1 = Sxy / Sxx and the intercept is b0 = mean of y - b1 x mean of x.
Last updated
Analogy
Pull a stretched string through a cloud of points so the total squared miss is as small as it can be.
Example
Same data: Sxy = 6, Sxx = 10Slope = 6 / 10 = 0.6Intercept = 4 - 0.6 x 3 = 2.2y-hat = 2.2 + 0.6xAt x = 6: 2.2 + 3.6 = 5.8
In practice
Used to predict one variable from another, such as sales from advertising spend.
Practice
Flashcards
- Formula for the least squares slope?b1 = Sxy / Sxx.
- Formula for the least squares intercept?Mean of y minus slope times mean of x.
Interview questions
- For the points (1,2), (2,4), (3,5), (4,4), (5,5), find the least squares line and R-squared.
- Means 3 and 4
- Slope Sxy / Sxx = 6 / 10 = 0.6
- Intercept = 4 - 1.8 = 2.2
- R-squared = 0.60
- Residuals sum to zero
Questions people ask
Why does the line minimise squared gaps?
Squaring stops positive and negative gaps from cancelling, and it punishes big misses more. It also gives clean formulas for the slope and intercept. Other criteria exist, but least squares is the standard and has well-understood properties that make later tests and intervals possible.
Why is extrapolating beyond the data risky?
The line only describes the range you observed. Outside it the relationship may curve, flatten or break. Predicting a 40-hour study week from data on 0 to 10 hours could give an impossible score. Stay inside the data range unless you have strong outside reasons to trust the pattern continues.