StudyVisuallyStart studying
Statistics (university) · Part 08

Correlation and regression

Measure how two variables move together and fit a line to predict one from the other.

Last updated

  1. Correlation coefficientThe correlation r measures the strength and direction of a straight-line link between two numerical variables. It runs from -1 (perfect downhill line) to +1 (perfect uphill line). Zero means no straight-line link.
  2. Least squares regression lineThe least squares line y-hat = b0 + b1 x is the straight line that makes the sum of squared vertical gaps to the points as small as possible. The slope is b1 = Sxy / Sxx and the intercept is b0 = mean of y - b1 x mean of x.
  3. Slope and interceptThe slope is the predicted change in y for each one-unit increase in x. The intercept is the predicted y when x is 0, which may be meaningless if x = 0 is far from your data.
  4. ResidualsA residual is observed y minus predicted y. Positive residuals mean the point is above the line. Least squares residuals always add up to zero. Plotting them against x reveals problems with the model.
  5. R-squaredR-squared is the share of the variation in y explained by the model: 1 - (residual sum of squares / total sum of squares). For a straight line with one predictor it equals r squared.
  6. Regression assumptionsLinear regression assumes a straight-line relationship, independent errors, errors with constant spread, and roughly normal errors. Remember it as LINE: Linear, Independent, Normal, Equal spread.
  7. Inference for the slopeTo test whether x really predicts y, test H0: true slope = 0. The statistic is t = b1 / SE(b1) with n - 2 degrees of freedom, where SE(b1) = s / square root of Sxx and s = square root of (residual SS / (n - 2)).
  8. Multiple regressionMultiple regression uses several predictors: y-hat = b0 + b1 x1 + b2 x2 + ... Each coefficient is the predicted change in y for a one-unit rise in that predictor while the others are held fixed.
  9. Correlation and causationCorrelation means two variables move together. It does not prove one causes the other. A third factor (a confounder) may drive both, or the link may be chance.