StratumReading Regression Diagnostics: R-squared, VIF and Cook's Distance
A regression's coefficients are only as trustworthy as its diagnostics. Here's how to read the handful of numbers and plots that tell you whether to believe the model.
It's easy to fit a regression and read off the coefficients. The harder, more valuable habit is checking whether the model is sound. Four diagnostics carry most of the load.
R-squared (and adjusted)
R-squared is the share of the response's variation the model explains. Higher looks better, but it always rises when you add predictors — even useless ones. Adjusted R-squared penalizes that, so it's the one to compare models with. And a high R-squared never rescues a model whose assumptions are violated.
VIF: multicollinearity
When predictors are themselves correlated, the model can't tell their effects apart — coefficients become unstable and hard to interpret. The variance inflation factor measures this. A common rule of thumb: VIF above ~5 is worth a look, above ~10 is a problem. The fix is usually to drop or combine the redundant predictors.
Leverage and Cook's distance: influential points
Leverage flags observations with unusual predictor values; Cook's distance measures how much the whole fit would change if you dropped a point. A single high-Cook's-distance observation can swing your coefficients, so it deserves a look — is it an error, or a real but extreme case?
Residual plots
Finally, plot the residuals. A random, patternless cloud is good. A curve means the relationship isn't linear; a fan shape means the variance isn't constant; a drifting Q-Q plot of the residuals means non-normal errors. The plots catch what the summary numbers can't.
Put it into practice with How to Run a Linear Regression.
Frequently asked questions
What is a good R-squared value?
It depends on the field — physical processes can hit 0.9+, while human-behavior data may be useful at 0.3. Compare models with adjusted R-squared, and never trust R-squared alone if the diagnostics fail.
What does VIF tell me?
The variance inflation factor measures multicollinearity — how much a predictor overlaps with the others. Above ~5 warrants a look; above ~10 usually means you should drop or combine predictors.
What is Cook's distance used for?
It measures how much the regression would change if a single observation were removed — a way to find influential points that may be dominating your coefficients.
Why do I need to look at residual plots?
Because the summary numbers can look fine while assumptions fail. Residual plots reveal non-linearity, non-constant variance, and non-normal errors that R-squared hides.