An unusual data point—in its predictor values, its residual, or both—may influence the regression estimates. The next step is to identify which observation has that influence. Two useful deletion diagnostics are Cook’s distance and DFFITS (difference in fits).
What’s Cook’s distance?
Cook’s distance measures the aggregate shift in all fitted values after removing one observation. The goal is to find points that have a large overall impact on the regression model.
When should I use it?
Start with Cook’s distance when you want a general check for influential observations and how much the fit changes overall. A commonly used screening rule is to inspect observations with Di > 4/n, where n is the number of observations.
What’s DFFITS?
DFFITS compares the fitted value for observation i with the value predicted at that same point after observation i is removed. It standardises that difference by an estimated error scale. Unlike Cook’s distance, its sign tells us whether including the point moves its fitted value up or down.
When should I use it?
Use DFFITS when your focus is on how much an individual fitted value changes. A common screening rule is to inspect observations whose absolute DFFITS exceeds 2√(p/n), where p is the number of model parameters, including the intercept.
Cook’s distance also uses an error-variance estimate, so it is not a measure that avoids MSE. Neither diagnostic is exclusively for small or large datasets. An observation flagged by either measure deserves a closer look—not automatic removal.
In short: Cook’s distance asks, “How much does this point change the fit overall?” DFFITS asks, “How much does this point change its own fitted value, and in which direction?”
References
The screening rules and definitions above follow Penn State’s regression diagnostics lesson, the olsrr Cook’s-distance documentation, and statsmodels’ DFFITS guidance. R provides cooks.distance() and dffits().