Quick summary
Summarize this blog with AI
Introduction
VIF means variance inflation factor. It describes the effect of predictor collinearity on coefficient uncertainty in a specified model; it is not an automatic feature-removal rule. This lesson compares ordinary VIFs for one-degree-of-freedom terms with generalized VIFs for a factor, and separates high collinearity from exact aliasing.
Tested with R 4.5.1 and car 3.1.5. Run the code blocks in order. Install the named packages before starting; inputs are supplied below.
Compute VIFs for numeric predictors
Use one complete-case analysis population. For an ordinary unweighted linear model with an intercept, VIF for predictor j equals 1/(1−Rj²), where Rj² comes from regressing that predictor on the other predictors.
d <- data.frame(x1 = 1:12)
d$x2 <- d$x1 + rep(c(1, -1, 0), 4)
d$group <- factor(rep(c("A", "B", "C"), 4))
d$y <- 2 + d$x1 + c(1,-2,2,-1,0,1,-2,2,0,1,-1,2)
model <- lm(y ~ x1 + x2, data = d, na.action = na.fail)
v <- car::vif(model)
print(round(v, 5)) # x1 18.12855; x2 18.12855
r2 <- summary(lm(x1 ~ x2, data = d))$r.squared
stopifnot(isTRUE(all.equal(unname(v["x1"]), 1 / (1 - r2))))
This is the auxiliary regression R-squared, not the R-squared of y on all predictors. Under the usual model assumptions, collinearity inflates uncertainty; it does not by itself introduce bias into ordinary least-squares estimates.
Read a GVIF matrix without treating it as a vector
A three-level factor uses two model columns. car::vif returns a matrix when a term has multiple degrees of freedom.
factor_model <- lm(y ~ x1 + group, data = d, na.action = na.fail)
gv <- car::vif(factor_model)
print(round(gv, 6))
# GVIF Df GVIF^(1/(2*Df))
# x1 1.059259 1 1.029203
# group 1.059259 2 1.014497
stopifnot(is.matrix(gv), gv["group", "Df"] == 2)
stopifnot(isTRUE(all.equal(gv[, "GVIF^(1/(2*Df))"],
gv[, "GVIF"] ^ (1 / (2 * gv[, "Df"])))))
The adjusted column allows comparison of terms with different dimensions; it is not on the raw VIF scale. Even for a one-degree-of-freedom term it is sqrt(VIF). Do not apply a raw cutoff of 5 to this adjusted column or select names from an entire matrix as if it were a named vector. The car documentation explains GVIF and its degrees-of-freedom adjustment.
Find exact redundancy before interpreting VIF
If x_copy is exactly twice x1, the design is rank deficient. A coefficient is aliased, and an ordinary finite VIF summary is not available.
d$x_copy <- 2 * d$x1
singular <- lm(y ~ x1 + x_copy + x2, data = d)
X <- model.matrix(singular)
stopifnot(qr(X)$rank < ncol(X), anyNA(coef(singular)))
print(alias(singular)$Complete)
stopifnot(inherits(tryCatch(car::vif(singular), error = identity), "error"))
# Remove only the deliberately duplicated measurement in this example.
repaired <- lm(y ~ x1 + x2, data = d)
stopifnot(all(is.finite(car::vif(repaired))))
The duplicate-column fix restores estimability but does not remove the genuine high correlation between x1 and x2. Pairwise correlations alone can miss redundancy involving several columns.
Choose a response based on the analysis goal
Rescaling a predictor changes coefficient units but does not remove ordinary linear collinearity. Verify that directly.
d$x1_scaled <- as.numeric(scale(d$x1))
scaled <- car::vif(lm(y ~ x1_scaled + x2, data = d))
stopifnot(isTRUE(all.equal(unname(scaled), unname(v))))
bad <- d; bad$x1[1] <- NA_real_
stopifnot(inherits(tryCatch(lm(y ~ x1 + x2, data = bad, na.action = na.fail),
error = identity), "error"))
Investigate redundant measurement, sampling coverage and the precision required for the coefficient of interest. Removing a confounder can change the scientific question. For prediction, evaluate a justified regularized alternative on held-out data; a lower VIF is not proof of better predictions. Interactions, no-intercept models and generalized models require the documented method-specific interpretation, beyond this additive linear-model example.
