Quick summary
Summarize this blog with AI
Introduction
Feature scaling changes a numeric variable’s scale. Converting inches to centimeters is a unit conversion; min–max scaling and z-score standardization instead use estimated data parameters. Neither makes skewed data normal nor guarantees a better model. Fit preprocessing on training data and reuse those same parameters for validation, testing and future observations.
Tested with R 4.5.1. Run the code blocks in order. Install the named packages before starting; inputs are supplied below.
Fit a min–max transform on training data
For a training minimum a and positive range b, transform x to (x − a)/b. The training extremes become 0 and 1; future values outside that range legitimately map outside[0,1]. This implementation omits missing values only while fitting, preserves them while applying, and rejects infinite values, constant training features and unsupported numeric ranges.
numeric_vector <- function(x) {
if (!is.numeric(x) || !is.null(dim(x)) || any(is.infinite(x)))
stop("Need a numeric vector without infinite values")
}
fit_minmax <- function(x) {
numeric_vector(x)
observed <- x[!is.na(x)]
if (length(observed) < 2L) stop("Need at least two observed training values")
limits <- range(observed)
span <- diff(limits)
if (!is.finite(span) || span <= 0) stop("Need a positive finite training range")
list(min = limits[1], span = span)
}
apply_minmax <- function(x, fitted) {
numeric_vector(x)
out <- (x - fitted$min) / fitted$span
if (any(is.infinite(out))) stop("Transformed values exceed the supported numeric range")
out
}
train <- c(10, 20, 30, 40, 50, NA)
test <- c(5, 25, 60, NA)
mm <- fit_minmax(train)
train_scaled <- apply_minmax(train, mm)
test_scaled <- apply_minmax(test, mm)
print(train_scaled)
print(test_scaled)
stopifnot(isTRUE(all.equal(train_scaled, c(0, .25, .5, .75, 1, NA))),
isTRUE(all.equal(test_scaled, c(-.125, .375, 1.25, NA))))
Test values become −0.125,0.375,1.25,NA. Refitting on test values would give them a different coordinate system and leak evaluation information. Clipping them to[0,1] changes the transform and should be a separate, justified policy.
Reuse training means and sample standard deviations
For centered columns, scale() divides by sample SD and returns a matrix with fitted parameters stored as attributes. Select the intended numeric features explicitly. Scaling IDs, categories or all columns indiscriminately changes their meaning.
training <- data.frame(age = c(20, 30, 40, 50), amount = c(10, 20, 40, 50))
future <- data.frame(age = c(25, 60), amount = c(15, 70))
standardized <- scale(training)
centers <- attr(standardized, "scaled:center")
spreads <- attr(standardized, "scaled:scale")
stopifnot(all(is.finite(spreads) & spreads > 0),
identical(names(training), names(future)))
future_z <- scale(future, center = centers, scale = spreads)
print(round(future_z, 3))
stopifnot(all(abs(colMeans(standardized)) < 1e-12),
all(abs(apply(standardized, 2, sd) - 1) < 1e-12),
is.matrix(future_z))
restored <- sweep(sweep(standardized, 2, spreads, "*"), 2, centers, "+")
stopifnot(isTRUE(all.equal(unname(restored), unname(as.matrix(training)), check.attributes = FALSE)))
Only the training columns are centered at 0 with sample SD 1. Future data need not have either property. Normality is not required to compute z-scores, and an affine transformation does not remove skewness or outliers.
Decimal scaling needs a numeric rule, not a string length
For the less common decimal-scaling convention used here, choose the smallest nonnegative integer j such that the training maximum absolute value divided by 10^j is strictly below 1. Already subunit values use j = 0; an all-zero feature also uses 0. The explicit finite-divisor check rejects values for which 10^j would overflow.
fit_decimal <- function(x) {
numeric_vector(x)
observed <- x[!is.na(x)]
if (!length(observed)) stop("Need observed training values")
m <- max(abs(observed))
j <- if (m < 1) 0 else floor(log10(m)) + 1
divisor <- 10^j
if (!is.finite(divisor)) stop("Decimal divisor exceeds supported numeric range")
# Account for rounding near a power of ten.
while (divisor > 1 && m / (divisor / 10) < 1) divisor <- divisor / 10
if (m / divisor >= 1) divisor <- divisor * 10
if (!is.finite(divisor)) stop("Decimal divisor exceeds supported numeric range")
divisor
}
decimal_divisor <- fit_decimal(c(-120, 30, 100))
print(c(-120, 30, 100) / decimal_divisor)
stopifnot(decimal_divisor == 1000, fit_decimal(c(0, 0)) == 1,
fit_decimal(c(-0.5, 0.2)) == 1, fit_decimal(c(10, -10)) == 100,
fit_decimal(c(1e16 - 2, 0)) == 1e16)
The scaled values are −0.12,0.03,0.10. Save the fitted divisor for future values. This convention scales magnitude without centering; it does not estimate variance or produce a probability distribution.
Test the edge policies and preserve the original data
Constant or all-missing features need an explicit modeling decision, such as excluding them after review; division by zero is not a useful scaling policy. Missing values remain missing until a separately fitted imputation step is justified. Within cross-validation, fit both imputation and scaling independently in each training fold.
fails <- function(expr) inherits(tryCatch(expr, error = identity), "error")
stopifnot(fails(fit_minmax(numeric())), fails(fit_minmax(c(NA_real_, NA_real_))),
fails(fit_minmax(c(3, 3))), fails(fit_minmax(c(1, Inf))),
fails(fit_minmax("10")), fails(fit_minmax(matrix(1:4, 2))),
fails(fit_minmax(c(-1e308, 1e308))), fails(fit_decimal(c(1e308, 2))),
identical(apply_minmax(numeric(), mm), numeric()),
is.na(apply_minmax(NA_real_, mm)))
stopifnot(identical(train, c(10, 20, 30, 40, 50, NA)))
