Quick summary
Summarize this blog with AI
Introduction
Use is.na() to locate missing values and anyNA() to ask whether any exist. Detecting missingness is different from deciding what to replace or exclude. This lesson keeps those decisions visible and plots the missingness indicator itself, so missing observations remain visible.
Tested with R 4.5.1 and ggplot2 4.0.0. Run the code blocks in order. Install the named packages before starting; inputs are supplied below.
Inspect a mask rather than comparing with NA
is.na() returns a logical mask with the input dimensions. For a data frame it returns a logical matrix. Numeric NaN is also considered missing by is.na(); Inf is not. The character string "NA" is an ordinary string unless an import or conversion rule explicitly treats it as missing.
dat <- data.frame(id = 1:3, a = c(1, NA, 3), b = c(4, 2, NA))
mask <- is.na(dat[c("a", "b")])
print(mask)
print(colSums(mask))
print(rowSums(mask))
stopifnot(is.matrix(mask), identical(dim(mask), c(3L, 2L)),
identical(unname(colSums(mask)), c(1, 1)), anyNA(dat),
identical(is.na(c(NA, NaN, Inf, 0)), c(TRUE, TRUE, FALSE, FALSE)),
!is.na("NA"), is.na(NA_character_))
Each measured column has one missing value; row missing counts are 0,1,1. x == NA does not produce a usable missingness test because the comparison itself is unknown.
Prevent unknown conditions from creating empty-looking rows
Logical operators retain uncertainty when the answer depends on it: TRUE & NA is NA, FALSE & NA is FALSE, and TRUE | NA is TRUE. Base row indexing with an NA mask entry produces a missing output row. Explicitly keep only known TRUE conditions.
condition <- !is.na(dat$a) & dat$b > 3
print(condition)
safe <- dat[!is.na(condition) & condition, , drop = FALSE]
also_safe <- dat[which(condition), , drop = FALSE]
print(safe)
stopifnot(identical(condition, c(TRUE, FALSE, NA)),
identical(safe, also_safe), identical(safe$id, 1L),
identical(c(TRUE & NA, FALSE & NA, TRUE | NA), c(NA, FALSE, TRUE)))
The raw condition is TRUE,FALSE,NA. Both safe filters return only row 1, with a = 1 and b = 4. This choice excludes unknown conditions; keeping them for review would be a different explicit policy.
Count denominators with every summary
Removing missing values changes the denominator. A summary with no observed values should not pretend to be zero. The helper below accepts a numeric vector and returns the count, observed count, missing count and observed mean; it returns NA when no values are observed.
describe_missing <- function(x) {
if (!is.numeric(x) || !is.null(dim(x)) || any(is.infinite(x)))
stop("Need a numeric vector without infinite values")
observed <- x[!is.na(x)]
c(rows = length(x), observed = length(observed), missing = sum(is.na(x)),
mean = if (length(observed)) mean(observed) else NA_real_)
}
print(describe_missing(dat$a))
stopifnot(describe_missing(dat$a)["mean"] == 2,
is.na(describe_missing(c(NA_real_, NaN))["mean"]),
is.na(describe_missing(numeric())["mean"]))
analysis_rows <- complete.cases(dat[c("a", "b")])
stopifnot(identical(dat$id[analysis_rows], 1L))
Columna has 3 rows,2 observations,1 missing value and observed mean 2. Requiring both a and b selects only row 1. Restrict complete.cases() to variables required by the analysis rather than discarding rows for unrelated fields.
Draw missingness at finite row and variable coordinates
A point whose y value is NA cannot be placed at a numeric y coordinate. Instead, this tile plot uses row and variable positions for every cell, with missingness represented by fill. The plot reveals a pattern, not the reason data are absent.
library(ggplot2)
missing_cells <- expand.grid(row = seq_len(nrow(mask)), variable = colnames(mask))
missing_cells$status <- factor(ifelse(as.vector(mask), "Missing", "Observed"),
levels = c("Observed", "Missing"))
p <- ggplot(missing_cells, aes(variable, factor(row), fill = status)) +
geom_tile(color = "white", linewidth = 1) +
scale_fill_manual(values = c(Observed = "#bddae5", Missing = "#ad3b29"), drop = FALSE) +
labs(title = "Missingness by row and variable", x = "Variable", y = "Row", fill = "Status") +
theme_minimal(base_size = 12)
print(p)
stopifnot(nrow(ggplot_build(p)$data[[1]]) == 6,
sum(missing_cells$status == "Missing") == 2)
The red cells are a in row 2 and b in row 3. All six measured cells are visible, including the two missing ones.
Detection does not justify imputation
Observed patterns can suggest collection problems but do not by themselves establish whether missingness is completely random, conditionally random given observed data, or related to unobserved values. Replacing unknown measurements with zero changes their meaning. A single fitted-value imputation understates uncertainty if treated as observed data; multiple-imputation inference combines analyses across completed datasets rather than reporting one completion as certain. Preserve the raw values and document the assumptions behind the chosen method. See the missing-value workflow for selective filtering and justified replacement.
empty <- dat[FALSE, , drop = FALSE]
stopifnot(identical(dim(is.na(empty)), c(0L, 3L)),
all(colSums(is.na(empty)) == 0), !anyNA(empty))
all_missing <- data.frame(a = c(NA_real_, NA_real_), b = c(NA_real_, NA_real_))
stopifnot(all(rowSums(is.na(all_missing)) == 2),
!any(complete.cases(all_missing)))