Quick summary
Summarize this blog with AI
Introduction
A confidence interval describes uncertainty about a specified population parameter. Start by naming that parameter: a mean, a difference of means, or a proportion. The examples below separate these targets and then contrast a frequentist interval with a Bayesian credible interval.
Tested with R 4.5.1. Run the code blocks in order. Install the named packages before starting; inputs are supplied below.
What 95% confidence means
If we repeatedly sampled under the model assumptions and constructed intervals the same way, about 95% would contain the fixed parameter. An interval from one observed sample either contains it or does not. The confidence level is not a posterior probability for that particular interval, nor a statement that it contains 95% of observations. Selection bias, dependence and measurement errors are not repaired by a narrow interval.
A t interval for one population mean
These six fictional independent measurements have sample mean 12. For a normal population with unknown variance, the t interval uses the sample standard deviation and n − 1 degrees of freedom. With nonnormal data it is an approximation whose adequacy depends on sample size and distribution. The manual calculation below agrees with t.test().
x <- c(8, 10, 11, 13, 14, 16)
fit <- t.test(x, conf.level = 0.95)
ci <- as.numeric(fit$conf.int)
manual <- mean(x) + qt(c(0.025, 0.975), length(x) - 1) * sd(x) / sqrt(length(x))
print(round(c(mean = mean(x), lower = ci[1], upper = ci[2]), 3))
stopifnot(isTRUE(all.equal(ci, manual)))
The mean is 12; the interval endpoints are approximately 8.958 and 15.042. This estimates the population mean, not the range of a future individual measurement.
A difference of means has a direction
For independent groups, the default two-sample t.test uses Welch’s unequal-variance method. Supplying new scores first makes the parameter new minus traditional. A paired design instead requires matched observations and paired = TRUE; do not choose pairing merely because vector lengths agree.
traditional <- c(88, 92, 75, 85, 78)
new <- c(95, 89, 88, 94, 90)
difference <- t.test(new, traditional)
print(round(c(estimate = mean(new) - mean(traditional),
lower = difference$conf.int[1], upper = difference$conf.int[2]), 3))
reverse <- t.test(traditional, new)
stopifnot(isTRUE(all.equal(as.numeric(reverse$conf.int),
-rev(as.numeric(difference$conf.int)))))
The observed mean difference is 7.6 score points, with a 95% interval of approximately [−0.988, 16.188]. Report its interval and units; the observational comparison alone does not establish a teaching-method effect.
A proportion and a difference of proportions
The following score interval estimates one campaign’s success probability. The two-group call estimates new minus old, not two separate success intervals. We explicitly disable continuity correction so the calculation is reproducible. These approximations assume independent binomial trials; clustered customers or repeated visits need a design-aware analysis.
single <- prop.test(150, 1000, correct = FALSE)
comparison <- prop.test(c(150, 120), c(1000, 1000), correct = FALSE)
print(round(rbind(single_probability = single$conf.int,
new_minus_old = comparison$conf.int), 4))
stopifnot(all(single$conf.int >= 0 & single$conf.int <= 1),
comparison$conf.int[1] > 0, comparison$conf.int[2] > 0)
# Exact binomial interval remains defined at the boundary.
zero_success <- binom.test(0, 10)$conf.int
stopifnot(zero_success[1] == 0, zero_success[2] > 0)
The estimated difference is 0.03, or 3 percentage points. Its interval is approximately [0.0001, 0.0599] and barely excludes zero. That does not establish a causal improvement or guarantee a practically worthwhile effect.
Make missingness and invalid input explicit
A wrapper can enforce a data contract; it does not create a distribution-free interval. This version rejects missing/nonfinite values and requires at least two nonconstant observations. If you choose to omit missing rows, report how many and why that subset answers your question.
mean_interval <- function(x, level = 0.95) {
if (!is.numeric(x) || !is.null(dim(x)) || length(x) < 2L ||
any(!is.finite(x)) || !is.numeric(level) || length(level) != 1L ||
!is.finite(level) || level <= 0 || level >= 1)
stop("Need a finite numeric vector and a level strictly between 0 and 1")
if (!is.finite(sd(x)) || sd(x) == 0) stop("Need positive finite sample SD")
as.numeric(t.test(x, conf.level = level)$conf.int)
}
fails <- function(expr) inherits(tryCatch(expr, error = identity), "error")
stopifnot(isTRUE(all.equal(mean_interval(x), ci)),
fails(mean_interval(numeric())), fails(mean_interval(c(1, NA))),
fails(mean_interval(c(2, 2))), fails(mean_interval(c(1, Inf))),
fails(mean_interval(matrix(1:4, 2))), fails(mean_interval(x, 1)))
A separate Bayesian example: a binomial probability
For the same 150 successes in 1,000 trials, choose a Beta(1,1) prior explicitly. Multiplying its density by the binomial likelihood gives a Beta(151,851) posterior. Its central 95% credible interval does have 95% posterior probability under this specified prior and likelihood. qbeta() supplies the quantiles; this is not a Bayesian bootstrap.
posterior <- qbeta(c(0.025, 0.975), shape1 = 151, shape2 = 851)
print(round(posterior, 4))
stopifnot(abs(diff(pbeta(posterior, 151, 851)) - 0.95) < 1e-12)
The interval is approximately [0.1292, 0.1735]. Prior sensitivity matters especially with little data. A Bayesian interval and a frequentist interval answer different probability questions even when their endpoints are close.
Choose a method that estimates your target
A percentile bootstrap can approximate uncertainty by resampling independent observations, but small, biased or dependent samples need more care; resampling does not fix the sampling design. A Wilcoxon signed-rank interval concerns a pseudomedian and relies on symmetry for its usual location interpretation, as the R documentation explains. It is not a universal distribution-free mean interval. For survey estimates, retain sampling weights and design; for a future month’s sales, use an appropriate forecasting or prediction interval rather than a confidence interval for a constant historical mean.
