gamlss2

Calibration Plots for Binary Responses

Description

Compute and plot calibration curves for models with 0/1 responses. The function can handle one or several fitted gamlss2 models and compares their calibration in a single plot.

Usage

calibration(..., newdata = NULL,
  y = NULL, parameter = NULL, breaks = seq(0, 1, by = 0.1),
  minn = 20, main = "Calibration plot",
  xlab = "Predicted probability",
  ylab = "Observed proportion", plot = TRUE,
  add_loess = TRUE, smooth_n = 200,
  col = NULL, lty = NULL, legend = TRUE, pos = "topleft",
  xlim = NULL, ylim = NULL)

Arguments

… One or several fitted gamlss2 model objects to be assessed.

newdata Optional data frame for out-of-sample calibration.

y Optional numeric or factor vector with the binary response.

parameter Character, parameter name(s) to predict. The default NULL selects all parameters in the family’s names order.

breaks Numeric vector of break points used to construct bins for the predicted probabilities.

minn Integer, the minimum number of observations required in each bin.

main Character, main title for the plot.

xlab Character, label for the x-axis.

ylab Character, label for the y-axis.

plot Logical, should a calibration plot be produced?

add_loess Logical, should a loess smooth be added for each model?

smooth_n Integer, number of evaluation points for the loess curve.

col Colors used for the models.

lty Line types used for the loess curves.

legend Logical, should a legend be added when more than one model is supplied?

pos Character, legend position passed to legend.

xlim The x limits of the plot.

ylim The y limits of the plot.

Details

For each fitted model, predicted probabilities are obtained via predict.gamlss2 using type = “parameter” and the selected parameter (all family parameters when NULL). If the family provides a probabilities() function, it converts these predictions to probabilities; otherwise, the predicted parameters are used directly. If the result is a matrix or data frame, only its first column is used, after any conversion. Thus, without probabilities(), multiple predicted parameters are reduced to the first parameter column. The values used for calibration must represent the probability of y = 1.

Predicted probabilities are grouped into bins defined by breaks. Within each bin, the mean predicted probability, the observed proportion of 1s, and the number of observations are computed. Bins with fewer than minn observations are dropped.

Value

Invisibly returns a data frame with the columns interval, probs, y, n, and model. For a single model, the model column is dropped.

References

Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW (2019). “Calibration: The Achilles Heel of Predictive Analytics”. BMC Medicine, 17, 230. doi:10.1186/s12916-019-1466-7

Examples

library("gamlss2")

n <- 400
x1 <- runif(n, -3, 3)
x2 <- runif(n, -3, 3)
eta <- sin(x1) + 0.5 * x2
p <- plogis(eta)
y <- rbinom(n, size = 1, prob = p)
d <- data.frame(y = y, x1 = x1, x2 = x2)

m1 <- gamlss2(y ~ x1, family = BI, data = d)
m2 <- gamlss2(y ~ s(x1), family = BI, data = d)
m3 <- gamlss2(y ~ s(x1) + x2, family = BI, data = d)

calibration(m1, m2, m3)
head(calibration(m1, m2, m3, plot = FALSE))