Thom Volker

@thomvolker.bsky.social

377 Followers 341 Following

PhD Candidate in Statistics, Utrecht University Creates fake data for a living. thomvolker.github.io

Making a symbolic visit to Copenhagen, a bipartisan delegation of U.S. lawmakers — including senior members of the House and Senate — tried to reassure leaders of Denmark and Greenland, and their increasingly anxious citizens, that most Americans do not support President Donald Trump’s plan to annex or buy Greenland, let alone the prospect of military action against a fellow NATO ally.
sim <- function(n, s) { set.seed(1) x <- runif(n, 0, 20) y <- 0.3 * x + sin(x) + rnorm(n, 0, s) data.frame(s = var(0.3*x + sin(x))/var(y), x = x, y = y) } data <- purrr::map_dfr(1:40, \(i) sim(1000, i/20)) library(ggplot2) library(gganimate) ggplot(data, aes(x = x, y = y)) + geom_point() + transition_time(-s) + ease_aes('linear') + theme_minimal() + labs(title = bquote(R^2~'= {round(-frame_time, 2)}')) sim(1000, 2) |> lm(y~x, data = _) |> summary()
library(mice, warn.conflicts = FALSE) imp <- mice(nhanes, m = 1) mice <- function(..., print = FALSE) { mice::mice(..., print = print) } imp <- mice(nhanes, m = 1)
Figure showing the pooled prediction intervals for 5 unseen test cases, of which case 1 and 4 have some missing data. Code to reproduce the figure is included in the alt-text of the previous post.
Image of R code. To reproduce: library(ggplot2) library(dplyr) library(mice, warn.conflicts = FALSE) imp <- mice(nhanes, m = 5, maxit = 5, seed = 1, ignore = rep(c(FALSE, TRUE), c(20, 5)), print = FALSE) impdats <- complete(imp, "all") train <- lapply(impdats, function(dat) subset(dat, !imp$ignore)) test <- lapply(impdats, function(dat) subset(dat, imp$ignore)) fits <- lapply(train, function(dat) lm(age ~ bmi + hyp + chl, data = dat)) preds <- predict_mi(object = fits, newdata = test, pool = TRUE, interval = "prediction") preds preds %>% as.data.frame() %>% mutate(case = 1:nrow(preds), y = test[[1]]$age) %>% ggplot(aes(x = fit, y = case, col = rowSums(is.na(nhanes[imp$ignore,]))>0)) + geom_point() + geom_errorbar(aes(xmin = lwr, xmax = upr)) + theme_minimal() + scale_color_manual(values = mice::mdc(1:2), labels = c("observed", "missing")) + theme(legend.title = element_blank(), legend.position = "bottom") + labs(x = "prediction", title = "Pooled prediction intervals")
#Image with the following code: params <- sapply( # stack predictions in array: 1st dim observations predlist, # 2nd dim terms function(pred) { # 3rd dim pred or var array( # 4th dim imputations c(as.matrix(pred$fit)[,seq_len(nterms)], pred$se.fit), dim = c(n, nterms, 2) ) }, simplify = "array" )
Three figures with x = 1:10, y = f(x), first on original scale (y = x), second on log scale (y = log(x)), third on logarithmic scale with large constant (y = log(x + 10000)).
Small example of the speed-up when using the `mirai` backend to execute purrr functions in parallel.
My balcony with coffee and a paper
Predictor matrix with columns and rows x, y and z. All diagonal elements are set to zero initially (you cannot impute a variable with itself), and all off-diagonal elements are set to one. If, in mice, I don't want to impute x using z, I can set that particular cell in the matrix to zero.