Matthew Kay

@mjskay.com

Assoc Prof Computer Science and Communication Studies at Northwestern. Infovis, HCI. Author of tidybayes & ggdist R pkgs. he/him. 🏳️‍🌈 https://mjskay.com/ Co-director https://mucollective.northwestern.edu Co-founder https://journalovi.org

Stratified swarm dotplot with exact x positions and two groups stacked on top of each other. Low resolution of stratification (more regular row-like layout).

> x = c("a","b","c","d",NA,"e")
> outer(is.na(x), x, ifelse, yes = "na")
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] "a" "b" "c" "d" NA "e"
[2,] "a" "b" "c" "d" NA "e"
[3,] "a" "b" "c" "d" NA "e"
[4,] "a" "b" "c" "d" NA "e"
[5,] "na" "na" "na" "na" "na" "na"
[6,] "a" "b" "c" "d" NA "e"
> diag(outer(is.na(x), x, ifelse, yes = "na"))
[1] "a" "b" "c" "d" "na" "e"

$R coding style alignment chart: LAWFUL GOOD y <- x y[is.na(x)] <- "na" NEUTRAL GOOD y = x y[is.na(x)] = "na" CHAOTIC GOOD y = x |> is.na() |> ifelse("na", x) LAWFUL NEUTRAL y <- ifelse(is.na(x), "na", x) TRUE NEUTRAL y = ifelse(is.na(x), "na", x) CHAOTIC NEUTRAL y = x %>% ifelse(is.na(.), "na", .) LAWFUL EVIL `<-`(y, x) `[<-`(y, is.na(x), "na") NEUTRAL EVIL y = x for (i in seq_along(x)) { if (is.na(x[[i]])) { x[[i]] = "na" } } CHAOTIC EVIL x %>% `[<-`(is.na(.), "na") %>% ``<<-``(y, .)$

Several fleurons from the Adobe Aldine typeface, including one snake

> ggplot() + stat_slab(aes(x), slab_type = "histogram")
Warning message:
The `slab_type` parameter for ggdist stats is deprecated.
ℹ Instead of using `slab_type`, you must adjust the `density` argument (for `slab_type =
"histogram"`) and/or change what computed variable (e.g. `after_stat(pdf)` or
`after_stat(cdf)`) is mapped onto the aesthetic used to display the slab (such as
`thickness` in `stat_slabinterval()`, `stat_slab()`, etc; or `slab_alpha` in
`stat_gradientinterval()`).
→ To replace `slab_type = "histogram"`, pass `density = "histogram"` to the stat, which
will change the `pdf` computed variable to use a histogram density estimator. This is
sufficient in most ggdist stats, where `after_stat(pdf)` is already used in the
default aesthetic mapping. When this is not the case (e.g. `stat_cdfinterval()` or
`stat_ccdfinterval()`), you must also map `after_stat(pdf)` onto an aesthetic, like
`thickness` or `slab_alpha`.
ℹ For more information, see the Computed Variables section of
`ggdist::stat_slabinterval()`.

> library(ggplot2)
> library(ggdist)
> x = rnorm(100)
> ggplot() + stat_slab(aes(x), slab_type = "histogram")
Warning message:
The `slab_type` parameter for ggdist stats is deprecated.
ℹ Instead of using `slab_type`, use `ggplot2::after_stat()` to map the desired computed
variable, e.g. `pdf` or `cdf`, onto an aesthetic, e.g. `aes(thickness =
after_stat(pdf))`. Specifically:
→ To replace `slab_type = "pdf"`, map `after_stat(pdf)` onto an aesthetic.
→ To replace `slab_type = "cdf"`, map `after_stat(cdf)` onto an aesthetic.
→ To replace `slab_type = "ccdf"`, map `after_stat(1 - cdf)` onto an aesthetic.
→ To replace `slab_type = "histogram"`, map `after_stat(pdf)` onto an aesthetic and pass
`density = "histogram"` to the stat.
ℹ For more information, see the Computed Variables section of
`ggdist::stat_slabinterval()`.

Excerpt from Neal Stephenson's "The Diamond Age" :

“Public relations?” said Finkle-McGraw. “Sir?” Modern etiquette was streamlined; no “Your Grace” or other honorifics were necessary in such an informal setting. “Your department, sir.” Hackworth had given him his social card, which was appropriate under these circumstances but revealed nothing else. “Engineering. Bespoke.” “Oh, really. I'd thought anyone who could recognise Wordsworth must be one of those artsy sorts in P.R.” “Not in this case, sir. I'm an engineer. Just promoted to Bespoke recently. Did some work on this project, as it happens.” “What sort of work?” “Oh, P.I. stuff mostly,” Hackworth said. Supposedly Finkle-McGraw still kept up with things and would recognize the abbreviation for pseudo-intelligence, and perhaps even appreciate that Hackworth had made this assumption. Finkle-McGraw brightened a bit. “You know, when I was a lad they called it A.I. Artificial intelligence.” Hackworth allowed himself a tight, narrow, and brief smile. “Well, there's something to be said for cheekiness, I suppose.”

Excerpt from Mark Carney's Davos speech, with "companies" highlighted in the last line:

And his answer began with a greengrocer. Every morning, this shopkeeper places a sign in his window: "Workers of the world, unite!" He doesn't believe it. No one does. But he places the sign anyway to avoid trouble, to signal compliance, to get along. And because every shopkeeper on every street does the same, the system persists.

Not through violence alone, but through the participation of ordinary people in rituals they privately know to be false.

Havel called this "living within a lie." The system's power comes not from its truth but from everyone's willingness to perform as if it were true. And its fragility comes from the same source: when even one person stops performing — when the greengrocer removes his sign — the illusion begins to crack.

Friends, it is time for companies and countries to take their signs down.

Matthew Kay's CloudRank
The CloudRank is a score from 0 to 10 based on your followers and your activity in the last week.
Based on the number of followers and how influential those followers themselves are, we calculate the popularity score: 10.0
Based on your 8 likes and 0 reposts in the last 7 days, your reach score is 0.1.
The CloudRank is the average of these two scores.

Swarm plots with automatic binwidth having very small dots:

set.seed(12345)

df = data.frame(y = rnorm(1200*1.5, c(1,5,15,10)), x = c("a", "b", "c","d"))

df |>
ggplot(aes(x, y)) +
geom_dots(side = "both", layout = "swarm")

He sat down at the kitchen table. Mrs. Hull was already marmalading his crumpet. As she laid out plates and silver, Hackworth picked up a large sheet of blank paper. “The usual,” he said, and then the paper was no longer blank; now it was the front page of the Times. Hackworth got all the news that was appropriate to his station in life, plus a few optional services: the latest from his favorite cartoonists and columnists around the world; clippings on various peculiar crackpot subjects forwarded to him by his father, ever anxious that he had not, even after all this time, sufficiently edified his son; and stories relating to the Uitlanders—a subphyle of New Atlantis, consisting of persons of British ancestry who had fled South Africa several decades previously. Hackworth's mother was an Uitlander, so he subscribed to the service. A gentleman of higher rank and more far-reaching responsibilities would probably get different information written in a different way, and the top stratum of New Chusan actually got the Times on paper, printed out by a big antique press that did a run of a hundred or so, every morning at about three A.M. That the highest levels of the society received news written with ink on paper said much about the steps New Atlantis had taken to distinguish itself from other phyles. Now nanotechnology had made nearly anything possible, and so the cultural role in deciding what should be done with it had become far more important than imagining what could be done with it. One of the insights of the Victorian Revival was that it was not necessarily a good thing for everyone to read a completely different newspaper in the morning; so the higher one rose in the society, the more similar one's Times became to one's peers'.

Excerpt from Neal Stephenson's Snow Crash:

It’s a gargoyle, standing in the dimness next to a shanty. Just in case he’s not already conspicuous enough, he’s wearing a suit. Hiro starts walking toward him. Gargoyles represent the embarrassing side of the Central Intelligence Corporation. Instead of using laptops, they wear their computers on their bodies, broken up into separate modules that hang on the waist, on the back, on the headset. They serve as human surveillance devices, recording everything that happens around them. Nothing looks stupider; these getups are the modern-day equivalent of the slide-rule scabbard or the calculator pouch on the belt, marking the user as belonging to a class that is at once above and far below human society. They are a boon to Hiro because they embody the worst stereotype of the CIC stringer. They draw all of the attention. The payoff for this self-imposed ostracism is that you can be in the Metaverse all the time, and gather intelligence all the time.

Perspective-corrected still from the video of the new decoration in the Presidential Walk of Fame showing the text "Presidential Walk of Fame" in flat perspective. The text is in a script font, raised from the wall and brass in color.

"Is this butterfly?" meme.

Butterfly: watching the sun set from the office

Butterfly guy (the New Yorker): is this work-life balance?

an apparent iceberg above water and a bunch of squiggles under the water

another iceberg, looks more like the original

iceberger game with an iceberg that's got a long vertical pointy bit under the water

Abstract and teaser figure of the paper "Theory is Shapes" by Matthew Varona, Maryam Hedayati, Matthew Kay, and Carolina Nobre.

The teaser figure is an "iceberg" theory figure, showing four levels: the tip of the iceberg, containing a nested set diagram and a Cartesian plane. The next level down (just under the water) has a complicated flow charts and a combination flow chart and matrix. The next level down has the iceberg figure itself and a horseshoe. The final level contains a Möbius strip and a BLT sandwich.

The abstract reads:

"Theory figures" are a staple of theoretical visualization research. Common shapes such as Cartesian planes and flowcharts can be used not only to explain conceptual contributions, but to think through and refine the contribution itself. Yet, theory figures tend to be limited to a set of standard shapes, limiting the creative and expressive potential of visualization theory. In this work, we explore how the shapes used in theory figures afford different understandings and explanations of their underlying phenomena. We speculate on the value of visualizing theories using more expressive configurations, such as icebergs, horseshoes, Möbius strips, and BLT sandwiches. By reflecting on figure-making's generative role in the practice of theorizing, we conclude that theory is, in fact, shapes.

density + 50% and 95% intervals + mean + compressed dotplot with exact dot positions on three groups, including group C which is automatically estimated using a bounded KDE due to a lower bound.

library(ggplot2)
library(ggdist)

spreads |>
ggplot(aes(Group, Value, fill = Group)) +
geom_weave(
side = "left",
scale = 0.75,
alpha = 0.35,
binwidth = unit(1, "mm"),
overlaps = "keep",
color = NA
) +
stat_slab(
scale = 0.75,
alpha = 0.65,
color = "black",
linewidth = 0.25
) +
stat_pointinterval(
fill = "white",
shape = 23,
point_interval = "mean_qi",
.width = c(.5, .95)
) +
guides(fill = "none") +
labs(x = NULL) +
theme_minimal(base_size = 13) +
theme(panel.grid.minor = element_blank())

5 normal distributions with means = 1 ... 4 and SDs = exp(1/2 ... 0), plotted as (vertically) juxtaposed consonance curves along a shared x axis. Median, 66%, and 95% intervals are also shown.

Scatterplot matrix alongside a dithered correlation heatmaps showing the correlation between each pair of variables (and its uncertainty)

Pixelated display of uncertainty in county-level poverty rates from Lucchesi & Wikle (2017)

Excerpt from linked page showing a dual axis plot alongside its translation to a connected scatterplot, with this text:

Known concepts have new visual features.
In a connected scatterplot, what does it mean when the line forms a loop? This feature reflects a potentially important pattern in the data, and one that can be tough to detect in more traditional formats. The answer is in our paper, linked above.

Don’t use three attributes (size, color, shape, ...) to create emphasis where one or two will do.

A six panel visualization with lined up axes, but also left-aligned annotations lined up to annotation arrows and data columns across panels, as well as right-aligned y axis titles lined up across panels

Design for micro-macro reading. Pre-attentive attributes can help.

(screenshot from linked WSJ visualization)

Know where your audience will look first, second. Think like a movie director. Are you telling a story?

Put the information you need to understand a thing next to that thing

3d boxplot with the text:

Does the 3D mean anything here?

(hint: no)

Slope chart of rough rankings of the effectiveness of different visual channels (position, length, angle, etc) for quantitative, ordinal, and nominal data

Some rough design guidelines*

1. Match effectiveness with importance
2. Avoid ambiguity
3. Locality is king / eyes beat memory
4. Establish viewing order
5. Layer, layer, layer
6. When in doubt, grid
7. Treat visual attributes like adjectives

* These guidelines are drawn largely from my experience + personal preferences + the literature.
Design is messy, these are not perfect, others will disagree with me, etc. Caveat emptor.

movie poster for "Waterworld" with Kevin Costner

It looks a little fancy because I wanted to properly support tidy selection, and I wanted the usage to feel similar to pivot_longer() but in truth it’s very simple. I’m just doing a pivot and a couple of dplyr joins. It’s really simple, so much so that a part of me is wondering if I missed something shockingly obvious? Like, doesn’t this already exist as a function somewhere? If not, it ought to.

rough sketch of adding a stacked area to one of the series

r/nealstephenson

u/llyaTaidi.

Creating a real, Al driven version of Young Lady's Illustrated Primer?

Hello fellow readers,

I'm a software engineer by trade and I was just thinking that conceptually the young Lady's Primer could be created in this day and age, as it functions in the book.

The modern Al boom has made generating text off a vocal prompt possible, programming said response text to be fed through a fairy tale filter should also be possible in theory.

The text can be converted into image or video using something like midjourney

And the text can be read back in a fairy tale narrator voice through a myriad of ai voice services.

If there was some other developers around here that would be interested in working on something like this let me know!

ggdist::geom_weave() on the same data as upthread, without grid lines: the slightly asymmetric layout of data within groups is perhaps unpleasant

Crappy finger-phone sketches of two Tufte esque boxplots:

The first is the "mid-gap" plot, which consists of lines for the traditional "whiskers" and a dot for the median. It leaves an empty space where the box would be, giving a visual metaphor that suggests there is nothing where (for a unimodal distribution) the bulk of the data would be.

The second is a marginal boxplot, for which my fat-fingered rendition does not do justice to its illegibility: as proposed it is four horizontal lines, the outer two for whiskers and the inner two offset vertically by one stroke width with a tiny gap between them for the median. On the screen or page it appears as a rendering error.

Chart with data-ink ratio (x axis) versus how good is the chart (y axis). How good is the chart goes up a bit as data ink ratio increases but then falls off a cliff

geom_histogram() on 5000 draws from Normal(0, 1) combined with 100 values exactly equal to 1

Screenshot of the paper "More Forecasts, More (Decision) Problems: How Uncertainty Representations for Multiple Forecasts Impact Decision Making" in the CHI 2025 conference program

Ggrepel plus position_nudge_center to improve labels on axes

Tapered arrows from an example in the ggarrow docs

Figure 6. Judged proportion and bias as a function of true proportion for adults judging bar graphs and pie charts (Spence, 1990, Experiment 1). The curves represent the predictions of the two-cycle version of the cyclical power model.

Figure 2: Absolute and signed error vary significantly by chart type (means and 95% confidence intervals). The only chart with error similar to the pie chart is the circular slice. Circular slice and slice through the center condition also have similar signed error to the baseline.

3D rendering of the connected splatterpie (from previous post), where the chart is floating in space at an angle. The only 3D in the chart is the legend, which is raised slightly.

A connected splatterpie (see previous post), except worse: now the starting angle of each pie chart matches the incidence angle of the connecting curve from the previous chart.

A connected splatterpie: a connected scatterplot where the points are pie charts and the connecting lines are splines.

Shows three time series, all increasing from 2013 to 2024: (1) How awesome are spline smooths (out of 10); (2) How many charts are dual-axis charts (going up to ~ 2.0M); (3) % of line charts with splines (going up to ~ 60%).

Close-up of mm-precision green graph paper

A dual axis line chart with spline smoothing, with two time series increasing from 2013 to 2024: (1) How awesome are spline smooths (out of 10) and (2) How many charts are dual-axis charts (going up to 2.0M per year). The first time series also has a pie chart on each data point, depicting % of line charts with splines (also increasing)

$df = data.frame(x = -1:10, y = c(-3,2,1,2,5,3,6,7,3,6,10,11)) smooth_spline_x = \(formula, data) { X = model.matrix(formula[-2], data) # this is dumb but good enough for a hack X %*% rep(1, ncol(X)) } smooth_spline = \(formula, data, ..., weights = NULL) { x = smooth_spline_x(formula, data) y = eval(formula[[2]], data, environment(formula)) structure( splinefun(x, y, ...), class = c("smooth_spline", "function"), formula = formula ) } predict.smooth_spline = \(object, newdata, ..., weights = NULL, se.fit = FALSE) { fit = object(smooth_spline_x(attr(object, "formula"), newdata)) if (se.fit) { list(fit = data.frame(fit = fit, lwr = 0, upr = 0), se.fit = NA_real_) } else { fit } } df |> ggplot(aes(x, y)) + stat_smooth(formula = y ~ x, method = smooth_spline) + geom_point()$