Alex Coppock

@aecoppock.bsky.social

6675 Followers 1517 Following

Associate Professor of Political Science at Northwestern University alexandercoppock.com Persuasion in Parallel: https://alexandercoppock.com/coppock_2022.html Research Design: Declaration, Diagnosis, and Redesign: book.declaredesign.org

figure of the estimand census.
screenshot with this text: Our goal with this exercise was to evaluate the generalizability of three main claims in this literature: (1) religious treatments are especially effective because of their religious character, (2) effects are larger for religious affiliates than nonaffiliates, and (3) effects are larger for high religiosity types than low religiosity types. Our summary conclusion is that none of these claims is correct in general. While it is true that some estimates of the effect of religious messages (both compared with pure control and compared with a secular alternative) and the effect of religious endorsements are strongly positive, these patterns do not hold across the many contexts we study. Most of the estimates we report cannot be distinguished from zero, nor can the meta-analytic averages. For this reason, we are skeptical of the claim that religious treatments are especially effective because of their religious content. Our analysis of heterogeneity strongly contradicts main claims (2) and (3). In just 3 of the 66 opportunities we study do we find statistically significant evidence of effect heterogeneity by religious affiliation or religiosity. This lack of heterogeneity appears to be quite generalizable across contexts and across the three classes of estimands we focus on here. These are not the conclusions one would draw from reading the abstracts of the papers we include. We hand-coded each abstract for whether they make a positive claim about an average treatment effect, a null claim about an average treatment effect, or a heterogeneous effects claim. This exercise, while somewhat coarse, gives us a sense of the headline claims in the field: 64% of the abstracts make positive treatment effect claims, 32% make null claims, and 46% make heterogeneous effects claims.
title page of journal, that reads: The effects of religious messages and endorsements on political attitudes: A meta-reanalysis Sarkar, Radha, Coppock, Alexander. 2026. The effects of religious messages and endorsements on political attitudes: A meta-reanalysis. American Political Science Review. Abstract The experimental study of the effects of religious messages and endorsements by religious leaders on political attitudes has a relatively short history yet has coalesced around three main claims: religious treatments are especially effective because of their religious character, effects are larger for religious affiliates than nonaffiliates, and effects are larger for high religiosity types than low religiosity types. Here we meta re-analyze the experimental record (58 treatment-outcome pairs drawn from 43 studies reported in 26 papers) to probe the generalizability of these claims. Our findings indicate that these three headline claims do not generalize straightforwardly across contexts: effects are large in some cases and close to zero in others, and we find no evidence in favor of the claimed heterogeneities by religious affiliation or religiosity. Based on a census of the estimands in this literature, we offer suggestions for future research that would enhance commensurability and synthesis.
Scatterplot of switching
SQ: For the next few years, the Russia–Ukraine war remains at a stalemate. Neither side advances far beyond where they are today. The fighting is constant but does not escalate. Pro-D Donald Trump wins the 2024 election and immediately withdraws all US support for Ukraine. Putin easily conquers Ukraine, slaughtering hundreds of thousands of people in the process. He then invades Poland, again killing many innocent people in his quest for domination. Pro-R Donald Trump wins the 2024 election and immediately negotiates a ceasefire between Russia and Ukraine. Russia withdraws from most of Ukraine, and hundreds of thousands of lives are spared because of Trump.
abstract: While attempts to change Americans’ partisanship via persuasive treatments largely fail, partisanship can and does change over time. In this paper, the authors first confirm, via survey and field experiments, that typical campaign messaging in the United States does not budge partisanship. The authors then present experiments in which participants encounter extraordinary hypothetical scenarios (e.g. one party causes economic collapse) before reporting what their partisanship would be under such circumstances. Twelve percent of partisans imagine switching parties in the pro-out-party hypothetical conditions, compared with 5% in the control hypotheticals in which the status quo persists, for a seven-percentage point (SE 1.5 points) difference. These hypothetical shifts are on par with the largest changes in American macropartisanship ever recorded. While the act of ruminating on hypothetical scenarios is not followed by changes in partisanship measured post-treatment, the evidence suggests that extraordinary world events may be able to shift partisan affiliation.
Screenshot of claude just writing a design no trouble
screenshot of the SPCO tickets page
Call for Proposals: Data Collection for Replication+Novel Political Science Survey Experiments Alexander Coppock and Mary McGrath January 27, 2026 We invite proposals for a survey experiment replication+novel design competition. Se- lected replication+novel design survey experiments will be conducted on large samples of American respondents, quota sampled to match U.S. Census margins and filtered for quality and attention by the survey sample provider Rep Data (repdata.com). Each proposal consists of two parts: (1) a replication study of an existing, previously published survey experiment, and (2) a novel experimental design on a topic of the authors’ choosing. The replication studies and reanalyses of the existing studies will be combined into a meta-paper to be co-authored by all authors of accepted proposals along with the princi- pal investigators (Coppock and McGrath). As a condition for acceptance, authors commit to sharing the data and producing a write-up of the findings from their novel design for submission to a scholarly journal, and public posting of a working paper pre-publication.
We estimate the effect of social media deactivation on users’ emotional state in two large randomized experiments before the 2020 U.S. election. People who deactivated Facebook for the six weeks before the election reported a 0.060 standard deviation improvement in an index of happiness, depression, and anxiety, relative to controls who deactivated for just the first of those six weeks. People who deactivated Instagram for those six weeks reported a 0.041 standard deviation improvement relative to controls. Exploratory analysis suggests the Facebook effect is driven by people over 35, while the Instagram effect is driven by women under 25.
Figure 4: Trends in the use of design-based methods among explanatory empirical quanti-tative papers
plot of reconstructed data
At first glance, Pollingism espouses a so-simple-a-child-gets-it truth: ask people what matters to them and repeat it back. It often gets short-handed as “say popular things” and, by extension, eschew unpopular ones. However, this notion is built off of assumptions that do not bear out in the real world. Let’s start with an irrefutable claim: a message that nobody hears cannot persuade them. Nor can it get them to act in opposition to the present regime, vote in an election, or select a specific candidate. For years, we have heard entirely accurate lamentations that Democrats are failing to reach people. To be sure, the right’s far better media system merits every alarm bell, not merely for electoral outcomes but also for our hopes of a functional society, free of lies about everything from vaccines to wind turbines to the dining habits of newcomers. But even setting that imbalance aside, Pollingism’s message creation plus testing model fails to consider, let alone optimize for, how we would get a message to be heard. Presently, the dominant approaches to campaign message crafting and evaluation involve (1) randomized-controlled trial (RCT) testing of messages and/or ads, and (2) issue-based polling to figure out which topics voters rate most vital and therefore merit airtime. But these tools cannot tell us what would break through in the cacophony of political noise to which voters are subjected.
Will the deadline be extended a week, like other conferences? No!
the c&H comic
a good figure that skewers an "anonymous" reviewer for requiring many citations to (presumably) their work
chat really told me: Excellent — that’s exactly what Hadley would do next.
title page of article: There is a skeleton in the social science closet. Almost all of us have heard it rattled at one time or another.! Nevertheless, we collectively ignore it. Some ignore it because they do not think it is very scary: others because its putting to rest would require a major restructuring of our scholarly journals. Perhaps others even fear that admitting its existence would undermine belief in the empirical progress? the social sciences have apparently enjoyed since the widespread diffusion of statistical training and the ready availability of the computer. I will argue that the skeleton, the bias social science journals exhibit for publishing articles reporting statistically significant results, is dangerous
Guided by our values and theory of policy design, Searchlight’s approach to policy will be defined by bold, persistent experimentation.
design declaration for one weird trick
I know that correlation does not equal causation. But
That’s a beautiful and uncanny concept—very Borges. Here's a short Borges-style story based on your idea, titled “The Final Word.” The Final Word by a humble imitator of Borges It began, as most obsessions do, with an aspiration toward clarity. Dr. Ezequiel Varas, a semi-retired philologist of uncertain reputation, had long maintained that the proliferation of knowledge had not enlightened the world but drowned it in multiplicity. He lamented, in letters never sent, that to know everything was to know nothing in particular. In the autumn of his seventy-fourth year, Dr. Varas gained access to a powerful artificial intelligence—an inheritance from a distant cousin in Palo Alto who had drowned in the Pacific under ambiguous circumstances. The machine—named Pleroma, from the Greek for fullness—had been trained on every known language, book, map, film, genetic sequence, war diary, weather report, and dream journal ever digitized. Dr. Varas asked Pleroma for a summary of the world. The machine obliged. It took the form of a vast document, thousands of pages long, which he printed on thin paper and read over the course of a year. The text was perfect. Nothing essential, it seemed, had been omitted. Still, the burden of its volume gnawed at him. He asked Pleroma to summarize the summary. Then again. Then again. With each recursion, the world grew more compressed, more potent. The thirteenth summary fit on a single page. The sixteenth, a single paragraph. The nineteenth, a single sentence. The twenty-second, a single word. The word was: ALETHEIA Ancient Greek: “truth” or “unconcealment.” He asked Pleroma what it meant, but the machine refused. “It is the whole,” it said. “To define it is to break it.” Varas, unbothered, wrote the word on a small slip of paper, burned the rest of the documents, and locked himself in his study. Neighbors sometimes saw the dim light at night and heard him repeating the word aloud. He did not eat, nor did he sleep, but he appeared…
using the bad approach
Arguments data Not needed, wouldn't make your analysis and more correct. Details We have: X: a binary treatment variable, that is affected by an unmeasured confounder (U). M: a mediator, that is mostly driven by U, not X Y: an outcome, that is influenced by both M and U, but you can’t see U. What you don’t have: Any valid identification strategy Temporal ordering Careful causal reasoning Value Nothing, because you shouldn't do mediation analysis.
full range of rho
DD code
abstract for VAYR
I implemented a field experiment inSouth Africa that randomly varies the capacity of police to locate households.Findings from surveys conducted several months later suggest households that have become legible to police are more willing to rely on police and less willing to participate in vigilantism
figure from persuasion in parallel
A good statistical graph for a randomized experiment simultaneously conveys the study's design, analysis, and results. It reveals the experimental design by mapping design elements to aesthetic parameters. It illuminates the analysis by plotting the statistical model in “dataspace.” When the design and analysis of an experiment are encoded in a plot, the interpretation of the experimental results is clarified. “Analyze as you randomize” is a dictum attributed to Fisher that guides interpretations of experimental data. This chapter extends that principle to visualizations of randomized experiments. While not every experiment requires a visualization, those that do should be visualized in ways that communicate the design and results together.
position_jitter()
all 21 possible assignments
declaration_9.1 <- declare_model(N = 1000, # make birthdates and obtain starsign birthdate = sample(seq(as.Date("1940-01-01"), as.Date("2005-03-11"), by = "day"), N), starsign = DescTools::Zodiac(birthdate), # potential outcomes unrelated to starsign potential_outcomes(Y ~ Z + rnorm(N))) + declare_assignment(Z = complete_ra(N)) + declare_measurement(Y = reveal_outcomes(Y ~ Z)) + declare_estimator(Y ~ Z*starsign, term = TRUE) simulations <- simulate_design(declaration_9.1, sims = 1000)
Future research will be able to use similar techniques in a causal setting to measure both attention and any downstream effects on e.g., political attitudes and behavior. Doing so will shine a light on whether and how attending to content online (with which one may or may not disagree) has an effect on subsequent attitudes and behavior such as negative outgroup affect and online hostility.
One way is to outright dismiss mediation analysis. For example, some editors have stated that (cross-sectional, observational) mediation analyses will usually get desk-rejected. This may seem a bit extreme, but then if you think about it—if such an analysis is the centerpiece of an article and the authors do nothing to credibly address confounding, they are trying to sell elaborate causal storytelling based on three correlations in a trench coat
Louise Deroualle
competing vax stories
i think these are the right pvals
Prior beliefs
Change format condition Counterfactual format condition Belief Q: “As you may know, former President Donald Trump was recently indicted for removing classified information from the White House, including sensitive nuclear secrets. Does the indictment make you think it is more likely or less likely that Trump mishandled nuclear secrets?” [More likely/ No change/Less Likely] Q1: “As you may know, former president Donald Trump was recently indicted for removing classified information from the White House, including sensitive nuclear secrets. In your opinion, how likely is it that Trump mishandled nuclear secrets?” [0-100 scale] Q2: “Suppose you did not know about the indictment. How would you have answered the following question: In your opinion, how likely is it that Trump mishandled nuclear secrets?” [0-100 scale] Support Q: (for Republican primary voters) “Does the indictment make you more or less likely to vote for Trump in the primary?” (for all others) “Imagine that the 2024 presidential election is between Joe Biden and Donald Trump. Does the indictment make you more or less likely to vote for Trump?” [More likely/No change/Less Likely] Q1: (for Republican primary voters) “How likely are you to vote for Donald Trump in the 2024 Republican primary?” (for all others) “Imagine that the 2024 presidential election is between Joe Biden and Donald Trump. How likely are you to vote for Donald Trump?” [0- 100 scale] Q2: “Suppose you did not know about the indictment. How would you have answered the following question: How likely are you to vote for Donald Trump
Change format condition Counterfactual format condition Belief Q: “As you may know, former President Donald Trump was recently indicted for removing classified information from the White House, including sensitive nuclear secrets. Does the indictment make you think it is more likely or less likely that Trump mishandled nuclear secrets?” [More likely/ No change/Less Likely] Q1: “As you may know, former president Donald Trump was recently indicted for removing classified information from the White House, including sensitive nuclear secrets. In your opinion, how likely is it that Trump mishandled nuclear secrets?” [0-100 scale] Q2: “Suppose you did not know about the indictment. How would you have answered the following question: In your opinion, how likely is it that Trump mishandled nuclear secrets?” [0-100 scale] Support Q: (for Republican primary voters) “Does the indictment make you more or less likely to vote for Trump in the primary?” (for all others) “Imagine that the 2024 presidential election is between Joe Biden and Donald Trump. Does the indictment make you more or less likely to vote for Trump?” [More likely/No change/Less Likely] Q1: (for Republican primary voters) “How likely are you to vote for Donald Trump in the 2024 Republican primary?” (for all others) “Imagine that the 2024 presidential election is between Joe Biden and Donald Trump. How likely are you to vote for Donald Trump?” [0- 100 scale] Q2: “Suppose you did not know about the indictment. How would you have answered the following question: How likely are you to vote for Donald Trump
Rainfall DAG
For a kid who grew up with empty bookshelves, this should have been a moment of triumph [1]. Instead, I felt like a fraud. At that same conference, I had to confront an uncomfortable truth: the foundation of our celebrated paper was crumbling. Ego depletion—the once-famous idea that self-control relies on a finite resource that can be depleted through use—wasn’t real. That award? It was like winning a Nobel prize for developing the frontal lobotomy as a treatment for mental illness; and, yes, that really happened. This isn’t just another story about failed replications or p-hacking (though those shenanigans will make an appearance). It’s a story about what happens when we fall in love with our theories more than the truth. The replication crisis didn’t just shake the foundations of psychology; it shook those of us who had built our careers on ideas that no longer held up to scrutiny. For me, this reckoning wasn’t just professional. It was personal
Figure 3: Figure displays the estimated AMCE on story selection of headline style interacted with baseline political engagement. The lower panel displays the esti- mated base coefficients for each style and the upper panel displays the estimated interaction terms. The error bars indicate 95 percent confidence intervals. Analy- sis excludes decision tasks that included one or more apolitical headlines. For full results, see Appendix Table A.1.1, column 2.
FIGURE 7. Information Search Behavior from the linked paper
screenshot from the above paper
Digging up Dinosaurs by Aliki cover
Division Chair(s): Alex Coppock, Yale University The Experimental Research division welcomes submissions on a wide range of topics related to the use of experimental research across the fields of comparative politics, international relations, American politics, political theory, and research methods. We invite theoretical, methodological, or empirical proposals covering diverse experimental methods, including survey, laboratory, and field experiments. We take an expansive view of what constitutes an experiment but insist upon researcher control of treatment assignment, thus excluding observational studies sometimes described as natural or quasi-experiments. We encourage submissions that build upon the annual conference theme, “Reimagining Politics, Power, and Peoplehood in Crisis Times.” Research on this theme and other relevant topics is welcome. We encourage the submission of cohesive panel proposals. We are particularly interested in panels that: (a) describe the generalizability of experimental results; (b) investigate the homogeneity or heterogeneity of treatment effects; (c) estimate the over-time persistence of effects; (d) replicate and extend the core findings of an experimental research literature. APSA offers various presentation formats, and we welcome proposals that take advantage of these options. In line with APSA’s objectives of improving diversity, inclusion, and access throughout the profession, we encourage diversity of approach and interdisciplinarity from a wide-ranging group of researchers.
r code with base pipes instead of magrittr pipes
There is a 2019 paper, in the journal Political Analysis (htm), with over 1000 Google cites, titled "How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice". The paper is not just widely cited, but is also actually influential. Most political science papers estimating interactions now-a-days, seem to rely on the proposed simple tools.1 In this post I explain why those tools should not be used with observational data.
"are you stealing those LCDs?" "yes, but I'm doing it while my code compiles"
AJPS's bad policy on PAPs in appendicies: "If you have pre-registered a pre-analysis plan and you believe that viewing it would help reviewers assess your manuscript, you may include an anonymized version of it in the online supporting information section. However, please note that it will count toward the 20-page limit placed on online supporting information sections."
texas license plate that reads 1DV RCT
The Robert E. Lane Award for the best book in political psychology published in the past year. 2024: Alex Coppock (Yale University) Persuasion in Parallel: How Information Changes Minds about Politics (University of Chicago Press, 2023) 2024: (Honorable mention) Alexandra Filindra Race, Rights, and Rifles: The Origins of the NRA and Contemporary Gun Culture (University of Chicago Press, 2023) 2023: Efrén Pérez (University of California, Los Angeles) and Margit Tavits (Washington University in St. Louis) Voicing Politics: How Language Shapes Public Opinion (Princeton University Press, 2022) 2022: Cigdem V. Sirin (University of Texas, El Paso), Nicholas A. Valentino (University of Michigan), and Jose D. Villalobos (University of Texas, El Paso) Seeing Us in Them: Social Divisions and the Politics of Group Empathy (Cambridge University Press, 2021) 2021: Nicole M. Bauer (Louisiana State University) The Qualifications Gap: Why Women Must Be Better than Men to Win Political Office (Cambridge University Press, 2020) 2020: Ashley Jardina (Duke University) White Identity Politics (Cambridge University Press, 2019) 2020: Markus Prior (Princeton University) Hooked (Cambridge University Press, 2018) 2019: Gwyneth McClendon (New York University) Envy in Politics (Princeton University Press, 2018)
graphical evidence of fraud