Georg Ahnert

@ahnert.eurosky.social

316 Followers 610 Following

PhD Student in Social Data Science at University of Mannheim | LLMs and Surveys | georgahnert.de

Scatterplot that shows high correlation (r=0.97) and high NDCG@10 between estimated and true annotator quality for data from the Manifestos project
In 2026, IC2S2 has 774 submissions, 515 accepted papers (rate: 67%), and 555 registered participants
Cover image for our IC2S2 Tutorial titled "An Introduction to Simulation Human Survey Responses with LLMs: Potentials and Pitfalls", happening on Tuesday, July 28 between 1:15pm and 4:30pm. The image features a few of the choices researchers can take when simulating survey responses with LLMs: questionnaire presentation and prompt perturbations, amongst others.
A table that shows OLS regression coefficients for individual-level alignment. Restricted generation methods lead to significant improvements over the First-Token Probabilities methods.
Specification curves with Survey Response Generation Methods in different colors across 4 datasets: ANES 2016, GLES 2017, GLES 2025, and ATP 2021. The plots show that there are big differences in individual-level alignment between the Survey Response Generation Methods and that token probability-based methods underperform.
A table that shows how Survey Response Generation Methods can be grouped into token probability-based methods, restricted generation methods, and open generation methods. They differ in (1) whether they access token probabilities, (2) whether they enforce a format with instructions, (3) whether they restrict the LLM vocabulary, (4) whether they generate open-ended output first, and (5) whether they generate a probability distribution.
A scientific paper titled "Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models", written by Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, and Markus Strohmaier. Additionally, a central figure of the paper with specification curves is shown.
Laura Nelson on stage presenting her keynote at IC2S2. The slide lays out "A maturing field" of Computational Qualitative Research
The Swedish countryside as seen from a moving train, with a lake, a red and white house, and some cows.
A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.
Our estimates with Llama 3 Temporal Adapters show a strong positive and significant correlation with collective frustration, fear, boredom, and sadness. Our results vary strongly between emotions, but they are in line with a baseline method's estimates.
Overview of our method that shows how each week's Twitter data is used to train a separate Temporal Adapter, and how a weekly affect aggregate is then obtained from the LLM's token probabilities.
A lineplot that shows how scared people in the UK were over time, during the first COVID-19 lockdown. Our method (Llama 3 Temporal Adapters) produces similar estimates of as the survey data gathered by YouGov.