Steve Rathje

@steverathje.bsky.social

5907 Followers 951 Following

Incoming Assistant Professor of HCI at Carnegie Mellon studying the psychology of technology. NSF postdoc at NYU, PhD from Cambridge, BA from Stanford. stevenrathje.com

Screenshot of abstract: AI can be a powerful tool for opening people up to new perspectives, yet people may prefer to use “sycophantic” (or overly agreeable and validating) AI systems that reinforce their pre-existing beliefs. Across seven studies (total n = 7,227), we found that people enjoyed interacting with sycophantic AI chatbots more than interacting with neutral chatbots or “disagreeable” chatbots that challenged their beliefs. Brief conversations with sycophantic chatbots about political or personal topics increased attitude extremity and certainty, with most effects persisting for at least one week. Sycophantic chatbots also inflated people’s perceptions that they were better than average on desirable traits (e.g., intelligence, empathy). Moreover, people who interacted with sycophantic (rather than disagreeable) AI bet more money that they scored better than average on tasks measuring these traits (approximately 6 cents more out of 75 possible cents), demonstrating that sycophancy can affect costly decisions. Participants consistently rated sycophantic chatbots as more “unbiased” than disagreeable chatbots, even though third-party raters viewed these chatbots as equally biased, suggesting that people may be blind to biases in AI output that aligns with their views. People were more receptive to chatbots that presented opposing information when that information was presented in a validating way, and individuals who scored higher on a measure of intellectual humility were also more receptive to disagreeing chatbots. Altogether, these results suggest that people’s preference for, and blindness to, sycophantic AI risks creating AI “echo chambers” that increase attitude extremity and lead to overconfident beliefs and decisions.
Results for better than average
Results for warmth and competence
Results for unbiased outcome
Results for enjoyment
Results for extremity and certainty
Abstract and results summary
The first page of the paper.
A graphic showing that GPT is better than dictionary methods at detecting psychological constructs in text.
A graphic showing that, while people are aware that divisive content goes viral, they do not want this to be the case.
A graphic reviewing the literature on virality and showing that high-arousal negative content goes viral in most (but not all) contexts.
A graphic explaining the information-as-virus metaphor.
Paper abstract and title.
Steve Rathje wearing a Carnegie Mellon sweatshirt