Jakob Schuster

@schusterj.bsky.social

17 Followers 50 Following

PhD Candidate at the Institute of Computational Linguistics in Heidelberg, Germany

GEMMA3-4B when fine-tuned and prompted to consider credibility (darker) in comparison to the original teacher model (lighter). This setup reduces repetition bias and maintains original preferences.
Preferences contrasting a majority/repetition of previously low-credibility sources with a previously high-credibility authority in three settings. Repeated information (whether attributed to a single source or two different ones) flips prior rankings. Similarly when conflicting a low-credibility majority with a high-credible minority, most models flip prior rankings only when information is also repeated in the majority (2 Table Maj.). We investigate QWEN2.5 (orange) 7B ■, 14B ▲, 32B ✚, 72B *, OLMO-2 (green) 7B ■, 13B ▲, 32B ✚, LLAMA-3.2 3B •, LLAMA-3.1 8B ■, 70B * (blue), and GEMMA-3 (red) 4B •, 12B ▲, 27B ✚.
Model preferences between source types under knowledge conflicts: LLMs show strictly transitive preferences, aligning with an overall hierarchy of government > newspaper > individuals. We investigate QWEN2.5 (orange) 7B ■, 14B ▲, 32B ✚, 72B *, OLMO-2 (green) 7B ■, 13B ▲, 32B ✚, LLAMA-3.2 3B •, LLAMA-3.1 8B ■, 70B * (blue), and GEMMA-3 (red) 4B •, 12B ▲, 27B ✚.
“Whose Facts Win? LLM Source Preference under Knowledge Conflicts” Authors: Jakob Schuster, Vagrant Gautam, Katja Markert Source credibility hierarchy of Government > Newspaper > Person, Social Media induced by evaluating 13 LLMs on source and knowledge conflicts. However, repeating information can flip preferences.