Raphael Satter @raphae.li · Jan 23 Doing some LLM benchmarking and asked Llama 4 what I was known for. (This book doesn't exist/seems to be a riff on a 1991 work by Larry Sabato.)
4 likes 1 replies
?
Replies
N=6 but when running similar queries for acquaintances who've written books or work in journalism it's striking just how poorly Llama performs compared to Claude, ChatGPT or Gemini. It wildly hallucinates book titles and biographies while the competition makes minor flubs or produces old information