Blake Richards @tyrellturing.bsky.social · May 8 Thanks for response. 🙂 a) Hallucinations in certain contexts != poorer reasoning. Reasoning benchmarks show clear improvements (see below). b) Prediction different than factual Q&A, hallucinations meaningless concept for prediction. c) Again, paradigm shift is in data analysis, not modelling.
4 likes 1 replies
?
Replies
a) Serious failure modes getting worse --> the promise of scale didn't deliver. Inaccuracy a major issue any application should take seriously & why big tech invests in new architectures. b) our neurips 2023 shows how hallucinations impair reasoning on simple MDPs in 8 LLMs c) helpful but not novel