Raphael De Lio @raphaeldelio.dev · Mar 15 The thing is that the longer the context window is the worse retrieval accuracy is as well. For GPT 5.4 that was released last week, you can see that accuracy is ~97% up to 32k tokens, but drops to ~36% when over 512k tokens. You can fit a million tokens, but GPT will not use it effectively.
0 likes 2 replies
?
Replies
That's interesting. It still stands to reason that a model architecture prioritising RAG would have superior context retrieval. Maybe there is a study that says otherwise, like limitations with attention-head complexity. idk tho, hopefully my motivation to research remains when I wake up. I'm tired.