Jered Sutton @j9n.dev · Mar 1 Mind your prompt lengths. A new paper explores the relationship between input length and performance of large language models. The study found that performance can begin to significantly degrade with as few as 3000 tokens.
0 likes 1 replies
?
Replies
In this case, the tokens we are talking about are extraneous to the input needed to answer the question, but it brings to light the importance of managing your context.