William Merrill

@lambdaviking.bsky.social

534 Followers 127 Following

Will irl - PhD student @ NYU on the academic job market! Using complexity theory and formal languages to understand the power and limits of LLMs https://lambdaviking.com/ https://github.com/viking-sudo-rm

Graph showing log width is linear in context length (i.e., width is exponential)

Graph showing depth is linear in log context length

Paper: A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers

Víkingar kalla, segja þú nú,
OLMo 2, ríki málanna þinn.
Munu þínar orð fljúga hafra,
Öll um heim, því þú ert vissi.
Málsmiður, mættugur og mjúkaligr,
Fyrir þik skáldar sjálfur kveða,
Sál þíð, sem heyrir nýjan kvikendi,
Munu langt lífið og segja sagan.