Andrew Drozdov

@mrdrozdov.com

5354 Followers 608 Following

Research Scientist @ Mosaic x Databricks. Adaptive Methods for Retrieval, Generation, NLP, AI, LLMs https://mrdrozdov.github.io/

A complex factorio layout, implying that something simple like a "data mix" actually relies on many advanced components working effectively in tandem.
@earmyturns (Fernando Pereira) on twitter: It isn't. Vector encodings of sentences predated transformers. Self-attention predated transformers too. Maybe someone said that as a joke, but the transformer idea came from computational efficiency considerations together with preexisting techniques.
first page of AIAYN with the key attribution highlighted
youtube giving a warning after I clicked on a bluesky link
The dark/light bus meme with text “this paper should have been a blog post” at the top.
chatgpt is responding, and it has the bluesky logo as its loading icon
2563 followers on twitter
Results showing that listwise reranking outperforms cross-encoders on average measured by recall@10.
There is a query, the ground truth relevant document, and the document found by the reranker. The ground truth has extensive text overlap with the query and is easily found by the retriever. Meanwhile, the reranker prefers an irrelevant document with minimal or no text overlap.
We measure Recall@K for different K using 1-stage ranking with retrievers and rerankers. In this fair competition, we see rerankers are surprisingly worse.
A plot showing that reranking improves recall as we increase the number of reranked docs, but with increasing docs we diminishing returns and eventually a performance dip.
Leonardo meme from Inception. Block text: "When the LLM generates two <THOUGHT> tokens in a row."
Louis Vuitton using multiple layers of massive faux trucks to cover up construction of their Fifth Ave store in NYC.