David Hall

@dlwh.bsky.social

879 Followers 203 Following

Research Engineering Lead at @StanfordCRFM. I do NLP and foundation model things with JAX. Previously Semantic Machines, Microsoft, Berkeley, Breeze

Explanation of data shop: prompt or sample data comes in, llm finds more data, train a cheap model to find even more, train, --> llm

pareto frontier of flops vs bits-per-byte

Flowchart shoing Github issue (preregistration) -> pull request (experiment.py) -> execution (watch it live) -> WandB report (analysis)

open weights vs open source (weights + code + recipe) vs open development (+ process, anyone can contribute)