Ankur Guoswami

@ankurgos.bsky.social

129 Followers 608 Following

Living in and probably posting about seattle. Warriors fan, ml engineer (posts my own yada yada)

The real kicker for this is that in the above SFT stages, the best teacher models aren't normally the easies to integrate into the post-training recipe! The leading fully open, reasoning SFT works (open thoughts and olmo) have had a very hard time updating their recipes to use the strongest models as teachers. Often it is a smaller, surprising model which is the best teacher. This means that it may not even be the cutting edge models that are enabling distillation! Messy.
• Readers added context Per its announcement, Inkling was pretrained from scratch but used a small SFT bootstrap on synthetic data from open models incl. Kimi K2.5. Other models like Llama 3.1 were also trained from scratch without OpenAl/Anthropic distillation.
During a meeting this week open to all employees at Instagram, Meta chief product officer Chris Cox addressed the "difficult" and "brutal" environment created by the "insanity of this company" in the past few months, according to a recording heard by WIRED. Cox applauded Instagram employees for launching features and serving around 2 billion users amid what he compared to "running a marathon in the middle of a hailstorm and then, like, your teammate gets replaced and then we're recording you."
Harness metrics from Artificial Analysis on DeepSWE, each with Opus 4.7. OpenCode: 40, Cursor: 32, CC: 27
A large group of people marching