@bowphs.bsky.social

@bowphs.bsky.social

28 Followers 24 Following

A photo of me smiling broadly at the camera. I am standing on a sandy beach, and right behind me are the beautiful blue waves of the Balearic Sea stretching out to the horizon under a bright, sunny sky.
Me in front of Bruegel's painting "The Tower of Babel".
Table comparing text generations between early and late checkpoints for the concepts "earthquake" and "joy". Early checkpoint generations show language-specific text, while late checkpoint generations demonstrate a shift toward "language-agnostic" (= English) text.
Average overlap proportion of expert neurons across layers and training checkpoints. Later checkpoints exhibit more shared neurons, particularly in the middle layers.
Probing classifier performance comparison between early and late checkpoint across layers. While the early checkpoint shows uniformly high performance, the later checkpoint exhibits relatively high variance across layers.
Table comparing text generations between early and late checkpoints for the concepts "earthquake" and "joy". Early checkpoint generations show language-specific text, while late checkpoint generations demonstrate a shift toward "language-agnostic" (= English) text.
Average overlap proportion of expert neurons across layers and training checkpoints. Later checkpoints exhibit more shared neurons, particularly in the middle layers.
Sample generations demonstrating language-specific generation in early checkpoint and language-agnostic (= English) generation in late checkpoint.
Expert overlap proportion across layers for different training checkpoints.