Raphaël Millière @raphaelmilliere.com · Nov 7 The paper offers a more specific explanation that draws on three ideas. First, in-context learning is functionally equivalent to optimizing a task-relevant objective in context. Here, I draw on recent work formalizing ICL as mesa-optimization. 10/ arxiv.org/abs/2309.05858
0 likes 1 replies
?
Replies
We can think of ICL as a form of virtual fine-tuning in the forward pass. Real fine-tuning is effective at undoing alignment – so is ICL. The longer the context window, the more optimization steps can occur; this is concerning in the age of ever-increasing context windows. 11/