Luna Nova @lunnova.dev · Apr 22 Steering vectors can work as activation oracles! Inspired by @isolyth.dev 's recent work on instruct vectors, I've ported the same approach to create Activation Oracles. Seems to work surprisingly well given the param deficit, nearly matching Activation Oracle LoRA performance.
35 likes 3 replies
?
Replies
What’s the tldr for Activation Oracles?
One thing, it might be nice to see baseline/completely untrained models for comparison. How good is qwen on its own at this vs vectors?