Instead of attending to every pair of positions, you make two lightweight passes — one along rows, one along columns. I Just wrote up a small blogpost about it: 📖 jkobject.com/criss-c... Would love to hear from anyone exploring efficient attention mechanisms 🙂 #Transformers #Attention 2/2

0 likes 0 replies