Due to residual structures in Transformer-models it's possible that many layers contribute very little to the downstream performance of the network, allowing for removal those redundant layers with little impact. openreview.net/pdf/72eeca23...

12 likes 0 replies