Israel Mason-Williams giving a talk on knowledge distillation from a functional perspective. Does distillation lead to models with similar representation spaces, actually transferring knowledge, and doing so efficiently? Answer often no!

0 likes 0 replies