1 code implementation • 18 Dec 2023 • Baitan Shao, Ying Chen
To obtain early decoupled knowledge, an initialization scheme for the teacher is devised, and a 2D geometry-based analysis experiment is conducted under ideal conditions to showcase the effectiveness of this scheme.
1 code implementation • 15 Aug 2021 • Baitan Shao, Ying Chen
Considering the fact that students have different abilities to understand the knowledge imparted by teachers, a multi-granularity distillation mechanism is proposed for transferring more understandable knowledge for student networks.