Teacher Network Calibration Improves Cross-Quality Knowledge Distillation

1 code implementation • 15 Apr 2023 • Pia Čuk, Robin Senge, Mikko Lauri, Simone Frintrop

We investigate cross-quality knowledge distillation (CQKD), a knowledge distillation method where knowledge from a teacher network trained with full-resolution images is transferred to a student network that takes as input low-resolution images.

Image Classification Knowledge Distillation