Search Results for author:
Found 3 papers, 2 papers with code
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
From this observation, we consider an intuitive KD loss function, the mean squared error (MSE) between the logit vectors, so that the student model can directly learn the logit of the teacher model.
FINE Samples for Learning with Noisy Labels
Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative-free simple methods having theoretical guarantees.
Ranked #2 on
Image Classification
on WebVision
Understanding Knowledge Distillation
To verify this conjecture, we test an extreme logit learning model, where the KD is implemented with Mean Squared Error (MSE) between the student's logit and the teacher's logit.
