Search Results for author:
Found 2 papers, 1 papers with code
Co-training and Co-distillation for Quality Improvement and Compression of Language Models
2) The enhanced performance of the larger model further boosts the performance of the smaller model.
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models
Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
