Search Results for author:
Found 8 papers, 4 papers with code
ATOM: Attention Mixer for Efficient Dataset Distillation
To address these challenges in dataset distillation, we propose the ATtentiOn Mixer (ATOM) module to efficiently distill large datasets using a mixture of channel and spatial-wise attention in the feature matching process.
ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification
We validate the effectiveness of our framework through experimentation with datasets from the computer vision and medical imaging domains.
DataDAM: Efficient Dataset Distillation with Attention Matching
Emerging research on dataset distillation aims to reduce training costs by creating a small synthetic set that contains the information of a larger real dataset and ultimately achieves test accuracy equivalent to a model trained on the whole dataset.
End-to-End Supervised Multilabel Contrastive Learning
However, they fail to design an end-to-end training framework, leading to high computational complexity.
A New Probabilistic Distance Metric With Application In Gaussian Mixture Reduction
This paper presents a new distance metric to compare two continuous probability density functions.
Subclass Knowledge Distillation with Known Subclass Labels
In classification tasks with a small number of classes or binary detection, the amount of information transferred from the teacher to the student is restricted, thus limiting the utility of knowledge distillation.
On the Efficiency of Subclass Knowledge Distillation in Classification Tasks
These results show that the extra subclasses' knowledge (i. e., 0. 4656 label bits per training sample in our experiment) can provide more information about the teacher generalization, and therefore SKD can benefit from using more information to increase the student performance.
High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networks
Our offline selections yield CNN inference time savings up to 9% and CR up to 10x.
