Search Results for author:
Found 7 papers, 3 papers with code
The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth
While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which do not necessarily require a large model's complexity.
NEPENTHE: Entropy-Based Pruning as a Neural Network Depth's Reducer
While deep neural networks are highly effective at solving complex tasks, their computational demands can hinder their usefulness in real-time applications and with limited-resources systems.
The Quest of Finding the Antidote to Sparse Double Descent
In energy-efficient schemes, finding the optimal size of deep learning models is very important and has a broad impact.
Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?
Pruning is a widely used technique for reducing the size of deep neural networks while maintaining their performance.
Sparse Double Descent in Vision Transformers: real or phantom threat?
Vision transformers (ViT) have been of broad interest in recent theoretical and empirical works.
DSD$^2$: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?
Second, we introduce an entropy measure providing more insights into the insurgence of this phenomenon and enabling the use of traditional stop criteria.
Can we avoid Double Descent in Deep Neural Networks?
Very recently, an unexpected phenomenon, the ``double descent'', has caught the attention of the deep learning community.
