Search Results for author:
Found 9 papers, 1 papers with code
SpinQuant: LLM quantization with learned rotations
In this work, we identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures, and find that some random rotations lead to much better quantization than others, with an up to 13 points difference in downstream zero-shot reasoning performance.
Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
Large language models (LLMs) significantly enhance the performance of various applications, but they are computationally intensive and energy-demanding.
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0. 7%/0. 8% than MobileLLM 125M/350M.
Not All Weights Are Created Equal: Enhancing Energy Efficiency in On-Device Streaming Speech Recognition
This study delves into how weight parameters in speech recognition models influence the overall power consumption of these models.
On The Open Prompt Challenge In Conditional Audio Generation
Text-to-audio generation (TTA) produces audio from a text description, learning from pairs of audio samples and hand-annotated text.
Revisiting Sample Size Determination in Natural Language Understanding
Knowing exactly how many data points need to be labeled to achieve a certain model performance is a hugely beneficial step towards reducing the overall budgets for annotation.
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits.
Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding
Most existing continual learning approaches suffer from low accuracy and performance fluctuation, especially when the distributions of old and new data are significantly different.
Automatic Mixed-Precision Quantization Search of BERT
Knowledge distillation, Weight pruning, and Quantization are known to be the main directions in model compression.
