Search Results for author:
Found 3 papers, 1 papers with code
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
In this work, we question the use of STE for extreme LLM compression, showing that it can be sub-optimal, and perform a systematic study of quantization-aware fine-tuning strategies for LLMs.
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
We consider nonconvex stochastic optimization problems in the asynchronous centralized distributed setup where the communication times from workers to a server can not be ignored, and the computation and communication times are potentially different for all workers.
Kimad: Adaptive Gradient Compression with Bandwidth Awareness
In distributed training, communication often emerges as a bottleneck.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
