Search Results for author:
Found 4 papers, 3 papers with code
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
The key insight driving QServe is that the efficiency of LLM serving on GPUs is critically influenced by operations on low-throughput CUDA cores.
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads.
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
We then propose to search for the optimal per-channel scaling that protects the salient weights by observing the activation, not weights.
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Transformer, as an alternative to CNN, has been proven effective in many modalities (e. g., texts and images).
Cannot find the paper you are looking for? You can
Submit a new open access paper.
