1 code implementation • 16 Apr 2024 • J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Recently, several approaches successfully demonstrated that weight-sharing Neural Architecture Search (NAS) can effectively explore a search space of elastic low-rank adapters (LoRA), allowing the parameter-efficient fine-tuning (PEFT) and compression of large language models.
no code implementations • 30 Sep 2023 • Sixing Yu, J. Pablo Muñoz, Ali Jannesari
This is evident across tasks in both natural language processing and computer vision domains.
no code implementations • 19 May 2023 • Sixing Yu, J. Pablo Muñoz, Ali Jannesari
Foundation Models (FMs), such as LLaMA, BERT, GPT, ViT, and CLIP, have demonstrated remarkable success in a wide range of applications, driven by their ability to leverage vast amounts of data for pre-training.
no code implementations • 9 Nov 2022 • Sixing Yu, J. Pablo Muñoz, Ali Jannesari
To address these challenges, we propose Resource-aware Federated Learning (RaFL).
no code implementations • 16 Aug 2022 • Duy Phuong Nguyen, Sixing Yu, J. Pablo Muñoz, Ali Jannesari
This method allows efficient multi-model knowledge fusion and the deployment of resource-aware models while preserving model heterogeneity.
no code implementations • 17 Jun 2021 • Yash Akhauri, Adithya Niranjan, J. Pablo Muñoz, Suvadeep Banerjee, Abhijit Davare, Pasquale Cocchini, Anton A. Sorokin, Ravi Iyer, Nilesh Jain
The rapidly evolving field of Artificial Intelligence necessitates automated approaches to co-design neural network architecture and neural accelerators to maximize system efficiency and address productivity challenges.