no code implementations • 1 Mar 2024 • shiyi qi, Liangjian Wen, Yiduo Li, Yuanhang Yang, Zhe Li, Zhongwen Rao, Lujia Pan, Zenglin Xu
To substantiate this claim, we introduce the Cross-variable Decorrelation Aware feature Modeling (CDAM) for Channel-mixing approaches, aiming to refine Channel-mixing by minimizing redundant information between channels while enhancing relevant mutual information.
1 code implementation • 27 Feb 2024 • Yuanhang Yang, shiyi qi, Wenchao Gu, Chaozheng Wang, Cuiyun Gao, Zenglin Xu
To address this issue, we present \tool, a novel MoE designed to enhance both the efficacy and efficiency of sparse MoE models.
1 code implementation • 6 Jun 2023 • Bin Hu, Chenyang Zhao, Pu Zhang, ZiHao Zhou, Yuanhang Yang, Zenglin Xu, Bin Liu
In this paper, we explore how to enable intelligent cost-effective interactions between the agent and an LLM.
1 code implementation • 20 Dec 2022 • Zhuo Zhang, Yuanhang Yang, Yong Dai, Lizhen Qu, Zenglin Xu
To facilitate the research of PETuning in FL, we also develop a federated tuning framework FedPETuning, which allows practitioners to exploit different PETuning methods under the FL training paradigm conveniently.
1 code implementation • 11 Oct 2022 • Yuanhang Yang, shiyi qi, Chuanyi Liu, Qifan Wang, Cuiyun Gao, Zenglin Xu
Transformer-based models have achieved great success on sentence pair modeling tasks, such as answer selection and natural language inference (NLI).
1 code implementation • 24 Jul 2022 • Chaozheng Wang, Yuanhang Yang, Cuiyun Gao, Yun Peng, Hongyu Zhang, Michael R. Lyu
Besides, the performance of fine-tuning strongly relies on the amount of downstream data, while in practice, the scenarios with scarce data are common.