no code implementations • 22 Mar 2024 • Yunqi Zhu, Xuebing Yang, Yuanyuan Wu, Wensheng Zhang
Autoregressive decoding strategy is a commonly used method for text generation tasks with pre-trained language models, while early-exiting is an effective approach to speedup the inference stage.
1 code implementation • 15 May 2023 • Yunqi Zhu, Xuebing Yang, Yuanyuan Wu, Wensheng Zhang
The increasing size of language models raises great research interests in parameter-efficient fine-tuning such as LoRA that freezes the pre-trained model, and injects small-scale trainable parameters for multiple downstream tasks (e. g., summarization, question answering and translation).
no code implementations • 8 Feb 2023 • Yunqi Zhu, Xuebing Yang, Yuanyuan Wu, Wensheng Zhang
This study presents three deidentified large medical text datasets, named DISCHARGE, ECHO and RADIOLOGY, which contain 50K, 16K and 378K pairs of report and summary that are derived from MIMIC-III, respectively.
no code implementations • 31 Aug 2022 • Ding Li, Xuebing Yang, Yongqiang Tang, Chenyang Zhang, Wensheng Zhang
And the other introduces a new metric based on mutual information between adjacent action proposals and evaluates the informativeness of video samples, named Temporal Context Inconsistency (TCI).
1 code implementation • 8 Feb 2022 • Yunqi Zhu, Xuebing Yang, Yuanyuan Wu, Mingjin Zhu, Wensheng Zhang
ROUGE is a standard automatic evaluation metric based on n-grams for sequence-to-sequence tasks, while cross-entropy loss is an essential objective of neural network language model that optimizes at a unigram level.
no code implementations • 8 Dec 2021 • Zelin Ren, Xuebing Yang, Yuchen Jiang, Wensheng Zhang
In this work, to deal with the two drawbacks, a learnable faster realization of the conventional KPCA is proposed.