Search Results for author: Mingyu Xu

Found 7 papers, 3 papers with code

Base of RoPE Bounds Context Length

no code implementations23 May 2024 Xin Men, Mingyu Xu, Bingning Wang, Qingyu Zhang, Hongyu Lin, Xianpei Han, WeiPeng Chen

We revisit the role of RoPE in LLMs and propose a novel property of long-term decay, we derive that the \textit{base of RoPE bounds context length}: there is an absolute lower bound for the base value to obtain certain context length capability.

Position

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect

no code implementations6 Mar 2024 Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, WeiPeng Chen

As Large Language Models (LLMs) continue to advance in performance, their size has escalated significantly, with current LLMs containing billions or even trillions of parameters.

Quantization

Pseudo Labels Regularization for Imbalanced Partial-Label Learning

no code implementations6 Mar 2023 Mingyu Xu, Zheng Lian

Partial-label learning (PLL) is an important branch of weakly supervised learning where the single ground truth resides in a set of candidate labels, while the research rarely considers the label imbalance.

Long-tail Learning Partial Label Learning +2

IRNet: Iterative Refinement Network for Noisy Partial Label Learning

1 code implementation9 Nov 2022 Zheng Lian, Mingyu Xu, Lan Chen, Licai Sun, Bin Liu, JianHua Tao

In this paper, we relax this assumption and focus on a more general problem, noisy PLL, where the ground-truth label may not exist in the candidate set.

Data Augmentation Partial Label Learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.