Search Results for author: Jinbao Xue

Found 6 papers, 1 papers with code

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling

no code implementations23 May 2024 Shuaipeng Li, Penghao Zhao, Hailin Zhang, Xingwu Sun, Hao Wu, Dian Jiao, Weiyan Wang, Chengjun Liu, Zheng Fang, Jinbao Xue, Yangyu Tao, Bin Cui, Di Wang

First, we raise the scaling law between batch sizes and optimal learning rates in the sign of gradient case, in which we prove that the optimal learning rate first rises and then falls as the batch size increases.

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent

no code implementations6 Mar 2023 Xiaonan Nie, Yi Liu, Fangcheng Fu, Jinbao Xue, Dian Jiao, Xupeng Miao, Yangyu Tao, Bin Cui

Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models.

Management Scheduling

M6: A Chinese Multimodal Pretrainer

no code implementations1 Mar 2021 Junyang Lin, Rui Men, An Yang, Chang Zhou, Ming Ding, Yichang Zhang, Peng Wang, Ang Wang, Le Jiang, Xianyan Jia, Jie Zhang, Jianwei Zhang, Xu Zou, Zhikang Li, Xiaodong Deng, Jie Liu, Jinbao Xue, Huiling Zhou, Jianxin Ma, Jin Yu, Yong Li, Wei Lin, Jingren Zhou, Jie Tang, Hongxia Yang

In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.

Image Generation

A Multi-Semantic Metapath Model for Large Scale Heterogeneous Network Representation Learning

no code implementations19 Jul 2020 Xuandong Zhao, Jinbao Xue, Jin Yu, Xi Li, Hongxia Yang

In real-world applications, networks usually consist of billions of various types of nodes and edges with abundant attributes.

Link Prediction Network Embedding

Cannot find the paper you are looking for? You can Submit a new open access paper.