1 code implementation • EMNLP 2020 • Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
We also study the transferability of models trained on MedDialog to low-resource medical dialogue generation tasks.
2 code implementations • 4 Apr 2024 • Longxu Dou, Qian Liu, Guangtao Zeng, Jia Guo, Jiahui Zhou, Wei Lu, Min Lin
We present Sailor, a family of open language models ranging from 0. 5B to 7B parameters, tailored for South-East Asian (SEA) languages.
2 code implementations • 4 Jan 2024 • Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, Wei Lu
We present TinyLlama, a compact 1. 1B language model pretrained on around 1 trillion tokens for approximately 3 epochs.
1 code implementation • 23 Oct 2023 • Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan
We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.
1 code implementation • 28 May 2023 • Guangtao Zeng, Peiyuan Zhang, Wei Lu
Fine-tuning pre-trained language models for multiple tasks tends to be expensive in terms of storage.
1 code implementation • 23 Oct 2022 • Guangtao Zeng, Wei Lu
Training a good deep learning model requires substantial data and computing resources, which makes the resulting neural model a valuable intellectual property.
1 code implementation • ACL 2021 • Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie
Training complex dialog generation models on small datasets bears high risk of overfitting.
1 code implementation • 11 May 2020 • Wenmian Yang, Guangtao Zeng, Bowen Tan, Zeqian Ju, Subrato Chakravorty, Xuehai He, Shu Chen, Xingyi Yang, Qingyang Wu, Zhou Yu, Eric Xing, Pengtao Xie
On these two datasets, we train several dialogue generation models based on Transformer, GPT, and BERT-GPT.