no code implementations • 19 May 2024 • Xuanli He, Qiongkai Xu, Jun Wang, Benjamin I. P. Rubinstein, Trevor Cohn
Modern NLP models are often trained on public datasets drawn from diverse sources, rendering them vulnerable to data poisoning attacks.
no code implementations • 30 Apr 2024 • Xuanli He, Jun Wang, Qiongkai Xu, Pasquale Minervini, Pontus Stenetorp, Benjamin I. P. Rubinstein, Trevor Cohn
The implications of backdoor attacks on English-centric large language models (LLMs) have been widely examined - such attacks can be achieved by embedding malicious behaviors during training and activated under specific conditions that trigger malicious outputs.
1 code implementation • 24 Apr 2024 • Wanru Zhao, Vidit Khazanchi, Haodi Xing, Xuanli He, Qiongkai Xu, Nicholas Donald Lane
Large language model (LLM) services have recently begun offering a plugin ecosystem to interact with third-party API services.
no code implementations • 8 Apr 2024 • Giwon Hong, Aryo Pradipta Gema, Rohit Saxena, Xiaotang Du, Ping Nie, Yu Zhao, Laura Perez-Beltrachini, Max Ryabinin, Xuanli He, Clémentine Fourrier, Pasquale Minervini
Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate human-like text.
no code implementations • 3 Apr 2024 • Jun Wang, Qiongkai Xu, Xuanli He, Benjamin I. P. Rubinstein, Trevor Cohn
Our aim is to bring attention to these vulnerabilities within MNMT systems with the hope of encouraging the community to address security concerns in machine translation, especially in the context of low-resource languages.
no code implementations • 29 Feb 2024 • Ansh Arora, Xuanli He, Maximilian Mozes, Srinibas Swain, Mark Dras, Qiongkai Xu
The democratization of pre-trained language models through open-source initiatives has rapidly advanced innovation and expanded access to cutting-edge technologies.
no code implementations • 29 Feb 2024 • Anton Lozhkov, Raymond Li, Loubna Ben allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Jian Zhu, Terry Yue Zhuo, Evgenii Zheltonozhskii, Nii Osae Osae Dade, Wenhao Yu, Lucas Krauß, Naman jain, Yixuan Su, Xuanli He, Manan Dey, Edoardo Abati, Yekun Chai, Niklas Muennighoff, Xiangru Tang, Muhtasham Oblokulov, Christopher Akiki, Marc Marone, Chenghao Mou, Mayank Mishra, Alex Gu, Binyuan Hui, Tri Dao, Armel Zebaze, Olivier Dehaene, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sebastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, Harm de Vries
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #25 on Code Generation on MBPP
no code implementations • 23 Feb 2024 • Aditya Desu, Xuanli He, Qiongkai Xu, Wei Lu
As machine- and AI-generated content proliferates, protecting the intellectual property of generative models has become imperative, yet verifying data ownership poses formidable challenges, particularly in cases of unauthorized reuse of generated data.
1 code implementation • 16 Nov 2023 • Jiayi Wang, David Ifeoluwa Adelani, Sweta Agrawal, Marek Masiak, Ricardo Rei, Eleftheria Briakou, Marine Carpuat, Xuanli He, Sofia Bourhim, Andiswa Bukula, Muhidin Mohamed, Temitayo Olatoye, Tosin Adewumi, Hamam Mokayed, Christine Mwase, Wangui Kimotho, Foutse Yuehgoh, Anuoluwapo Aremu, Jessica Ojo, Shamsuddeen Hassan Muhammad, Salomey Osei, Abdul-Hakeem Omotayo, Chiamaka Chukwuneke, Perez Ogayo, Oumaima Hourrane, Salma El Anigri, Lolwethu Ndolela, Thabiso Mangwana, Shafie Abdi Mohamed, Ayinde Hassan, Oluwabusayo Olufunke Awoyomi, Lama Alkhaled, sana al-azzawi, Naome A. Etori, Millicent Ochieng, Clemencia Siro, Samuel Njoroge, Eric Muchiri, Wangari Kimotho, Lyse Naomi Wamba Momo, Daud Abolade, Simbiat Ajao, Iyanuoluwa Shode, Ricky Macharm, Ruqayya Nasir Iro, Saheed S. Abdullahi, Stephen E. Moore, Bernard Opoku, Zainab Akinjobi, Abeeb Afolabi, Nnaemeka Obiefuna, Onyekachi Raphael Ogbu, Sam Brian, Verrah Akinyi Otiende, Chinedu Emmanuel Mbonu, Sakayo Toadoum Sari, Yao Lu, Pontus Stenetorp
Despite the recent progress on scaling multilingual machine translation (MT) to several under-resourced African languages, accurately measuring this progress remains challenging, since evaluation is often performed on n-gram matching metrics such as BLEU, which typically show a weaker correlation with human judgments.
no code implementations • 13 Nov 2023 • Xuanli He, Yuxiang Wu, Oana-Maria Camburu, Pasquale Minervini, Pontus Stenetorp
Recent studies demonstrated that large language models (LLMs) can excel in many tasks via in-context learning (ICL).
no code implementations • 24 Aug 2023 • Maximilian Mozes, Xuanli He, Bennett Kleinberg, Lewis D. Griffin
Spurred by the recent rapid increase in the development and distribution of large language models (LLMs) across industry and academia, much recent work has drawn attention to safety- and security-related threats and vulnerabilities of LLMs, including in the context of potentially criminal activities.
1 code implementation • 14 Aug 2023 • Anthony Colas, Haodi Ma, Xuanli He, Yang Bai, Daisy Zhe Wang
Knowledge Graph (KG)-to-Text Generation has seen recent improvements in generating fluent and informative sentences which describe a given KG.
1 code implementation • 25 May 2023 • Xuanli He, Jun Wang, Benjamin Rubinstein, Trevor Cohn
Backdoor attacks are an insidious security threat against machine learning models.
no code implementations • 22 May 2023 • Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu, Pontus Stenetorp
The burgeoning progress in the field of Large Language Models (LLMs) heralds significant benefits due to their unparalleled capacities.
1 code implementation • 19 May 2023 • Xuanli He, Qiongkai Xu, Jun Wang, Benjamin Rubinstein, Trevor Cohn
Modern NLP models are often trained over large untrusted datasets, raising the potential for a malicious adversary to compromise model behaviour.
no code implementations • 26 Mar 2023 • Thuy-Trang Vu, Xuanli He, Gholamreza Haffari, Ehsan Shareghi
In very recent years more attention has been placed on probing the role of pre-training data in Large Language Models (LLMs) downstream behaviour.
no code implementations • 20 Oct 2022 • Thuy-Trang Vu, Shahram Khadivi, Xuanli He, Dinh Phung, Gholamreza Haffari
Previous works mostly focus on either multilingual or multi-domain aspects of neural machine translation (NMT).
1 code implementation • 15 Sep 2022 • Terry Yue Zhuo, Qiongkai Xu, Xuanli He, Trevor Cohn
Round-trip translation could be served as a clever and straightforward technique to alleviate the requirement of the parallel evaluation corpus.
1 code implementation • 5 Dec 2021 • Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, Chenguang Wang
Nowadays, due to the breakthrough in natural language generation (NLG), including machine translation, document summarization, image captioning, etc NLG models have been encapsulated in cloud APIs to serve over half a billion people worldwide and process over one hundred billion word generations per day.
no code implementations • 30 Oct 2021 • Xuanli He, Iman Keivanloo, Yi Xu, Xiang He, Belinda Zeng, Santosh Rajagopalan, Trishul Chilimbi
To achieve this, we propose a novel idea, Magic Pyramid (MP), to reduce both width-wise and depth-wise computation via token pruning and early exiting for Transformer-based models, particularly BERT.
no code implementations • 29 Sep 2021 • Xuanli He, Islam Nassar, Jamie Ryan Kiros, Gholamreza Haffari, Mohammad Norouzi
To obtain strong task-specific generative models, we either fine-tune a large language model (LLM) on inputs from specific tasks, or prompt a LLM with a few input examples to generate more unlabeled examples.
1 code implementation • EMNLP 2021 • Thuy-Trang Vu, Xuanli He, Dinh Phung, Gholamreza Haffari
Once the in-domain data is detected by the classifier, the NMT model is then adapted to the new domain by jointly learning translation and domain discrimination tasks.
no code implementations • COLING 2022 • Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, Gholamreza Haffari
Machine-learning-as-a-service (MLaaS) has attracted millions of users to their splendid large-scale models.
1 code implementation • 11 Jun 2021 • Xuanli He, Islam Nassar, Jamie Kiros, Gholamreza Haffari, Mohammad Norouzi
This paper studies the use of language models as a source of synthetic unlabeled text for NLP.
no code implementations • 23 May 2021 • Chen Chen, Xuanli He, Lingjuan Lyu, Fangzhao Wu
In this work, we bridge this gap by first presenting an effective model extraction attack, where the adversary can practically steal a BERT-based API (the target/victim model) by only querying a limited number of queries.
1 code implementation • NAACL 2021 • Xuanli He, Lingjuan Lyu, Qiongkai Xu, Lichao Sun
Finally, we investigate two defence strategies to protect the victim model and find that unless the performance of the victim model is sacrificed, both model ex-traction and adversarial transferability can effectively compromise the target models
no code implementations • 1 Jan 2021 • Xuanli He, Lingjuan Lyu, Lichao Sun, Xiaojun Chang, Jun Zhao
We then demonstrate how the extracted model can be exploited to develop effective attribute inference attack to expose sensitive information of the training data.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Xuanli He, Quan Hung Tran, Gholamreza Haffari, Walter Chang, Trung Bui, Zhe Lin, Franck Dernoncourt, Nhan Dam
In this paper, we explore the novel problem of graph modification, where the systems need to learn how to update an existing scene graph given a new user's command.
2 code implementations • Findings of the Association for Computational Linguistics 2020 • Lingjuan Lyu, Xuanli He, Yitong Li
It has been demonstrated that hidden representation learned by a deep model can encode private information of the input, hence can be exploited to recover such information with reasonable accuracy.
no code implementations • 25 Jun 2020 • Lingjuan Lyu, Yitong Li, Xuanli He, Tong Xiao
Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model.
1 code implementation • ACL 2020 • Xuanli He, Gholamreza Haffari, Mohammad Norouzi
This paper introduces Dynamic Programming Encoding (DPE), a new segmentation algorithm for tokenizing sentences into subword units.
no code implementations • ALTA 2018 • Xuanli He, Quan Hung Tran, William Havard, Laurent Besacier, Ingrid Zukerman, Gholamreza Haffari
In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i. e. human transcriptions, instead of Automatic Speech Recognition (ASR)'s transcriptions.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +5
no code implementations • CONLL 2018 • Xuanli He, Gholamreza Haffari, Mohammad Norouzi
In this paper, we develop a novel sequence to sequence mixture (S2SMIX) model that improves both translation diversity and quality by adopting a committee of specialized translation models rather than a single translation model.
no code implementations • WS 2017 • Ekaterina Vylomova, Trevor Cohn, Xuanli He, Gholamreza Haffari
Dealing with the complex word forms in morphologically rich languages is an open problem in language processing, and is particularly important in translation.