no code implementations • 12 May 2024 • Kareem Ahmed, Stefano Teso, Paolo Morettin, Luca Di Liello, Pierfrancesco Ardino, Jacopo Gobbi, Yitao Liang, Eric Wang, Kai-Wei Chang, Andrea Passerini, Guy Van Den Broeck
We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training by minimizing the network's violation of such dependencies, steering the network towards predicting distributions satisfying the underlying structure.
1 code implementation • 15 Sep 2023 • Luca Di Liello
This thesis focuses on improving the pre-training of natural language models using unsupervised raw data to make them more efficient and aligned with downstream applications.
Ranked #1 on Question Answering on TrecQA (using extra training data)
no code implementations • 24 May 2023 • Luca Di Liello, Siddhant Garg, Alessandro Moschitti
Answer Sentence Selection (AS2) is a core component for building an accurate Question Answering pipeline.
Ranked #4 on Question Answering on TrecQA (using extra training data)
no code implementations • 24 Oct 2022 • Luca Di Liello, Matteo Gabburo, Alessandro Moschitti
In this paper, we study trade-offs between efficiency, cost and accuracy when pre-training Transformer encoders with different pre-training objectives.
no code implementations • 20 May 2022 • Luca Di Liello, Siddhant Garg, Luca Soldaini, Alessandro Moschitti
An important task for designing QA systems is answer sentence selection (AS2): selecting the sentence containing (or constituting) the answer to a question from a set of retrieved relevant documents.
Ranked #1 on Answer Selection on ASNQ
1 code implementation • NAACL 2022 • Luca Di Liello, Siddhant Garg, Luca Soldaini, Alessandro Moschitti
Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks.
Ranked #3 on Fact Verification on FEVER
no code implementations • 20 Apr 2021 • Luca Di Liello, Matteo Gabburo, Alessandro Moschitti
The Transformer architecture deeply changed the natural language processing, outperforming all previous state-of-the-art models.
1 code implementation • NeurIPS 2020 • Luca Di Liello, Pierfrancesco Ardino, Jacopo Gobbi, Paolo Morettin, Stefano Teso, Andrea Passerini
Generative Adversarial Networks (GANs) struggle to generate structured objects like molecules and game maps.