no code implementations • 3 May 2024 • Guillem Ramírez, Alexandra Birch, Ivan Titov
Either a cascading strategy is used, where a smaller LLM or both are called sequentially, or a routing strategy is used, where only one model is ever called.
no code implementations • 26 Jan 2024 • Masaru Isonuma, Ivan Titov
This paper presents UnTrac, which estimates the influence of a training dataset by unlearning it from the trained model.
no code implementations • 5 Dec 2023 • Xinnuo Xu, Ivan Titov, Mirella Lapata
Data-to-text generation involves transforming structured data, often represented as predicate-argument tuples, into coherent textual descriptions.
1 code implementation • 16 Nov 2023 • Maike Züfle, Verna Dankers, Ivan Titov
We challenge hate speech models via new train-test splits of existing datasets that rely on the clustering of models' hidden representations.
no code implementations • 9 Nov 2023 • Verna Dankers, Ivan Titov, Dieuwke Hupkes
When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others.
no code implementations • 25 Oct 2023 • Max Müller-Eberstein, Rob van der Goot, Barbara Plank, Ivan Titov
We identify critical learning phases across tasks and time, during which subspaces emerge, share information, and later disentangle to specialize.
2 code implementations • 23 Oct 2023 • Danis Alukaev, Semen Kiselev, Ilya Pershin, Bulat Ibragimov, Vladimir Ivanov, Alexey Kornaev, Ivan Titov
Concept Bottleneck Models (CBMs) assume that training examples (e. g., x-ray images) are annotated with high-level concepts (e. g., types of abnormalities), and perform classification by first predicting the concepts, followed by predicting the label relying on these concepts.
1 code implementation • 21 Oct 2023 • Yanpeng Zhao, Ivan Titov
We consider a zero-shot transfer learning setting where a model is trained on the source domain and is directly applied to target domains, without any further training.
no code implementations • 20 Oct 2023 • Guillem Ramírez, Matthias Lindemann, Alexandra Birch, Ivan Titov
To curtail the frequency of these calls, one can employ a smaller language model -- a student -- which is continuously trained on the responses of the LLM.
no code implementations • 1 Oct 2023 • Matthias Lindemann, Alexander Koller, Ivan Titov
Strong inductive biases enable learning from little data and help generalization outside of the training distribution.
1 code implementation • 29 May 2023 • Victor Prokhorov, Ivan Titov, N. Siddharth
Conditional neural processes (CNPs) are a flexible and efficient family of models that learn to learn a stochastic process from data.
1 code implementation • 26 May 2023 • Matthias Lindemann, Alexander Koller, Ivan Titov
Our model outperforms pretrained seq2seq models and prior work on realistic semantic parsing tasks that require generalization to longer examples.
1 code implementation • 31 Jan 2023 • Verna Dankers, Ivan Titov
We illustrate that comparing data's representations in models with and without the bottleneck can be used to produce a compositionality metric.
1 code implementation • 15 Nov 2022 • Bailin Wang, Ivan Titov, Jacob Andreas, Yoon Kim
We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference.
1 code implementation • 6 Oct 2022 • Matthias Lindemann, Alexander Koller, Ivan Titov
Seq2seq models have been shown to struggle with compositional generalisation, i. e. generalising to new and potentially more complex structures than seen during training.
1 code implementation • ACL 2022 • Verna Dankers, Christopher G. Lucas, Ivan Titov
In this work, we investigate whether the non-compositionality of idioms is reflected in the mechanics of the dominant NMT model, Transformer, by analysing the hidden states and attention patterns for models with English as source language and one of seven European languages as target language.
no code implementations • 13 Dec 2021 • Nicola De Cao, Leon Schmid, Dieuwke Hupkes, Ivan Titov
Typically, interpretation methods i) do not guarantee that the model actually uses the encoded information, and ii) do not discover small subsets of neurons responsible for a considered phenomenon.
1 code implementation • EMNLP 2021 • Arthur Bražinskas, Mirella Lapata, Ivan Titov
Opinion summarization has been traditionally approached with unsupervised, weakly-supervised and few-shot learning techniques.
1 code implementation • EMNLP 2021 • Nicola De Cao, Wilker Aziz, Ivan Titov
Generative approaches have been recently shown to be effective for both Entity Disambiguation and Entity Linking (i. e., joint mention detection and disambiguation).
Ranked #5 on Entity Linking on AIDA-CoNLL
no code implementations • EMNLP 2021 • Elena Voita, Rico Sennrich, Ivan Titov
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process.
1 code implementation • ACL 2021 • Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich
Document-level contextual information has shown benefits to text-based machine translation, but whether and how context helps end-to-end (E2E) speech translation (ST) is still under-studied.
1 code implementation • Findings (ACL) 2021 • Christos Baziotis, Ivan Titov, Alexandra Birch, Barry Haddow
Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT), by drastically reducing the need for large parallel data.
1 code implementation • ACL 2021 • Henry Conklin, Bailin Wang, Kenny Smith, Ivan Titov
Natural language is compositional; the meaning of a sentence is a function of the meaning of its parts.
1 code implementation • NeurIPS 2021 • Bailin Wang, Mirella Lapata, Ivan Titov
Despite success in many domains, neural models struggle in settings where train and test examples are drawn from different distributions.
3 code implementations • EMNLP 2021 • Nicola De Cao, Wilker Aziz, Ivan Titov
We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive re-training or fine-tuning.
3 code implementations • EMNLP 2021 • Biao Zhang, Ivan Titov, Rico Sennrich
Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants.
1 code implementation • NAACL 2021 • Bailin Wang, Mirella Lapata, Ivan Titov
Based on the observation that programs which correspond to NL utterances must be always executable, we propose to encourage a parser to generate executable programs for unlabeled utterances.
2 code implementations • EACL (AdaptNLP) 2021 • Yanpeng Zhao, Ivan Titov
Compound probabilistic context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction.
no code implementations • 23 Nov 2020 • Artem Malyeyev, Ivan Titov, Philipp Bender, Mathias Bersweiler, Vitaliy Pipich, Sebastian Mühlbauer, Semih Ener, Oliver Gutfleisch, Andreas Michels
We report the results of an unpolarized small-angle neutron scattering (SANS) study on Mn-Bi-based rare-earth-free permanent magnets.
Materials Science
1 code implementation • EMNLP 2020 • Denis Emelin, Ivan Titov, Rico Sennrich
Word sense disambiguation is a well-known source of translation errors in NMT.
1 code implementation • WMT (EMNLP) 2020 • Biao Zhang, Ivan Titov, Rico Sennrich
Instead of assuming independence between neighbouring tokens (semi-autoregressive decoding, SA), we take inspiration from bidirectional sequence generation and introduce a decoder that generates target words from the left-to-right and right-to-left directions simultaneously.
no code implementations • EMNLP 2021 • Chunchuan Lyu, Shay B. Cohen, Ivan Titov
In contrast, we treat both alignment and segmentation as latent variables in our model and induce them as part of end-to-end training.
Ranked #22 on AMR Parsing on LDC2017T10
no code implementations • NAACL 2021 • Bailin Wang, Mirella Lapata, Ivan Titov
The importance of building semantic parsers which can be applied to new domains and generate programs unseen at training has long been acknowledged, and datasets testing out-of-domain performance are becoming increasingly available.
1 code implementation • ACL 2021 • Elena Voita, Rico Sennrich, Ivan Titov
We find that models trained with more data tend to rely on source information more and to have more sharp token contributions; the training process is non-monotonic with several stages of different nature.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich
Information in speech signals is not evenly distributed, making it an additional challenge for end-to-end (E2E) speech translation (ST) to learn to focus on informative features.
1 code implementation • ICLR 2021 • Michael Sejr Schlichtkrull, Nicola De Cao, Ivan Titov
In this work, we introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges.
1 code implementation • EMNLP 2020 • Yanpeng Zhao, Ivan Titov
In this work, we study visually grounded grammar induction and learn a constituency parser from both unlabeled text and its visual groundings.
1 code implementation • 1 May 2020 • Yanpeng Zhao, Ivan Titov
Nominal roles are not labeled in the training data, and the learning objective instead pushes the labeler to assign roles predictive of the arguments.
2 code implementations • EMNLP 2020 • Nicola De Cao, Michael Schlichtkrull, Wilker Aziz, Ivan Titov
Attribution methods assess the contribution of inputs to the model prediction.
1 code implementation • EMNLP 2020 • Arthur Bražinskas, Mirella Lapata, Ivan Titov
In this work, we show that even a handful of summaries is sufficient to bootstrap generation of the summary text with all expected properties, such as writing style, informativeness, fluency, and sentiment preservation.
no code implementations • 30 Apr 2020 • Serhii Havrylov, Ivan Titov
Variational autoencoders (VAEs) are a standard framework for inducing latent variable models that have been shown effective in learning text representations as well as in text generation.
3 code implementations • ACL 2020 • Biao Zhang, Philip Williams, Ivan Titov, Rico Sennrich
Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations.
1 code implementation • Findings (ACL) 2021 • Biao Zhang, Ivan Titov, Rico Sennrich
Inspired by these observations, we explore the feasibility of specifying rule-based patterns that mask out encoder outputs based on information such as part-of-speech tags, word frequency and word position.
2 code implementations • EMNLP 2020 • Elena Voita, Ivan Titov
Instead, we propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL).
3 code implementations • ACL 2020 • Arthur Bražinskas, Mirella Lapata, Ivan Titov
At test time, when generating summaries, we force the novelty to be minimal, and produce a text reflecting consensus opinions.
no code implementations • 11 Oct 2019 • Shangmin Guo, Yi Ren, Serhii Havrylov, Stella Frank, Ivan Titov, Kenny Smith
Since first introduced, computer simulation has been an increasingly important tool in evolutionary linguistics.
1 code implementation • IJCNLP 2019 • Xinchi Chen, Chunchuan Lyu, Ivan Titov
In every network layer, the capsules interact with each other and with representations of words in the sentence.
1 code implementation • EMNLP 2020 • Diego Marcheggiani, Ivan Titov
Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles.
1 code implementation • IJCNLP 2019 • Bailin Wang, Ivan Titov, Mirella Lapata
Semantic parsing aims to map natural language utterances onto machine interpretable meaning representations, aka programs whose execution against a real-world environment produces a denotation.
Ranked #13 on Semantic Parsing on WikiTableQuestions
1 code implementation • IJCNLP 2019 • Chunchuan Lyu, Shay B. Cohen, Ivan Titov
Modern state-of-the-art Semantic Role Labeling (SRL) methods rely on expressive sentence encoders (e. g., multi-layer LSTMs) but tend to model only local (if any) interactions between individual argument labeling decisions.
no code implementations • IJCNLP 2019 • Elena Voita, Rico Sennrich, Ivan Titov
In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective.
1 code implementation • IJCNLP 2019 • Elena Voita, Rico Sennrich, Ivan Titov
For training, the DocRepair model requires only monolingual document-level data in the target language.
1 code implementation • IJCNLP 2019 • Biao Zhang, Ivan Titov, Rico Sennrich
The general trend in NLP is towards increasing model capacity and performance via deeper neural networks.
1 code implementation • WS 2019 • Denis Emelin, Ivan Titov, Rico Sennrich
The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context.
1 code implementation • ACL 2019 • Caio Corro, Ivan Titov
We treat projective dependency trees as latent variables in our probabilistic model and induce them in such a way as to be beneficial for a downstream task, without relying on any direct tree supervision.
1 code implementation • ACL 2019 • Phong Le, Ivan Titov
First, we construct a high recall list of candidate entities for each mention in an unlabeled document.
Ranked #17 on Entity Disambiguation on AIDA-CoNLL
1 code implementation • NAACL 2019 • Yang Liu, Ivan Titov, Mirella Lapata
In this paper, we conceptualize single-document extractive summarization as a tree induction problem.
1 code implementation • ACL 2019 • Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, Ivan Titov
Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation.
1 code implementation • ACL 2019 • Jasmijn Bastings, Wilker Aziz, Ivan Titov
The success of neural networks comes hand in hand with a desire for more interpretability.
1 code implementation • ACL 2019 • Phong Le, Ivan Titov
As the learning signal is weak and our surrogate labels are noisy, we introduce a noise detection component in our model: it lets the model detect and disregard examples which are likely to be noisy.
1 code implementation • ACL 2019 • Elena Voita, Rico Sennrich, Ivan Titov
Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems.
1 code implementation • WS 2020 • Zhifeng Hu, Serhii Havrylov, Ivan Titov, Shay B. Cohen
We introduce an idea for a privacy-preserving transformation on natural language data, inspired by homomorphic encryption.
4 code implementations • 9 Apr 2019 • Nicola De Cao, Ivan Titov, Wilker Aziz
Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions.
Ranked #1 on Density Estimation on UCI MINIBOONE
no code implementations • 18 Jan 2019 • Jasmijn Bastings, Wilker Aziz, Ivan Titov, Khalil Sima'an
Recently it was shown that linguistic structure predicted by a supervised parser can be beneficial for neural machine translation (NMT).
1 code implementation • NAACL 2019 • Nicola De Cao, Wilker Aziz, Ivan Titov
Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs.
no code implementations • ICLR 2019 • Caio Corro, Ivan Titov
Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages.
no code implementations • ACL 2018 • Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov
Standard machine translation systems process sentences in isolation and hence ignore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and improve translation coherence.
2 code implementations • ACL 2018 • Chunchuan Lyu, Ivan Titov
AMR parsing is challenging partly due to the lack of annotated alignments between nodes in the graphs and words in the corresponding sentences.
Ranked #1 on AMR Parsing on LDC2015E86
2 code implementations • ACL 2018 • Phong Le, Ivan Titov
Entity linking involves aligning textual mentions of named entities to their corresponding entries in a knowledge base.
no code implementations • NAACL 2018 • Diego Marcheggiani, Jasmijn Bastings, Ivan Titov
Semantic representations have long been argued as potentially useful for enforcing meaning preservation and improving generalization performance of machine translation methods.
Ranked #8 on Machine Translation on WMT2016 English-German
1 code implementation • COLING 2018 • Arthur Bražinskas, Serhii Havrylov, Ivan Titov
Rather than assuming that a word embedding is fixed across the entire text collection, as in standard word embedding methods, in our Bayesian model we generate it from a word-specific prior density for each occurrence of a given word.
no code implementations • NeurIPS 2017 • Serhii Havrylov, Ivan Titov
Learning to communicate through interaction, rather than relying on explicit supervision, is often considered a prerequisite for developing a general AI.
no code implementations • EMNLP 2017 • Jasmijn Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, Khalil Sima'an
We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation.
1 code implementation • CONLL 2017 • Phong Le, Ivan Titov
Coreference evaluation metrics are hard to optimize directly as they are non-differentiable functions, not easily decomposable into elementary decisions.
27 code implementations • 17 Mar 2017 • Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, Max Welling
We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification.
Ranked #1 on Node Classification on AIFB
2 code implementations • EMNLP 2017 • Diego Marcheggiani, Ivan Titov
GCNs over syntactic dependency trees are used as sentence encoders, producing latent feature representations of words in a sentence.
Ranked #2 on Chinese Semantic Role Labeling on CoNLL-2009
no code implementations • TACL 2017 • Ashutosh Modi, Ivan Titov, Vera Demberg, Asad Sayeed, Manfred Pinkal
Recent research in psycholinguistics has provided increasing evidence that humans predict upcoming content.
2 code implementations • CONLL 2017 • Diego Marcheggiani, Anton Frolov, Ivan Titov
However, when automatically predicted part-of-speech tags are provided as input, it substantially outperforms all previous local models and approaches the best reported results on the English CoNLL-2009 dataset.
1 code implementation • NAACL 2016 • Simon Šuster, Ivan Titov, Gertjan van Noord
We present an approach to learning multi-sense word embeddings relying both on monolingual and bilingual information.
1 code implementation • TACL 2016 • Diego Marcheggiani, Ivan Titov
We present a method for unsupervised open-domain relation discovery.
1 code implementation • TACL 2016 • Hoang Cuong, Khalil Sima{'}an, Ivan Titov
Existing work on domain adaptation for statistical machine translation has consistently assumed access to a small sample from the test distribution (target domain) at training time.
1 code implementation • 31 Aug 2015 • Simon Šuster, Gertjan van Noord, Ivan Titov
Word representations induced from models with discrete latent variables (e. g.\ HMMs) have been shown to be beneficial in many NLP applications.
no code implementations • 19 Dec 2014 • Ivan Titov, Ehsan Khoddam
In this work, we propose a new method to integrate two recent lines of work: unsupervised induction of shallow semantics (e. g., semantic roles) and factorization of relations in text and knowledge bases.
no code implementations • HLT 2015 • Ivan Titov, Ehsan Khoddam
We introduce a new approach to unsupervised estimation of feature-rich semantic role labeling models.
no code implementations • 18 Dec 2013 • Ashutosh Modi, Ivan Titov
Induction of common sense knowledge about prototypical sequences of events has recently received much attention.