Losing Heads in the Lottery: Pruning Transformer Attention in Neural Machine Translation

no code implementations • EMNLP 2020 • Maximiliana Behnke, Kenneth Heafield

The attention mechanism is the crucial component of the transformer architecture.

Machine Translation Translation

Efficient Machine Translation with Model Pruning and Quantization

1 code implementation • WMT (EMNLP) 2021 • Maximiliana Behnke, Nikolay Bogoychev, Alham Fikri Aji, Kenneth Heafield, Graeme Nail, Qianqian Zhu, Svetlana Tchistiakova, Jelmer Van der Linde, Pinzhen Chen, Sidharth Kashyap, Roman Grundkiewicz

We participated in all tracks of the WMT 2021 efficient machine translation task: single-core CPU, multi-core CPU, and GPU hardware with throughput and latency conditions.

Decoder Knowledge Distillation +3

Pruning Neural Machine Translation for Speed Using Group Lasso

no code implementations • WMT (EMNLP) 2021 • Maximiliana Behnke, Kenneth Heafield

In the WMT 2021 Efficiency Task, our pruned and quantised models are 1. 9–2. 7x faster at the cost 0. 9–1. 7 BLEU in comparison to the unoptimised baselines.

Decoder Machine Translation +1

Edinburgh's Submissions to the 2020 Machine Translation Efficiency Task

no code implementations • WS 2020 • Nikolay Bogoychev, Roman Grundkiewicz, Alham Fikri Aji, Maximiliana Behnke, Kenneth Heafield, Sidharth Kashyap, Emmanouil-Ioannis Farsarakis, Mateusz Chudyk

We participated in all tracks of the Workshop on Neural Generation and Translation 2020 Efficiency Shared Task: single-core CPU, multi-core CPU, and GPU.

Machine Translation Quantization +1

Improving Machine Translation of Educational Content via Crowdsourcing

no code implementations • LREC 2018 • Maximiliana Behnke, Antonio Valerio Miceli Barone, Rico Sennrich, Vilelmini Sosoni, Thanasis Naskos, Eirini Takoulidou, Maria Stasimioti, Menno van Zaanen, Sheila Castilho, Federico Gaspari, Panayota Georgakopoulou, Valia Kordoni, Markus Egg, Katia Lida Kermanidis

Machine Translation Transfer Learning +1