RelGAN: Relational Generative Adversarial Networks for Text Generation
Generative adversarial networks (GANs) have achieved great success at generating realistic images. However, the text generation still remains a challenging task for modern GAN architectures. In this work, we propose RelGAN, a new GAN architecture for text generation, consisting of three main components: a relational memory based generator for the long-distance dependency modeling, the Gumbel-Softmax relaxation for training GANs on discrete data, and multiple embedded representations in the discriminator to provide a more informative signal for the generator updates. Our experiments show that RelGAN outperforms current state-of-the-art models in terms of sample quality and diversity, and we also reveal via ablation studies that each component of RelGAN contributes critically to its performance improvements. Moreover, a key advantage of our method, that distinguishes it from other GANs, is the ability to control the trade-off between sample quality and diversity via the use of a single adjustable parameter. Finally, RelGAN is the first architecture that makes GANs with Gumbel-Softmax relaxation succeed in generating realistic text.
PDF AbstractCode
Tasks
Datasets
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Text Generation | COCO Captions | RelGAN (100) | BLEU-2 | 0.849 | # 4 | |
BLEU-3 | 0.687 | # 3 | ||||
BLEU-4 | 0.502 | # 4 | ||||
Text Generation | EMNLP2017 WMT | RelGAN | BLEU-2 | 0.881 | # 3 | |
BLEU-3 | 0.705 | # 2 | ||||
BLEU-4 | 0.501 | # 2 | ||||
BLEU-5 | 0.319 | # 5 |