A Generative Adversarial Constraint Encoder-Decoder Model for the Text Summarization

Abstract

As a new method of training generative models, Generative Adversarial Net(GAN) has problems when it is applied to the summary generator to generate discrete tokens. This paper considers introducing GAN into the Encoder stage and proposes a new framework EDA(Encoder-Decoder with Adversarial training) for text summarization by combining adversarial training into the traditional Encoder-Decoder architecture. We construct a new training loop and an objective function in this framework to optimize the encoder and decoder. EDA uses GAN to learn the alignment between the encoder representation and the target summary representation to improve the encoding quality and generate more accurate summary by using semantic distance and fine-tuning with decoded summary. It simultaneously alleviates problems of discrete data processing and conditional generation. In particularly, we introduce keyword information of the original document into attention mechanism so that more informative representation and summary can be generated. The results on LCSTS dataset show that our method improves the performance and robustness of Encoder-Decoder model applied in text summarization.