Zero-Shot Cross-Lingual Neural Headline Generation

Abstract

Neural headline generation (NHG) has been proven to be effective in generating a fully abstractive headline recently. Existing NHG systems are only capable of producing headline of the same language as the original document. Cross lingual headline generation is an important task since it provides an efficient way to understand the key point of a document in a different language. Due to the lack of those parallel corpora of direct source language articles and target language headlines, we propose to deal with the cross-lingual neural headline generation (CNHG) under the zero-shot scenario. A trivial solution is to translate and summarize the source document in a pipeline way. However, a pipeline solution will lead to error propagation in the translation and summarization phases. This challenge motivates us to build a direct source-to-target CNHG model based on existing parallel corpora of translation and monolingual headline generation. Specifically, we let a parameterized CNHG model (student model) mimic the output of a pretrained translation or headline generation model (teacher model). To the best of our knowledge, this is the first effort to address CNHG problem. Besides, we construct English-Chinese headline generation evaluation datasets by manual translation. Experimental results on English-to-Chinese cross-lingual headline generation demonstrate that our proposed method significantly outperforms the baseline models.