Document-Level Neural Machine Translation With Document Embeddings

Abstract

Standard neural machine translation (NMT) assumes that document-level context information is irrespective. Most existing document-level NMT methods are satisfied with a smattering sense of shallow document-level information, such as using a few context sentences surrounding the source sentence as document-level information. Our work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings, which can sufficiently model deeper and richer document-level context. The proposed document-level NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end. We compress the entire document text with explicit boundaries into a token-size global static document embedding, and the neighboring sentences as a token-size local dynamic document embedding and concatenate with the source tokens. Experiments reveal that the proposed method significantly improves the translation performance over strong baselines and other related studies.