Chinese-English machine translation model based on transfer learning and self-attention

Abstract

With the continuous development of machine learning and neural networks, neural machine translation (NMT) has been widely used due to its strong translation ability. Lexical information is overused in the construction of the internal nodes that make up the structure. Using phrase structure encoders can lead to over-translation problems. In addition, the number of model parameters increases with the use of grammatical structures, and the phrase nodes may not always be beneficial to the neural translation model. Therefore, we propose a novel Chinese-English machine translation model based on transfer learning and self-attention. In order to make use of the position information between words, the absolute position information of words is represented by sine-cosine position encoding in the machine translation model based on self-attention mechanism. However, while this method can reflect relative distance, it lacks direction. In this paper, a new machine translation model is proposed by combining transfer learning with self-attention mechanism. This model not only inherits the high efficiency of self-attention mechanism, but also preserves the distance information and direction information between words. The results of translation experiments show that the proposed transfer learning model is significantly better than the traditional tree model.