Predicting Machine Translation Adequacy with Document Embeddings

Abstract

This paper describes USAAR's submission to the the metrics shared task of the Workshop on Statistical Machine Translation (WMT) in 2015.The goal of our submission is to take advantage of the semantic overlap between hypothesis and reference translation for predicting MT output adequacy using language independent document embeddings.The approach presented here is learning a Bayesian Ridge Regressor using document skip-gram embeddings in order to automatically evaluate Machine Translation (MT) output by predicting semantic adequacy scores.The evaluation of our submission -measured by the correlation with human judgements -shows promising results on system-level scores.