Computing Word Similarity on Large-Scale Corpus

Abstract

This paper proposes a novel approach for word similarity computation based on word sense vectors. The word sense vector is built using HIT-IR Tongyici Cilin (extended) for concept generalization and is further modified by the use of relative and absolute frequency filters. Experiments show that the approach not only overcomes the problem of similarity computation of unseen words but also yields a result closer to human judgment when compared to word similarity computation approaches based on dictionaries.