Format-Aware Item Response Theory for Predicting Vocabulary Proficiency

Abstract

Vocabulary proficiency testing plays a vital role in identifying the learner's level of vocabulary knowledge, which can be used to provide personalized materials and feedback in lan-guage-learning applications. Item Response Theory (IRT) is a classical method that can provide interpretable parameters, such as the learner's ability, question discrimination, and question difficulty in many language proficiency testing environments. Many vocabulary proficiency tests include more than one type of question format. However, traditional IRT lacks the ability to tap into the information present within question texts and question formats which can be ideally used to gauge a learner's underlying skills in more detail. In addressing this, we propose a model to reinforce traditional IRT with deep learning to exploit the information hidden within question texts and formats. Experimental results on a sample real-world dataset demonstrate the effectiveness of the proposed model, highlighting that question-related information can be utilized to gauge a learner's proficiency more effectively.