Arabic information retrieval: Stemming or lemmatization?

Abstract

The Arabic language is expanding in the world. According to UNESCO, the Arabic language is spoken by more than 422 million native speakers around 29 countries and among 1.6 billion Muslims worldwide use it to perform their daily prayers. The presence of the Arabic language on the internet grew around 6.091% in the last fifteen years (2000-2015), it is the highest growth of the ten top online languages. Therefore, the number of Arabic documents increases rapidly. This calls for the necessity to improve Arabic Information Retrieval (IR) techniques. Many researchers agree on the benefits of both stemming and lemmatization in IR, primarily with highly inflective languages, short documents and limited space for storing data. The chief purpose of the current study is assessing the impact of stemming and lemmatization on Arabic IR. In this paper, we illustrate several concepts of Arabic morphology, including stemming and lemmatization algorithms. Then, we highlight the use of these latter and their benefits for different Arabic IR systems. Finally, an experiment is conducted to calculate the occurrence of all Quranic surface word, stem, and lemma forms by searching their similarities in both Classical and Modern Standard Arabic resources. In doing so, recent and efficient analyzers AlKhalil Morpho Sys and MADAMIRA are used.