Information Gain Based Feature Selection for Improved Textual Sentiment Analysis

Abstract

Sentiment analysis or opinion mining is the process of mining the emotion from a given text. It is a text mining technique that effectively measures the inclination of public opinions and aids in analysing the subjective information from the given context. Sentiment analysis evaluates the opinion of a sentiment as either positive or negative or neutral. Sentiments are very specific and with respect to the underlying content, it plays a very crucial role in depicting the real-world scenario. Sentiment analysis can be performed at three levels namely document level, sentence level and feature level. This paper proposes a novel Information Gain based Feature Selection algorithm that selects highly correlated features by removing inappropriate content. Using this algorithm, extensive sentimental analysis is performed at the document level, sentence level and feature level. Datasets from Cornell and Kaggle are exploited for experimental purposes. Compared to other baseline classifiers experimental results show that the proposed Information Gain based classifier resulted in an accuracy of 95, 96.3 and 97.4% for document, sentence and feature levels respectively. The proposed method is also tested with higher dimensional datasets namely Movielens 1M, 10M and 25M datasets. Experimental results proved that the proposed method works better even for high dimensional datasets.