Unsupervised Representative Feature Selection Algorithm Based on Information Entropy and Relevance Analysis

Abstract

Feature selection plays an important role in preprocessing in pattern recognition and data mining, especially in large scale image, digital text, and biological data. Specifically, class label information is unavailable for conducting the selection of a minimal feature subset in unsupervised learning, which is full of challenging and interesting problems. In this paper, we present an unsupervised REPresentative Feature Selection (REPFS) algorithm based on information entropy and relevance analysis. The proposed method seeks to find a high-quality feature subset through feature clustering without using any learning algorithms. More importantly, the features' relevance will be computed based on an information metric of the relevance gain, which provides an information theoretical foundation for finding a minimum of the redundancy between features. Our results on nine benchmark data sets demonstrate that REPFS can significantly improve upon state-of-the-art unsupervised algorithms.