Supervised Feature Selection With a Stratified Feature Weighting Method

Abstract

Feature selection has been a powerful tool to handle high-dimensional data. Most of these methods are biased toward the highest rank features which may be highly correlated with each other. In this paper, we address this problem proposing stratified feature ranking (SFR) method for supervised feature ranking of high-dimensional data. Given a dataset with class labels, we first propose a subspace feature clustering (SFC) to simultaneously identify feature clusters and the importance of each feature for each class. In the SFR method, the features in different feature clusters are separately ranked according to the subspace weight produced by SFC. After that, we propose a stratified feature weighting method for ranking the features such that the high rank features are both informative and diverse. We have conducted a series of experiments to verify the effectiveness and scalability of SFC for feature clustering. The proposed SFR method was compared with six feature selection methods on a set of high-dimensional datasets and the results show that SFR was superior to most of these feature selection methods.