Wrapper Feature Subset Selection for Dimension Reduction Based on Ensemble Learning Algorithm

Abstract

Feature selection is a technique to choose a subset of variables from the multidimensional data which can improve the classification accuracy in diversity datasets. In addition, the best feature subset selection method can reduce the cost of feature measurement. This work focuses on the use of wrapper feature selection. This study use methods of sequential forward selection (SFS), sequential backward selection (SBS) and optimize selection (evolutionary) based on ensemble algorithms namely Bagging and AdaBoost by subset evaluations which are performed using two classifiers; Decision Tree and Naïve Bayes. Thirteen datasets containing different numbers of attributes and dimensions are obtained from the UCI Machine Learning Repository. This study shows that the search technique using SFS based on the bagging algorithm using Decision Tree obtained better results in average accuracy (89.60%) than other methods. The benefits of the feature subset selection are an increased accuracy rate and a reduced run-time when searching multimedia data consisting of a large number of multidimensional datasets.