Feature Selection for Modular Neural Network Classifiers

Abstract

An N-class problem can be fully decomposed into N independent small neural networks called modules (or sub-problems) in a modular neural network classifier. Each sub-problem is a two-class (‘yes’ or ‘no’) problem. Hence, the optimal input feature space for each module is also likely to be a subset of the original feature space. Therefore, feature selection plays an important role in finding these useful features. There are some feature selection techniques developed from different perspectives. However, they are not suitable for the two-class problems resulting from complete task decomposition. In this paper, we propose two feature selection techniques – Relative Importance Factor (RIF) and Relative FLD Weight Analysis (RFWA) for modular neural network classifiers. Our approaches involved the use of Fisher’s linear discriminant (FLD) function to obtain the importance of each feature and find out correlation among features. In RIF, the input features are classified as relevant and irrelevant based on their contribution in classification. In RFWA, the irrelevant features are further classified into noise or redundant features based on the correlation among features. The proposed techniques have been applied to several classification problems. The results show that they can successfully detect the irrelevant features in each module and improve accuracy while reducing computation effort.