Feature Selection with Dynamic Classifier Ensembles

Abstract

With the advance in technology, the volume of available data grows massively. Therefore, feature selection has become an essential preprocessing step to extract valuable information. Feature selection is the task of reducing the number of features by removing redundant features from data while preserving the classification accuracy. It is a multiobjective problem as there are two objectives. In general, multiobjective selection algorithms with machine learning techniques are utilized to find the most promising feature subsets; however, classification performances of these machine learning techniques are analyzed separately. In this study, we propose a new multiobjective selection model that dynamically searches for the best ensemble of five classifiers to extract the best representative feature subsets. We present the experiment results on 12 well-known datasets. The results show that the proposed method performs significantly better than all the machine learning techniques when they are executed separately. Moreover, the proposed method outperforms two existing ensemble algorithms, namely AdaBoost and Gradient Boosting.