Boosting feature selection

It is possible to reduce the error rate of a single classifier using a classifier ensemble. However, any gain in performance is undermined by the increased computation of performing classification several times. Here the Adaboost_FS algorithm is proposed which builds on two popular areas of ensemble research: Adaboost and Ensemble Feature Selection (EFS). The aim of Adaboost_FSs is to reduce the number of features used by each base classifer and hence the overall computation required by the ensemble. To do this the algorithm combines a regularised version of Boosting Adaboost_Reg [1] with a floating feature search for each base classifier. Adaboost _FS is compared using four benchmark data sets to Adaboost _All, which uses all features and to Adaboost_RSM, which uses a random selection of features. Performance is assessed based on error rate, ensemble error and diversity, and the total number of features used for classification. Results show that Adaboost_FS achieves a lower error rate and higher diversity than Adaboost_All, and achieves a lower error rate and comparable diversity to Adaboost_RSM However, over the other methods Adaboost_FS produces a significant reduction in the number of features required for classification in each base classifier and the entire ensemble. © Springer-Verlag Berlin Heidelberg 2005.