D2TS: a dual diversity tree selection approach to pruning of random forests

Hani Ragab Hassen*, Yassin Zain Alabdeen, Mohamed Medhat Gaber, Megha Sharma

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
26 Downloads (Pure)

Abstract

Random Forest is one of the most effective classification techniques. It is an ensemble technique with typically decision trees as its classifiers. Each tree votes for an outcome when a new instance is being classified, and a majority vote is taken to decide the final output. Two main factors play an essential role in Random Forests performance, namely diversity among trees in the forest and their number. Higher diversity increases prediction accuracy, whereas lower numbers of trees result in faster predictions. This paper aims at optimizing these two factors by using clustering analysis of trees in order to prune correlated trees while keeping outlier trees to maintain diversity. We group the trees into clusters and only take a number of representatives from each cluster while also keeping some or all of the outliers to preserve diversity. The resulting subset of trees will constitute a random forest of a reduced size. We will use the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm for clustering. DBSCAN is one of the most commonly used clustering techniques and is immune to outliers. We will use DBSCAN to (i) group trees in clusters based on their prediction behaviour and (ii) identify outliers. Each of the clustered and outlier trees bring an element of diversity into the pruned random forest, thus giving our approach its dual diversity aspect. Our approach achieved up to a 99% pruning level while resulting in similar, or even better, accuracy compared to the original forests for 19 public datasets with varying properties. Our source code is publicly available on GitHub.

Original languageEnglish
Pages (from-to)467-481
Number of pages15
JournalInternational Journal of Machine Learning and Cybernetics
Volume14
Issue number2
Early online date15 Sept 2022
DOIs
Publication statusPublished - Feb 2023

Keywords

  • DBSCAN
  • Pruning
  • Random forests
  • Similarity scores

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'D2TS: a dual diversity tree selection approach to pruning of random forests'. Together they form a unique fingerprint.

Cite this