An Immune-Inspired Approach to Macro-Level Neural Ensemble Search

Recent years have seen a renewed interest in evolutionary computation applied to the automatic design of deep neural network architectures, i.e. Neural Architecture Search (NAS). The advantages of evolutionary approaches in NAS include their conceptual simplicity and their flexibility with regards to search space definition and/or optimization objective.However, Artificial Immune Systems (AIS) that follow the evolutionary computation paradigm are less explored in NAS. In this research, we aim to leverage their intrinsic and excellent ability to balance performance and population diversity to develop a novel Neural Ensemble Search method, based on the Clonal Selection Algorithm [1]. For more generality, we focus on designing macro-architectures rather than architectural components.Experiments on popular computer vision benchmarks demonstrate that our method reaches competitive accuracy and efficiency despite minimal augmentation and post-processing. We show that the AIS brings tangible benefits, including maintaining the diversity of solutions, a semantically straightforward implementation, and high efficiency. Moreover, this AIS can exhibit a "secondary response": when presented with a related but more difficult task, the ensemble will perform competently with zero modification to the architectures or the training protocol.