TY - GEN
T1 - DeepSwarm
T2 - Optimising Convolutional Neural Networks using Swarm Intelligence
AU - Byla, Edvinas
AU - Pang, Wei
PY - 2019/8/30
Y1 - 2019/8/30
N2 - In this paper we propose DeepSwarm, a novel neural architecture search (NAS) method based on Swarm Intelligence principles. At its core DeepSwarm uses Ant Colony Optimization (ACO) to generate ant population which uses the pheromone information to collectively search for the best neural architecture. Furthermore, by using local and global pheromone update rules our method ensures the balance between exploitation and exploration. On top of this, to make our method more efficient we combine progressive neural architecture search with weight reusability. Furthermore, due to the nature of ACO our method can incorporate heuristic information which can further speed up the search process. After systematic and extensive evaluation, we discover that on three different datasets (MNIST, Fashion-MNIST, and CIFAR-10) when compared to existing systems our proposed method demonstrates competitive performance. Finally, we open source DeepSwarm (https://github.com/Pattio/DeepSwarm) as a NAS library and hope it can be used by more deep learning researchers and practitioners.
AB - In this paper we propose DeepSwarm, a novel neural architecture search (NAS) method based on Swarm Intelligence principles. At its core DeepSwarm uses Ant Colony Optimization (ACO) to generate ant population which uses the pheromone information to collectively search for the best neural architecture. Furthermore, by using local and global pheromone update rules our method ensures the balance between exploitation and exploration. On top of this, to make our method more efficient we combine progressive neural architecture search with weight reusability. Furthermore, due to the nature of ACO our method can incorporate heuristic information which can further speed up the search process. After systematic and extensive evaluation, we discover that on three different datasets (MNIST, Fashion-MNIST, and CIFAR-10) when compared to existing systems our proposed method demonstrates competitive performance. Finally, we open source DeepSwarm (https://github.com/Pattio/DeepSwarm) as a NAS library and hope it can be used by more deep learning researchers and practitioners.
KW - Ant Colony Optimization
KW - Neural Architecture Search
U2 - 10.1007/978-3-030-29933-0_10
DO - 10.1007/978-3-030-29933-0_10
M3 - Conference contribution
SN - 9783030299323
T3 - Advances in Intelligent Systems and Computing
SP - 119
EP - 130
BT - Advances in Computational Intelligence Systems. UKCI 2019
A2 - Ju, Zhaojie
A2 - Yang, Longzhi
A2 - Yang, Chenguang
A2 - Gegov, Alexander
A2 - Zhou, Dalin
PB - Springer
ER -