Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing

Mohamed Fakhfakh*, Bassem Bouaziz, Hadj Batatia, Lotfi Chaari

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Artificial neural networks (ANNs) are today the most popular machine learning algorithms. ANNs are widely applied in various fields such as medical imaging and remote sensing. One of the main challenges related to the use of ANNs is the inherent optimization problem to be solved during the training phase. This optimization step is generally performed using a gradient-based approach with a backpropagation strategy. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used to promote sparse networks, this optimization becomes challenging. Classical gradient-based optimizers cannot be used due to differentiability issues. In this paper, we propose an efficient optimization scheme formulated in a Bayesian framework. Hamiltonian dynamics are used to design an efficient sampling scheme. Promising results show the usefulness of the proposed method to allow ANNs with low complexity levels reaching high accuracy rates while performing faster that with other optimizers.

Original languageEnglish
Title of host publicationProceedings of International Conference on Information Technology and Applications. ICITA 2021
EditorsAbrar Ullah, Steve Gill, Álvaro Rocha, Sajid Anwar
PublisherSpringer
Pages39-49
Number of pages11
ISBN (Electronic)9789811676185
ISBN (Print)9789811676178
DOIs
Publication statusPublished - 21 Apr 2022
Event15th International Conference on Information Technology and Applications 2021 - Dubai, United Arab Emirates
Duration: 13 Nov 202114 Nov 2021

Publication series

NameLecture Notes in Networks and Systems
Volume350
ISSN (Print)2367-3370
ISSN (Electronic)2367-3389

Conference

Conference15th International Conference on Information Technology and Applications 2021
Abbreviated titleICITA 2021
Country/TerritoryUnited Arab Emirates
CityDubai
Period13/11/2114/11/21

Keywords

  • Artificial neural networks
  • Deep learning
  • Hamiltonian dynamics
  • Machine learning
  • MCMC
  • Optimization

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing'. Together they form a unique fingerprint.

Cite this