Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing

Mohamed Fakhfakh; Bassem Bouaziz; Hadj Batatia; Lotfi Chaari

doi:10.1007/978-981-16-7618-5_4

Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing

Mohamed Fakhfakh^*, Bassem Bouaziz, Hadj Batatia, Lotfi Chaari

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Artificial neural networks (ANNs) are today the most popular machine learning algorithms. ANNs are widely applied in various fields such as medical imaging and remote sensing. One of the main challenges related to the use of ANNs is the inherent optimization problem to be solved during the training phase. This optimization step is generally performed using a gradient-based approach with a backpropagation strategy. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used to promote sparse networks, this optimization becomes challenging. Classical gradient-based optimizers cannot be used due to differentiability issues. In this paper, we propose an efficient optimization scheme formulated in a Bayesian framework. Hamiltonian dynamics are used to design an efficient sampling scheme. Promising results show the usefulness of the proposed method to allow ANNs with low complexity levels reaching high accuracy rates while performing faster that with other optimizers.

Original language	English
Title of host publication	Proceedings of International Conference on Information Technology and Applications. ICITA 2021
Editors	Abrar Ullah, Steve Gill, Álvaro Rocha, Sajid Anwar
Publisher	Springer
Pages	39-49
Number of pages	11
ISBN (Electronic)	9789811676185
ISBN (Print)	9789811676178
DOIs	https://doi.org/10.1007/978-981-16-7618-5_4
Publication status	Published - 21 Apr 2022
Event	15th International Conference on Information Technology and Applications 2021 - Dubai, United Arab Emirates Duration: 13 Nov 2021 → 14 Nov 2021

Publication series

Name	Lecture Notes in Networks and Systems
Volume	350
ISSN (Print)	2367-3370
ISSN (Electronic)	2367-3389

Conference

Conference	15th International Conference on Information Technology and Applications 2021
Abbreviated title	ICITA 2021
Country/Territory	United Arab Emirates
City	Dubai
Period	13/11/21 → 14/11/21

Keywords

Artificial neural networks
Deep learning
Hamiltonian dynamics
Machine learning
MCMC
Optimization

ASJC Scopus subject areas

Control and Systems Engineering
Signal Processing
Computer Networks and Communications

Access to Document

10.1007/978-981-16-7618-5_4

Cite this

Fakhfakh, M., Bouaziz, B., Batatia, H., & Chaari, L. (2022). Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing. In A. Ullah, S. Gill, Á. Rocha, & S. Anwar (Eds.), Proceedings of International Conference on Information Technology and Applications. ICITA 2021 (pp. 39-49). (Lecture Notes in Networks and Systems; Vol. 350). Springer. https://doi.org/10.1007/978-981-16-7618-5_4

Fakhfakh, Mohamed ; Bouaziz, Bassem ; Batatia, Hadj et al. / Bayesian Optimization for Sparse Artificial Neural Networks : Application to Change Detection in Remote Sensing. Proceedings of International Conference on Information Technology and Applications. ICITA 2021. editor / Abrar Ullah ; Steve Gill ; Álvaro Rocha ; Sajid Anwar. Springer, 2022. pp. 39-49 (Lecture Notes in Networks and Systems).

@inproceedings{7e290bd4999048b8b18d0aaaff0f127c,

title = "Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing",

abstract = "Artificial neural networks (ANNs) are today the most popular machine learning algorithms. ANNs are widely applied in various fields such as medical imaging and remote sensing. One of the main challenges related to the use of ANNs is the inherent optimization problem to be solved during the training phase. This optimization step is generally performed using a gradient-based approach with a backpropagation strategy. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used to promote sparse networks, this optimization becomes challenging. Classical gradient-based optimizers cannot be used due to differentiability issues. In this paper, we propose an efficient optimization scheme formulated in a Bayesian framework. Hamiltonian dynamics are used to design an efficient sampling scheme. Promising results show the usefulness of the proposed method to allow ANNs with low complexity levels reaching high accuracy rates while performing faster that with other optimizers.",

keywords = "Artificial neural networks, Deep learning, Hamiltonian dynamics, Machine learning, MCMC, Optimization",

author = "Mohamed Fakhfakh and Bassem Bouaziz and Hadj Batatia and Lotfi Chaari",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 15th International Conference on Information Technology and Applications 2021, ICITA 2021 ; Conference date: 13-11-2021 Through 14-11-2021",

year = "2022",

month = apr,

day = "21",

doi = "10.1007/978-981-16-7618-5_4",

language = "English",

isbn = "9789811676178",

series = "Lecture Notes in Networks and Systems",

publisher = "Springer",

pages = "39--49",

editor = "Abrar Ullah and Steve Gill and {\'A}lvaro Rocha and Sajid Anwar",

booktitle = "Proceedings of International Conference on Information Technology and Applications. ICITA 2021",

}

Fakhfakh, M, Bouaziz, B, Batatia, H & Chaari, L 2022, Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing. in A Ullah, S Gill, Á Rocha & S Anwar (eds), Proceedings of International Conference on Information Technology and Applications. ICITA 2021. Lecture Notes in Networks and Systems, vol. 350, Springer, pp. 39-49, 15th International Conference on Information Technology and Applications 2021, Dubai, United Arab Emirates, 13/11/21. https://doi.org/10.1007/978-981-16-7618-5_4

Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing. / Fakhfakh, Mohamed; Bouaziz, Bassem; Batatia, Hadj et al.
Proceedings of International Conference on Information Technology and Applications. ICITA 2021. ed. / Abrar Ullah; Steve Gill; Álvaro Rocha; Sajid Anwar. Springer, 2022. p. 39-49 (Lecture Notes in Networks and Systems; Vol. 350).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Bayesian Optimization for Sparse Artificial Neural Networks

T2 - 15th International Conference on Information Technology and Applications 2021

AU - Fakhfakh, Mohamed

AU - Bouaziz, Bassem

AU - Batatia, Hadj

AU - Chaari, Lotfi

PY - 2022/4/21

Y1 - 2022/4/21

N2 - Artificial neural networks (ANNs) are today the most popular machine learning algorithms. ANNs are widely applied in various fields such as medical imaging and remote sensing. One of the main challenges related to the use of ANNs is the inherent optimization problem to be solved during the training phase. This optimization step is generally performed using a gradient-based approach with a backpropagation strategy. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used to promote sparse networks, this optimization becomes challenging. Classical gradient-based optimizers cannot be used due to differentiability issues. In this paper, we propose an efficient optimization scheme formulated in a Bayesian framework. Hamiltonian dynamics are used to design an efficient sampling scheme. Promising results show the usefulness of the proposed method to allow ANNs with low complexity levels reaching high accuracy rates while performing faster that with other optimizers.

AB - Artificial neural networks (ANNs) are today the most popular machine learning algorithms. ANNs are widely applied in various fields such as medical imaging and remote sensing. One of the main challenges related to the use of ANNs is the inherent optimization problem to be solved during the training phase. This optimization step is generally performed using a gradient-based approach with a backpropagation strategy. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used to promote sparse networks, this optimization becomes challenging. Classical gradient-based optimizers cannot be used due to differentiability issues. In this paper, we propose an efficient optimization scheme formulated in a Bayesian framework. Hamiltonian dynamics are used to design an efficient sampling scheme. Promising results show the usefulness of the proposed method to allow ANNs with low complexity levels reaching high accuracy rates while performing faster that with other optimizers.

KW - Artificial neural networks

KW - Deep learning

KW - Hamiltonian dynamics

KW - Machine learning

KW - MCMC

KW - Optimization

UR - http://www.scopus.com/inward/record.url?scp=85129302805&partnerID=8YFLogxK

U2 - 10.1007/978-981-16-7618-5_4

DO - 10.1007/978-981-16-7618-5_4

M3 - Conference contribution

AN - SCOPUS:85129302805

SN - 9789811676178

T3 - Lecture Notes in Networks and Systems

SP - 39

EP - 49

BT - Proceedings of International Conference on Information Technology and Applications. ICITA 2021

A2 - Ullah, Abrar

A2 - Gill, Steve

A2 - Rocha, Álvaro

A2 - Anwar, Sajid

PB - Springer

Y2 - 13 November 2021 through 14 November 2021

ER -

Fakhfakh M, Bouaziz B, Batatia H, Chaari L. Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing. In Ullah A, Gill S, Rocha Á, Anwar S, editors, Proceedings of International Conference on Information Technology and Applications. ICITA 2021. Springer. 2022. p. 39-49. (Lecture Notes in Networks and Systems). doi: 10.1007/978-981-16-7618-5_4

Bayesian Optimization for Sparse Artificial Neural Networks: Application to Change Detection in Remote Sensing

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this