A Variational Inequality Model for Learning Neural Networks

Patrick Louis Combettes; Jean-Christophe Pesquet; Audrey Repetti

doi:10.1109/icassp49357.2023.10095688

A Variational Inequality Model for Learning Neural Networks

Patrick Louis Combettes, Jean-Christophe Pesquet, Audrey Repetti

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

45 Downloads (Pure)

Abstract

Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training the layers of a neural network is a challenging task in many applications. The prevalent training procedure consists of minimizing highly non-convex objectives based on data sets of huge dimension. In this context, current methodologies are not guaranteed to produce global solutions. We present an alternative approach which foregoes the optimization framework and adopts a variational inequality formalism. The associated algorithm guarantees convergence of the iterates to a true solution of the variational inequality and it possesses an efficient block-iterative structure. A numerical application is presented.

Original language	English
Title of host publication	2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	IEEE
ISBN (Electronic)	9781728163277
DOIs	https://doi.org/10.1109/icassp49357.2023.10095688
Publication status	Published - 5 May 2023
Event	48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023 - Rhodes Island, Rhodes Island, Greece Duration: 4 Jun 2023 → 10 Jun 2023 Conference number: 48 https://2023.ieeeicassp.org/ https://2023.ieeeicassp.org

Conference

Conference	48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023
Abbreviated title	ICASSP 2023
Country/Territory	Greece
City	Rhodes Island
Period	4/06/23 → 10/06/23
Internet address	https://2023.ieeeicassp.org/ https://2023.ieeeicassp.org

Keywords

Block-iterative algorithm
MRI
neural networks
transfer learning
variational inequality

ASJC Scopus subject areas

Software
Signal Processing
Electrical and Electronic Engineering

Access to Document

10.1109/icassp49357.2023.10095688

Download pdf
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 428 KB

Cite this

@inproceedings{89b2610b608d4896a83e961b6c5aecb5,

title = "A Variational Inequality Model for Learning Neural Networks",

abstract = "Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training the layers of a neural network is a challenging task in many applications. The prevalent training procedure consists of minimizing highly non-convex objectives based on data sets of huge dimension. In this context, current methodologies are not guaranteed to produce global solutions. We present an alternative approach which foregoes the optimization framework and adopts a variational inequality formalism. The associated algorithm guarantees convergence of the iterates to a true solution of the variational inequality and it possesses an efficient block-iterative structure. A numerical application is presented.",

keywords = "Block-iterative algorithm, MRI, neural networks, transfer learning, variational inequality",

author = "Combettes, {Patrick Louis} and Jean-Christophe Pesquet and Audrey Repetti",

year = "2023",

month = may,

day = "5",

doi = "10.1109/icassp49357.2023.10095688",

language = "English",

booktitle = "2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

publisher = "IEEE",

address = "United States",

note = "48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023, ICASSP 2023 ; Conference date: 04-06-2023 Through 10-06-2023",

url = "https://2023.ieeeicassp.org/, https://2023.ieeeicassp.org",

}

Combettes, PL, Pesquet, J-C & Repetti, A 2023, A Variational Inequality Model for Learning Neural Networks. in 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)., 10095688, IEEE, 48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023, Rhodes Island, Greece, 4/06/23. https://doi.org/10.1109/icassp49357.2023.10095688

TY - GEN

T1 - A Variational Inequality Model for Learning Neural Networks

AU - Combettes, Patrick Louis

AU - Pesquet, Jean-Christophe

AU - Repetti, Audrey

N1 - Conference code: 48

PY - 2023/5/5

Y1 - 2023/5/5

N2 - Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training the layers of a neural network is a challenging task in many applications. The prevalent training procedure consists of minimizing highly non-convex objectives based on data sets of huge dimension. In this context, current methodologies are not guaranteed to produce global solutions. We present an alternative approach which foregoes the optimization framework and adopts a variational inequality formalism. The associated algorithm guarantees convergence of the iterates to a true solution of the variational inequality and it possesses an efficient block-iterative structure. A numerical application is presented.

AB - Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training the layers of a neural network is a challenging task in many applications. The prevalent training procedure consists of minimizing highly non-convex objectives based on data sets of huge dimension. In this context, current methodologies are not guaranteed to produce global solutions. We present an alternative approach which foregoes the optimization framework and adopts a variational inequality formalism. The associated algorithm guarantees convergence of the iterates to a true solution of the variational inequality and it possesses an efficient block-iterative structure. A numerical application is presented.

KW - Block-iterative algorithm

KW - MRI

KW - neural networks

KW - transfer learning

KW - variational inequality

UR - http://www.scopus.com/inward/record.url?scp=85177578432&partnerID=8YFLogxK

U2 - 10.1109/icassp49357.2023.10095688

DO - 10.1109/icassp49357.2023.10095688

M3 - Conference contribution

BT - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

T2 - 48th IEEE International Conference on Acoustics, Speech and Signal Processing 2023

Y2 - 4 June 2023 through 10 June 2023

ER -

A Variational Inequality Model for Learning Neural Networks

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this