Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems

Dan Yao; Stephen McLaughlin; Yoann Altmann

doi:10.1109/SSPD54131.2022.9896226

Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems

Dan Yao, Stephen McLaughlin, Yoann Altmann

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper addresses the estimation of large-scale sparse coefficients from noisy linear measurements using Expectation Propagation (EP) method for unsupervised approximate Bayesian inference. In the Bayesian model, the Laplace prior, Mixture of two Gaussians (MoG2) prior, and Spike-and-Slab (SaS) prior are adopted respectively as the sparsity-promoting priors of the unknown sparse parameter. In solving high-dimensional linear inverse problems, the proposed EP method directly provides the approximate minimum mean squared error (MMSE) estimate and the approximate posterior uncertainty by an approximating posterior distribution. Furthermore, to tackle the challenging problem of hyperparameter tuning, the EP posterior approximation is embedded in a variational Expectation Maximization (EM) approach to allow for unsupervised hyperparameter estimation. Experiments are conducted on synthetic datasets, including an imaging deconvolution problem, to illustrate the efficiency of the proposed unsupervised EP method and the advantage of using MoG2 and SaS priors in solving sparse linear inverse problems.

Original language	English
Title of host publication	2022 Sensor Signal Processing for Defence Conference (SSPD)
Publisher	IEEE
ISBN (Electronic)	9781665483483
DOIs	https://doi.org/10.1109/SSPD54131.2022.9896226
Publication status	Published - 23 Sept 2022
Event	11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision 2022 - London, United Kingdom Duration: 13 Sept 2022 → 14 Sept 2022

Conference

Conference	11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision 2022
Abbreviated title	SSPD 2022
Country/Territory	United Kingdom
City	London
Period	13/09/22 → 14/09/22

Keywords

Expectation Propagation
hyperparameter estimation
large-scale problem
sparse linear model
sparsity prior
Unsupervised approximate Bayesian inference

ASJC Scopus subject areas

Instrumentation
Artificial Intelligence
Computer Networks and Communications
Signal Processing
Safety, Risk, Reliability and Quality
Acoustics and Ultrasonics

Access to Document

10.1109/SSPD54131.2022.9896226

Cite this

@inproceedings{53bbbffa77e3446ea64e1ffbfd28fe76,

title = "Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems",

abstract = "This paper addresses the estimation of large-scale sparse coefficients from noisy linear measurements using Expectation Propagation (EP) method for unsupervised approximate Bayesian inference. In the Bayesian model, the Laplace prior, Mixture of two Gaussians (MoG2) prior, and Spike-and-Slab (SaS) prior are adopted respectively as the sparsity-promoting priors of the unknown sparse parameter. In solving high-dimensional linear inverse problems, the proposed EP method directly provides the approximate minimum mean squared error (MMSE) estimate and the approximate posterior uncertainty by an approximating posterior distribution. Furthermore, to tackle the challenging problem of hyperparameter tuning, the EP posterior approximation is embedded in a variational Expectation Maximization (EM) approach to allow for unsupervised hyperparameter estimation. Experiments are conducted on synthetic datasets, including an imaging deconvolution problem, to illustrate the efficiency of the proposed unsupervised EP method and the advantage of using MoG2 and SaS priors in solving sparse linear inverse problems.",

keywords = "Expectation Propagation, hyperparameter estimation, large-scale problem, sparse linear model, sparsity prior, Unsupervised approximate Bayesian inference",

author = "Dan Yao and Stephen McLaughlin and Yoann Altmann",

note = "Funding Information: This work was supported by the Royal Academy of Engineering under the Research Fellowship scheme RF201617/16/31, the Engineering and Physical Sciences Research Council of the UK (EPSRC) Grant number EP/S000631/1 and the UK MOD University Defence Research Collaboration (UDRC) in Signal Processing. Publisher Copyright: {\textcopyright} 2022 IEEE.; 11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision 2022, SSPD 2022 ; Conference date: 13-09-2022 Through 14-09-2022",

year = "2022",

month = sep,

day = "23",

doi = "10.1109/SSPD54131.2022.9896226",

language = "English",

booktitle = "2022 Sensor Signal Processing for Defence Conference (SSPD)",

publisher = "IEEE",

address = "United States",

}

Yao, D , McLaughlin, S & Altmann, Y 2022, Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems. in 2022 Sensor Signal Processing for Defence Conference (SSPD)., 9896226, IEEE, 11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision 2022, London, United Kingdom, 13/09/22. https://doi.org/10.1109/SSPD54131.2022.9896226

TY - GEN

T1 - Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems

AU - Yao, Dan

AU - McLaughlin, Stephen

AU - Altmann, Yoann

N1 - Funding Information: This work was supported by the Royal Academy of Engineering under the Research Fellowship scheme RF201617/16/31, the Engineering and Physical Sciences Research Council of the UK (EPSRC) Grant number EP/S000631/1 and the UK MOD University Defence Research Collaboration (UDRC) in Signal Processing. Publisher Copyright: © 2022 IEEE.

PY - 2022/9/23

Y1 - 2022/9/23

N2 - This paper addresses the estimation of large-scale sparse coefficients from noisy linear measurements using Expectation Propagation (EP) method for unsupervised approximate Bayesian inference. In the Bayesian model, the Laplace prior, Mixture of two Gaussians (MoG2) prior, and Spike-and-Slab (SaS) prior are adopted respectively as the sparsity-promoting priors of the unknown sparse parameter. In solving high-dimensional linear inverse problems, the proposed EP method directly provides the approximate minimum mean squared error (MMSE) estimate and the approximate posterior uncertainty by an approximating posterior distribution. Furthermore, to tackle the challenging problem of hyperparameter tuning, the EP posterior approximation is embedded in a variational Expectation Maximization (EM) approach to allow for unsupervised hyperparameter estimation. Experiments are conducted on synthetic datasets, including an imaging deconvolution problem, to illustrate the efficiency of the proposed unsupervised EP method and the advantage of using MoG2 and SaS priors in solving sparse linear inverse problems.

AB - This paper addresses the estimation of large-scale sparse coefficients from noisy linear measurements using Expectation Propagation (EP) method for unsupervised approximate Bayesian inference. In the Bayesian model, the Laplace prior, Mixture of two Gaussians (MoG2) prior, and Spike-and-Slab (SaS) prior are adopted respectively as the sparsity-promoting priors of the unknown sparse parameter. In solving high-dimensional linear inverse problems, the proposed EP method directly provides the approximate minimum mean squared error (MMSE) estimate and the approximate posterior uncertainty by an approximating posterior distribution. Furthermore, to tackle the challenging problem of hyperparameter tuning, the EP posterior approximation is embedded in a variational Expectation Maximization (EM) approach to allow for unsupervised hyperparameter estimation. Experiments are conducted on synthetic datasets, including an imaging deconvolution problem, to illustrate the efficiency of the proposed unsupervised EP method and the advantage of using MoG2 and SaS priors in solving sparse linear inverse problems.

KW - Expectation Propagation

KW - hyperparameter estimation

KW - large-scale problem

KW - sparse linear model

KW - sparsity prior

KW - Unsupervised approximate Bayesian inference

UR - http://www.scopus.com/inward/record.url?scp=85139596837&partnerID=8YFLogxK

U2 - 10.1109/SSPD54131.2022.9896226

DO - 10.1109/SSPD54131.2022.9896226

M3 - Conference contribution

AN - SCOPUS:85139596837

BT - 2022 Sensor Signal Processing for Defence Conference (SSPD)

PB - IEEE

T2 - 11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision 2022

Y2 - 13 September 2022 through 14 September 2022

ER -

Unsupervised Expectation Propagation Method for Large-Scale Sparse Linear Inverse Problems

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this