Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach. part ii: Theoretical analysis

Valentin De Bortoli, Alain Durmus, Marcelo Pereyra, Ana Fernandez Vidal

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [A. F. Vidal et al., SIAM J. Imaging Sci., 13 (2020), pp. 1945–1989] to set regularization parameters by marginal maximum likelihood estimation. We prove the convergence of a more general stochastic approximation scheme that includes the three algorithms of [A. F. Vidal et al., SIAM J. Imaging Sci., 13 (2020), pp. 1945–1989] as special cases. This includes asymptotic and nonasymptotic convergence results with natural and easily verifiable conditions, as well as explicit bounds on the convergence rates. Importantly, the theory is also general in that it can be applied to other intractable optimization problems. A main novelty of the work is that the stochastic gradient estimates of our scheme are constructed from inexact proximal Markov chain Monte Carlo samplers. This allows the use of samplers that scale efficiently to large problems and for which we have precise theoretical guarantees.

Original languageEnglish
Pages (from-to)1990-2028
Number of pages39
JournalSIAM Journal on Imaging Sciences
Volume13
Issue number4
DOIs
Publication statusPublished - 18 Nov 2020

Keywords

  • Empirical Bayes
  • Image processing
  • Inverse problems
  • Markov chain Monte Carlo methods
  • Proximal algorithms
  • Statistical inference
  • Stochastic optimization

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach. part ii: Theoretical analysis'. Together they form a unique fingerprint.

Cite this