TY - JOUR
T1 - Robust Single-Image Super-Resolution via CNNs and TV-TV Minimization
AU - Vella, Marija
AU - Mota, João F. C.
N1 - Funding Information:
Manuscript received December 4, 2020; revised March 30, 2021; accepted August 16, 2021. Date of publication September 10, 2021; date of current version September 16, 2021. This work was supported in part by U.K.’s Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/T026111/1 and Grant EP/S000631/1, in part by the Ministry of Defense (MOD) University Defence Research Collaboration, and in part by EPSRC under Grant EP/S018018/1. Part of this work has been presented in [1] [DOI: 10.5244/C.33.219], [2]. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Hairong Qi. (Corresponding author: João F. C. Mota.) The authors are with the School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh EH14 4AS, U.K. (e-mail: [email protected]; [email protected]). Digital Object Identifier 10.1109/TIP.2021.3108907
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2021
Y1 - 2021
N2 - Single-image super-resolution is the process of increasing the resolution of an image, obtaining a high-resolution (HR) image from a low-resolution (LR) one. By leveraging large training datasets, convolutional neural networks (CNNs) currently achieve the state-of-the-art performance in this task. Yet, during testing/deployment, they fail to enforce consistency between the HR and LR images: if we downsample the output HR image, it never matches its LR input. Based on this observation, we propose to post-process the CNN outputs with an optimization problem that we call TV-TV minimization, which enforces consistency. As our extensive experiments show, such post-processing not only improves the quality of the images, in terms of PSNR and SSIM, but also makes the super-resolution task robust to operator mismatch, i.e., when the true downsampling operator is different from the one used to create the training dataset.
AB - Single-image super-resolution is the process of increasing the resolution of an image, obtaining a high-resolution (HR) image from a low-resolution (LR) one. By leveraging large training datasets, convolutional neural networks (CNNs) currently achieve the state-of-the-art performance in this task. Yet, during testing/deployment, they fail to enforce consistency between the HR and LR images: if we downsample the output HR image, it never matches its LR input. Based on this observation, we propose to post-process the CNN outputs with an optimization problem that we call TV-TV minimization, which enforces consistency. As our extensive experiments show, such post-processing not only improves the quality of the images, in terms of PSNR and SSIM, but also makes the super-resolution task robust to operator mismatch, i.e., when the true downsampling operator is different from the one used to create the training dataset.
KW - Image super-resolution
KW - convolutional neural networks. (CNNs)
KW - image reconstruction
KW - prior information
KW - ℓ ℓ minimization
UR - http://www.scopus.com/inward/record.url?scp=85114730748&partnerID=8YFLogxK
U2 - 10.1109/TIP.2021.3108907
DO - 10.1109/TIP.2021.3108907
M3 - Article
C2 - 34506282
SN - 1057-7149
VL - 30
SP - 7830
EP - 7841
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
ER -