TY - JOUR
T1 - Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC
AU - Lin, Weiyao
AU - He, Xiaoyi
AU - Han, Xintong
AU - Liu, Dong
AU - See, John
AU - Zou, Junni
AU - Xiong, Hongkai
AU - Wu, Feng
N1 - Funding Information:
Manuscript received April 27, 2019; revised November 3, 2019; accepted December 16, 2019. Date of publication December 25, 2019; date of current version October 23, 2020. The paper is supported in part by the China Major Project for New Generation of AI Grant 2018AAA0100400, National Natural Science Foundation of China under Grants 61971277, 61772483, CREST Malaysia Grant T03C1-17. The basic idea of this paper appeared in our conference version. In this version, we extend our approach by introducing an adaptive-switching scheme, carry out detailed analysis, and present more performance results. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Sanjeev Mehrotra. (Corresponding author: Weiyao Lin.) W. Lin, X. He, and H. Xiong are with the Department of Electronic Engineering, Shanghai Jiao Tong University, China (e-mail: [email protected]; [email protected]; [email protected]). X. Han is with the Huya Inc., China (e-mail: [email protected]).
Publisher Copyright:
© 1999-2012 IEEE.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020/11
Y1 - 2020/11
N2 - This article addresses neural network based post-processing for the state-of-The-Art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-Aware convolution neural network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-Aware and adaptive-switching neural networks.
AB - This article addresses neural network based post-processing for the state-of-The-Art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-Aware convolution neural network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-Aware and adaptive-switching neural networks.
KW - convolutional neural network
KW - High Efficiency Video Coding
KW - post-processing
UR - http://www.scopus.com/inward/record.url?scp=85077265165&partnerID=8YFLogxK
U2 - 10.1109/TMM.2019.2962310
DO - 10.1109/TMM.2019.2962310
M3 - Article
AN - SCOPUS:85077265165
SN - 1520-9210
VL - 22
SP - 2749
EP - 2763
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
IS - 11
ER -