Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

Weiyao Lin, Xiaoyi He, Xintong Han, Dong Liu, John See, Junni Zou, Hongkai Xiong, Feng Wu

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

This article addresses neural network based post-processing for the state-of-The-Art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-Aware convolution neural network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-Aware and adaptive-switching neural networks.

Original languageEnglish
Pages (from-to)2749-2763
Number of pages15
JournalIEEE Transactions on Multimedia
Volume22
Issue number11
Early online date25 Dec 2019
DOIs
Publication statusPublished - Nov 2020

Keywords

  • convolutional neural network
  • High Efficiency Video Coding
  • post-processing

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC'. Together they form a unique fingerprint.

Cite this