Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization

Rui Qian, Yuxi Li, Huabin Liu, John See, Shuangrui Ding, Xian Liu, Dian Li, Weiyao Lin*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Citations (Scopus)

Abstract

The crux of self-supervised video representation learning is to build general features from unlabeled videos. However, most recent works have mainly focused on high-level semantics and neglected lower-level representations and their temporal relationship which are crucial for general video understanding. To address these challenges, this paper proposes a multi-level feature optimization framework to improve the generalization and temporal modeling ability of learned video representations. Concretely, high-level features obtained from naive and prototypical contrastive learning are utilized to build distribution graphs, guiding the process of low-level and mid-level feature learning. We also devise a simple temporal modeling module from multi-level features to enhance motion pattern learning. Experiments demonstrate that multi-level feature optimization with the graph constraint and temporal modeling can greatly improve the representation ability in video understanding. Code is available here.

Original languageEnglish
Title of host publication2021 IEEE/CVF International Conference on Computer Vision (ICCV)
PublisherIEEE
Pages7970-7981
Number of pages12
ISBN (Electronic)9781665428125
DOIs
Publication statusPublished - 28 Feb 2022
Event18th IEEE/CVF International Conference on Computer Vision 2021 - Virtual, Online, Canada
Duration: 11 Oct 202117 Oct 2021

Conference

Conference18th IEEE/CVF International Conference on Computer Vision 2021
Abbreviated titleICCV 2021
Country/TerritoryCanada
CityVirtual, Online
Period11/10/2117/10/21

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization'. Together they form a unique fingerprint.

Cite this