Less is more: Micro-expression recognition from video using apex frame

Sze-Teng Liong, John See, Kok Sheik Wong, Raphael C. W. Phan

Research output: Contribution to journalArticlepeer-review

70 Citations (Scopus)

Abstract

Despite recent interest and advances in facial micro-expression research, there is still plenty of room for improvement in terms of micro-expression recognition. Conventional feature extraction approaches for micro-expression video consider either the whole video sequence or a part of it, for representation. However, with the high-speed video capture of micro-expressions (100–200 fps), are all frames necessary to provide a sufficiently meaningful representation? Is the luxury of data a bane to accurate recognition? A novel proposition is presented in this paper, whereby we utilize only two images per video, namely, the apex frame and the onset frame. The apex frame of a video contains the highest intensity of expression changes among all frames, while the onset is the perfect choice of a reference frame with neutral expression. A new feature extractor, Bi-Weighted Oriented Optical Flow (Bi-WOOF) is proposed to encode essential expressiveness of the apex frame. We evaluated the proposed method on five micro-expression databases—CAS(ME)2, CASME II, SMIC-HS, SMIC-NIR and SMIC-VIS. Our experiments lend credence to our hypothesis, with our proposed technique achieving a state-of-the-art F1-score recognition performance of 0.61 and 0.62 in the high frame rate CASME II and SMIC-HS databases respectively.

Original languageEnglish
Pages (from-to)82-92
Number of pages11
JournalSignal Processing: Image Communication
Volume62
Early online date14 Dec 2017
DOIs
Publication statusPublished - Mar 2018

Keywords

  • Apex
  • Emotion
  • Micro-expressions
  • Optical flow
  • Optical strain
  • Recognition

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Less is more: Micro-expression recognition from video using apex frame'. Together they form a unique fingerprint.

Cite this