Effective recognition of facial micro-expressions with video motion magnification

Yandan Wang, John See, Yee-Hui Oh, Raphael C. W. Phan, Yogachandran Rahulamathavan, Huo-Chong Ling, Su-Wei Tan, Xujie Li

Research output: Contribution to journalArticlepeer-review

41 Citations (Scopus)

Abstract

Facial expression recognition has been intensively studied for decades, notably by the psychology community and more recently the pattern recognition community. What is more challenging, and the subject of more recent research, is the problem of recognizing subtle emotions exhibited by so-called micro-expressions. Recognizing a micro-expression is substantially more challenging than conventional expression recognition because these micro-expressions are only temporally exhibited in a fraction of a second and involve minute spatial changes. Until now, work in this field is at a nascent stage, with only a few existing micro-expression databases and methods. In this article, we propose a new micro-expression recognition approach based on the Eulerian motion magnification technique, which could reveal the hidden information and accentuate the subtle changes in micro-expression motion. Validation of our proposal was done on the recently proposed CASME II dataset in comparison with baseline and state-of-the-art methods. We achieve a good recognition accuracy of up to 75.30 % by using leave-one-out cross validation evaluation protocol. Extensive experiments on various factors at play further demonstrate the effectiveness of our proposed approach.

Original languageEnglish
Pages (from-to)21665-21690
Number of pages26
JournalMultimedia Tools and Applications
Volume76
Issue number20
Early online date8 Nov 2016
DOIs
Publication statusPublished - Oct 2017

Keywords

  • CASME II
  • EVM
  • Local binary patterns
  • Micro-expressions
  • Motion magnification

ASJC Scopus subject areas

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Effective recognition of facial micro-expressions with video motion magnification'. Together they form a unique fingerprint.

Cite this