Hybrid Facial Regions Extraction for Micro-expression Recognition System

Sze-Teng Liong*, John See, Raphael C. W. Phan, Kok Sheik Wong, Su-Wei Tan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

30 Citations (Scopus)


Micro-expressions can occur when a person attempts to conceal and suppress his true feelings and emotions, both deliberately or unconsciously.In recent years, facial micro-expression analysis has received tremendous attention in the field of psychology, media and computer vision. However, due to its subtlety and brief duration, development of automated micro-expression detection and recognition system are still great challenges in the field of computer vision. In this paper, we present a novel hybrid facial region extraction framework that combines heuristic and automatic approaches to better recognize spontaneous micro-expressions. Salient facial regions are statistically determined based on the occurrence frequency of facial action units instead of holistic utilization of the entire facial area. The regions were automatically selected according to the facial landmark coordinates. We tested on two recent publicly available datasets that provided sufficient samples while also fulfilling the criteria of being elicited spontaneously. To further confirm the reliability of the proposed method, two distinct feature extractors were employed to describe micro-expression information. Results show consistent and promising performance in all scenarios considered. The best result achieved is an improvement of approximately 10.5% in CASME II and an increment of nearly 10% in SMIC. We also report F-measure, precision and recall performance metrics that are most suited for the imbalanced nature of spontaneous micro-expression datasets.

Original languageEnglish
Pages (from-to)601-617
Number of pages17
JournalJournal of Signal Processing Systems
Issue number4
Early online date19 Aug 2017
Publication statusPublished - Apr 2018


  • Emotion
  • Micro-expressions
  • Optical strain
  • Recognition
  • Region of interest

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Theoretical Computer Science
  • Signal Processing
  • Information Systems
  • Modelling and Simulation
  • Hardware and Architecture


Dive into the research topics of 'Hybrid Facial Regions Extraction for Micro-expression Recognition System'. Together they form a unique fingerprint.

Cite this