Enriched long-term recurrent convolutional network for facial micro-expression recognition

Huai Qian Khor, John See, Raphael Chung Wei Phan, Weiyao Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Facial micro-expression (ME) recognition has posed a huge challenge to researchers for its subtlety in motion and limited databases. Recently, handcrafted techniques have achieved superior performance in micro-expression recognition but at the cost of domain specificity and cumbersome parametric tunings. In this paper, we propose an Enriched Long-term Recurrent Convolutional Network (ELRCN) that first encodes each micro-expression frame into a feature vector through CNN module(s), then predicts the micro-expression by passing the feature vector through a Long Short-term Memory (LSTM) module. The framework contains 2 different network variants: (1) Channel-wise stacking of input data for spatial enrichment, (2) Feature-wise stacking of features for temporal enrichment. We demonstrate that the proposed approach is able to achieve reasonably good performance, without data augmentation. In addition, we also present ablation studies conducted on the framework and visualizations of what CNN 'sees' when predicting the micro-expression classes.

Original languageEnglish
Title of host publication2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)
PublisherIEEE
Pages667-674
Number of pages8
ISBN (Electronic)9781538623350
DOIs
Publication statusPublished - 7 Jun 2018
Event13th IEEE International Conference on Automatic Face and Gesture Recognition 2018 - Xi'an, China
Duration: 15 May 201819 May 2018

Conference

Conference13th IEEE International Conference on Automatic Face and Gesture Recognition 2018
Abbreviated titleFG 2018
CountryChina
CityXi'an
Period15/05/1819/05/18

Keywords

  • Cross-database evaluation
  • LRCN
  • Micro-Expression Recognition
  • Network enrichment
  • Objective class

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Control and Optimization

Fingerprint Dive into the research topics of 'Enriched long-term recurrent convolutional network for facial micro-expression recognition'. Together they form a unique fingerprint.

Cite this