Automatic micro-expression recognition from long video using a single spotted apex

Sze-Teng Liong, John See, KokSheik Wong, Raphael Chung-Wei Phan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)

Abstract

Recently, micro-expression recognition has seen an increase of interest from psychological and computer vision communities. As micro-expressions are generated involuntarily on a person’s face, and are usually a manifestation of repressed feelings of the person. Most existing works pay attention to either the detection or spotting of micro-expression frames or the categorization of type of micro-expression present in a short video shot. In this paper, we introduced a novel automatic approach to micro-expression recognition from long video that combines both spotting and recognition mechanisms. To achieve this, the apex frame, which provides the instant when the highest intensity of facial movement occurs, is first spotted from the entire video sequence. An automatic eye masking technique is also presented to improve the robustness of apex frame spotting. With the single apex, we describe the spotted micro-expression instant using a state-of-the-art feature extractor before proceeding to classification. This is the first known work that recognizes micro-expressions from a long video sequence without the knowledge of onset and offset frames, which are typically used to determine a cropped sub-sequence containing the micro-expression. We evaluated the spotting and recognition tasks on four spontaneous micro-expression databases comprising only of raw long videos – CASME II-RAW, SMIC-E-HS, SMIC-E-VIS and SMIC-E-NIR. We obtained compelling results that show the effectiveness of the proposed approach, which outperform most methods that rely on human annotated sub-sequences.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2016 Workshops. ACCV 2016
EditorsChu-Song Chen, Jiwen Lu, Kai-Kuang Ma
PublisherSpringer
Pages345-360
Number of pages16
ISBN (Electronic)9783319544274
ISBN (Print)9783319544267
DOIs
Publication statusPublished - 16 Mar 2017
Event13th Asian Conference on Computer Vision 2016 - Taipei, Taiwan, Province of China
Duration: 20 Nov 201624 Nov 2016

Publication series

NameLecture Notes in Computer Science
Volume10117
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference13th Asian Conference on Computer Vision 2016
Abbreviated titleACCV 2016
Country/TerritoryTaiwan, Province of China
City Taipei
Period20/11/1624/11/16

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Automatic micro-expression recognition from long video using a single spotted apex'. Together they form a unique fingerprint.

Cite this