Hybrid Facial Regions Extraction for Micro-expression Recognition System

Micro-expressions can occur when a person attempts to conceal and suppress his true feelings and emotions, both deliberately or unconsciously.In recent years, facial micro-expression analysis has received tremendous attention in the field of psychology, media and computer vision. However, due to its subtlety and brief duration, development of automated micro-expression detection and recognition system are still great challenges in the field of computer vision. In this paper, we present a novel hybrid facial region extraction framework that combines heuristic and automatic approaches to better recognize spontaneous micro-expressions. Salient facial regions are statistically determined based on the occurrence frequency of facial action units instead of holistic utilization of the entire facial area. The regions were automatically selected according to the facial landmark coordinates. We tested on two recent publicly available datasets that provided sufficient samples while also fulfilling the criteria of being elicited spontaneously. To further confirm the reliability of the proposed method, two distinct feature extractors were employed to describe micro-expression information. Results show consistent and promising performance in all scenarios considered. The best result achieved is an improvement of approximately 10.5% in CASME II and an increment of nearly 10% in SMIC. We also report F-measure, precision and recall performance metrics that are most suited for the imbalanced nature of spontaneous micro-expression datasets.