Optical strain based recognition of subtle emotions

This paper presents a novel method to recognize subtle emotions based on optical strain magnitude feature extraction from the temporal point of view. The common way that subtle emotions are exhibited by a person is in the form of visually observed micro-expressions, which usually occur only over a brief period of time. Optical strain allows small deformations on the face to be computed between successive frames although these subtle changes can be minute. We perform temporal sum pooling for each frame in the video to a single strain map to summarize the features over time. To reduce the dimensionality of the input space, the strain maps are then resized to a pre-defined resolution for consistency across the database. Experiments were conducted on the SMIC (Spontaneous Micro-expression) Database, which was recently established in 2013. A best three-class recognition accuracy of 53.56% is achieved, with the proposed method outperforming the baseline reported in the original work by almost 5%. This is the first known optical strain based classification of micro-expressions. The closest related work employed optical strain to spot micro-expressions, but did not investigate its potential for determining the specific type of micro-expression.