Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application

Anh Cat Le Ngo, John See, Raphael C. W. Phan

Research output: Contribution to journalArticlepeer-review

53 Citations (Scopus)


Subtle emotions are present in diverse real-life situations: in hostile environments, enemies and/or spies maliciouslyconceal their emotions as part of their deception; in life-threatening situations, victims under duress have no choice but to withhold theirreal feelings; in the medical scene, patients with psychological conditions such as depression could either be intentionally orsubconsciously suppressing their anguish from loved ones. Under such circumstances, it is often crucial that these subtle emotions arerecognized before it is too late. These spontaneous subtle emotions are typically expressed through micro-expressions, which are tiny,sudden and short-lived dynamics of facial muscles; thus, such micro-expressions pose a great challenge for visual recognition. Theabrupt but significant dynamics for the recognition task are temporally sparse while the rest, i.e. irrelevant dynamics, are temporallyredundant. In this work, we analyze and enforce sparsity constraints to learn significant temporal and spectral structures whileeliminating irrelevant facial dynamics of micro-expressions, which would ease the challenge in the visual recognition of spontaneoussubtle emotions. The hypothesis is confirmed through experimental results of automatic spontaneous subtle emotion recognition withseveral sparsity levels on CASME II and SMIC, the two well-established and publicly available spontaneous subtle emotion databases.The overall performances of the automatic subtle emotion recognition are boosted when only significant dynamics of the originalsequences are preserved.

Original languageEnglish
Pages (from-to)396-411
Number of pages16
JournalIEEE Transactions on Affective Computing
Issue number3
Early online date3 Feb 2016
Publication statusPublished - Jul 2017


  • data sparsity
  • dynamic mode decomposition
  • emotion suppression
  • Micro-expression recognition
  • Spontaneous subtle emotions

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction


Dive into the research topics of 'Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application'. Together they form a unique fingerprint.

Cite this