Where Is the Emotion? Dissecting A Multi-Gap Network for Image Emotion Classification

Lucinda Lim, Huai Qian Khor, Phatcharawat Chaemchoy, John See, Lai Kuan Wong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)


Image emotion recognition has become an increasingly popular research domain in the area of image processing and affective computing. Despite fast-improving classification performance in this task, the understanding and interpretability of its performance are still lacking as there are limited studies on which part of an image would invoke a particular emotion. In this work, we propose a Multi-GAP deep neural network for image emotion classification, which is extensible to accommodate multiple streams of information. We also incorporate feature dependency into our network blocks by adding a bidirectional GRU network to learn transitional features. We report extensive results on the variants of our proposed network and provide valuable perspectives into the class-activated regions via Grad-CAM, and network depth contributions by truncation strategy.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings
Number of pages5
ISBN (Electronic)9781728163956
Publication statusPublished - Oct 2020
Event2020 IEEE International Conference on Image Processing - Virtual, Abu Dhabi, United Arab Emirates
Duration: 25 Oct 202028 Oct 2020


Conference2020 IEEE International Conference on Image Processing
Abbreviated titleICIP 2020
Country/TerritoryUnited Arab Emirates
CityVirtual, Abu Dhabi


  • class activation maps
  • CNN
  • Image emotion classification
  • multi-GAP
  • visualizations

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing


Dive into the research topics of 'Where Is the Emotion? Dissecting A Multi-Gap Network for Image Emotion Classification'. Together they form a unique fingerprint.

Cite this