Context-Aware Multi-Stream Networks for Dimensional Emotion Prediction in Images

Sidharrth Nagappan, Jia Qi Tan, Lai-Kuan Wong, John See

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Teaching machines to comprehend the nuances of emotion from photographs is a particularly challenging task. Emotion perception— naturally a subjective problem, is often simplified for computational purposes into categorical states or valence-arousal dimensional space, the latter being a lesser-explored problem in the literature. This paper proposes a multi-stream context-aware neural network model for dimensional emotion prediction in images. Models were trained using a set of object and scene data along with deep features for valence, arousal, and dominance estimation. Experimental evaluation on a large-scale image emotion dataset demonstrates the viability of our proposed approach. Our analysis postulates that the understanding of the depicted object in an image is vital for successful predictions whilst relying on scene information can lead to somewhat confounding effects.
Original languageEnglish
Title of host publication2023 IEEE International Conference on Image Processing (ICIP)
PublisherIEEE
Pages2480-2484
Number of pages5
ISBN (Electronic)9781728198354
DOIs
Publication statusPublished - 11 Sept 2023

Keywords

  • DES
  • deep neural networks
  • dimensional emotion
  • emotional analysis
  • image emotion prediction

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Context-Aware Multi-Stream Networks for Dimensional Emotion Prediction in Images'. Together they form a unique fingerprint.

Cite this