Sigh!!! There is more than just faces and verbal speech to recognize emotion in human-robot interaction

Understanding human emotions is paramount for effective human-human interactions. As technology advances, social robots are increasingly being developed with the capability to discern and respond to human emotions, with the ultimate aim of providing assistance and companionship. However, existing research on emotion recognition for human-robot interaction predominantly focuses on facial expressions or verbal speech, neglecting other potential mediums of emotional expression. In this study, we shed light on the significance of considering various forms of emotional expression, mainly nonverbal vocalization known as vocal bursts, which have been overlooked in emotion modeling for human-robot interaction. Vocal bursts, characterized by brief and intense vocal utterances, represent a rich source of emotional cues that can significantly enhance the capabilities of social robots in understanding and responding to human emotions. Driven by the increasing interest in vocal bursts within speech and affective computing research, we propose a baseline model for affective vocal burst recognition that can outperform large audio models. The proposed baseline model achieves weighted F1 scores of 0.606, 0.342, and 0.287 on 10, 24, and 30 emotion classes, respectively. Additionally, we identify challenges that must be addressed to enhance affective vocal burst recognition for human-robot interaction. Code available at /github.com/rahullabs/Sigh