TY - GEN
T1 - ViCA: Combining visual, social, and task-oriented conversational AI in a healthcare setting
AU - Pantazopoulos, Georgios
AU - Bruyere, Jeremy
AU - Nikandrou, Maria-Vasiliki
AU - Boissier, Thibaud
AU - Hemanthage, Supun
AU - Binha, Sachish
AU - Shah, Vidyul
AU - Dondrup, Christian
AU - Lemon, Oliver
N1 - Georgios Pantazopoulos, Jeremy Bruyere, Malvina Nikandrou, Thibaud Boissier, Supun Hemanthage, Sachish Binha, Vidyul Shah, Christian Dondrup, and Oliver Lemon. 2021. ViCA: Combining visual, social, and task- oriented conversational AI in a healthcare setting. In Proceedings of the 2021 International Conference on Multimodal Interaction (ICMI ’21), October 18–22, 2021, Montréal, QC, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3462244.3479909
PY - 2021/7/26
Y1 - 2021/7/26
N2 - Recent developments in computer vision and conversational systems have provided the AI community with novel perspectives towards improving the cognitive capabilities of engaging socially assistive robots. We show how to develop conversational skills for a hospital receptionist robot that incorporates social conversation based on visual information as well as task-based dialog. Fusing the traditional modular conversational system architecture with recent developments in computer vision and scene graph research, our agent (called ‘ViCA’) supports both visual question answering and social conversational capabilities based on the visual scene. In particular, our agent can provide guidance to users by locating visible objects in the room and can engage in social dialogue using visual prompts, such as the user’s clothing or possessions. We con- duct a comprehensive online evaluation study with 21 participants, showcasing that the ViCA system is perceived as both helpful and entertaining.
AB - Recent developments in computer vision and conversational systems have provided the AI community with novel perspectives towards improving the cognitive capabilities of engaging socially assistive robots. We show how to develop conversational skills for a hospital receptionist robot that incorporates social conversation based on visual information as well as task-based dialog. Fusing the traditional modular conversational system architecture with recent developments in computer vision and scene graph research, our agent (called ‘ViCA’) supports both visual question answering and social conversational capabilities based on the visual scene. In particular, our agent can provide guidance to users by locating visible objects in the room and can engage in social dialogue using visual prompts, such as the user’s clothing or possessions. We con- duct a comprehensive online evaluation study with 21 participants, showcasing that the ViCA system is perceived as both helpful and entertaining.
M3 - Conference contribution
BT - 23rd ACM International Conference on Multimodal Interaction
T2 - 23rd ACM International Conference on Multimodal Interaction 2021
Y2 - 18 October 2021 through 22 October 2021
ER -