Towards visual dialogue for human-robot interaction

Jose L. Part; Daniel Hernández Garciá; Yanchao Yu; Nancie Gunson; Christian Dondrup; Oliver Lemon

doi:10.1145/3434074.3447278

Towards visual dialogue for human-robot interaction

Jose L. Part, Daniel Hernández Garciá, Yanchao Yu, Nancie Gunson, Christian Dondrup, Oliver Lemon

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations about the surrounding world is critical. In this paper, we describe current progress towards developing the necessary integrated visual and conversational capabilities for a robot to operate in such environments. Concretely, we introduce an architecture for conversing about objects and other entities present in an environment. The work described in this paper has applications that extend well beyond healthcare and can be used on any robot that requires to interact with its visual and spatial environment in order to be able to perform its duties.

Original language	English
Title of host publication	HRI '21 Companion
Subtitle of host publication	Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction
Publisher	Association for Computing Machinery
Pages	670-672
Number of pages	3
ISBN (Electronic)	9781450382908
DOIs	https://doi.org/10.1145/3434074.3447278
Publication status	Published - 8 Mar 2021
Event	2021 ACM/IEEE International Conference on Human-Robot Interaction - Virtual Conference, Boulder, United States Duration: 8 Mar 2021 → 11 Mar 2021 https://humanrobotinteraction.org/2021/

Conference

Conference	2021 ACM/IEEE International Conference on Human-Robot Interaction
Abbreviated title	HRI '21
Country/Territory	United States
City	Boulder
Period	8/03/21 → 11/03/21
Internet address	https://humanrobotinteraction.org/2021/

Keywords

Conversational systems
Robotics
Visual dialogue

ASJC Scopus subject areas

Artificial Intelligence
Human-Computer Interaction
Electrical and Electronic Engineering

Access to Document

10.1145/3434074.3447278

Cite this

@inproceedings{0c62706ce14a4a3ababb656fb75b6213,

title = "Towards visual dialogue for human-robot interaction",

abstract = "The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations about the surrounding world is critical. In this paper, we describe current progress towards developing the necessary integrated visual and conversational capabilities for a robot to operate in such environments. Concretely, we introduce an architecture for conversing about objects and other entities present in an environment. The work described in this paper has applications that extend well beyond healthcare and can be used on any robot that requires to interact with its visual and spatial environment in order to be able to perform its duties.",

keywords = "Conversational systems, Robotics, Visual dialogue",

author = "Part, {Jose L.} and {Hern{\'a}ndez Garci{\'a}}, Daniel and Yanchao Yu and Nancie Gunson and Christian Dondrup and Oliver Lemon",

note = "Funding Information: This work has been supported by the European Union{\textquoteright}s H2020-ICT Research and Innovation Action SPRING – Grant Agreement No. 871245 (https://spring-h2020.eu/). Publisher Copyright: {\textcopyright} 2021 Owner/Author. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.; 2021 ACM/IEEE International Conference on Human-Robot Interaction, HRI '21 ; Conference date: 08-03-2021 Through 11-03-2021",

year = "2021",

month = mar,

day = "8",

doi = "10.1145/3434074.3447278",

language = "English",

pages = "670--672",

booktitle = "HRI '21 Companion",

publisher = "Association for Computing Machinery",

address = "United States",

url = "https://humanrobotinteraction.org/2021/",

}

Part, JL, Hernández Garciá, D, Yu, Y, Gunson, N , Dondrup, C & Lemon, O 2021, Towards visual dialogue for human-robot interaction. in HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. Association for Computing Machinery, pp. 670-672, 2021 ACM/IEEE International Conference on Human-Robot Interaction, Boulder, Colorado, United States, 8/03/21. https://doi.org/10.1145/3434074.3447278

TY - GEN

T1 - Towards visual dialogue for human-robot interaction

AU - Part, Jose L.

AU - Hernández Garciá, Daniel

AU - Yu, Yanchao

AU - Gunson, Nancie

AU - Dondrup, Christian

AU - Lemon, Oliver

N1 - Funding Information: This work has been supported by the European Union’s H2020-ICT Research and Innovation Action SPRING – Grant Agreement No. 871245 (https://spring-h2020.eu/). Publisher Copyright: © 2021 Owner/Author. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

PY - 2021/3/8

Y1 - 2021/3/8

N2 - The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations about the surrounding world is critical. In this paper, we describe current progress towards developing the necessary integrated visual and conversational capabilities for a robot to operate in such environments. Concretely, we introduce an architecture for conversing about objects and other entities present in an environment. The work described in this paper has applications that extend well beyond healthcare and can be used on any robot that requires to interact with its visual and spatial environment in order to be able to perform its duties.

AB - The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations about the surrounding world is critical. In this paper, we describe current progress towards developing the necessary integrated visual and conversational capabilities for a robot to operate in such environments. Concretely, we introduce an architecture for conversing about objects and other entities present in an environment. The work described in this paper has applications that extend well beyond healthcare and can be used on any robot that requires to interact with its visual and spatial environment in order to be able to perform its duties.

KW - Conversational systems

KW - Robotics

KW - Visual dialogue

UR - http://www.scopus.com/inward/record.url?scp=85102752735&partnerID=8YFLogxK

U2 - 10.1145/3434074.3447278

DO - 10.1145/3434074.3447278

M3 - Conference contribution

AN - SCOPUS:85102752735

SP - 670

EP - 672

BT - HRI '21 Companion

PB - Association for Computing Machinery

T2 - 2021 ACM/IEEE International Conference on Human-Robot Interaction

Y2 - 8 March 2021 through 11 March 2021

ER -

Towards visual dialogue for human-robot interaction

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this