Multimodal Representation Learning for Human Robot Interaction

Eli Sheppard; Katrin Solveig Lohan

doi:10.1145/3371382.3378265

Multimodal Representation Learning for Human Robot Interaction

Eli Sheppard, Katrin Solveig Lohan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

98 Downloads (Pure)

Abstract

We present a neural network based system capable of learning a multimodal representation of images and words. This representation allows for bidirectional grounding of the meaning of words and the visual attributes that they represent, such as colour, size and object name. We also present a new dataset captured specifically for this task.

Original language	English
Title of host publication	HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
Publisher	Association for Computing Machinery
Pages	445–446
Number of pages	2
ISBN (Electronic)	9781450370578
DOIs	https://doi.org/10.1145/3371382.3378265
Publication status	Published - 23 Mar 2020
Event	15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020 - Corn Exchange, Cambridge, United Kingdom Duration: 23 Mar 2020 → 26 Mar 2020 https://humanrobotinteraction.org/2020/

Conference

Conference	15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020
Abbreviated title	HRI 2020
Country/Territory	United Kingdom
City	Cambridge
Period	23/03/20 → 26/03/20
Internet address	https://humanrobotinteraction.org/2020/

Keywords

Datasets
Neural networks
Robotics
Symbol grounding
Unsupervised learning

ASJC Scopus subject areas

Artificial Intelligence
Human-Computer Interaction
Electrical and Electronic Engineering

Access to Document

10.1145/3371382.3378265

Download pdf
© 2020 Copyright held by the owner/author(s).
Accepted author manuscript, 3.42 MB

Cite this

@inproceedings{435471af0ca84386a571a43df52f3af9,

title = "Multimodal Representation Learning for Human Robot Interaction",

abstract = "We present a neural network based system capable of learning a multimodal representation of images and words. This representation allows for bidirectional grounding of the meaning of words and the visual attributes that they represent, such as colour, size and object name. We also present a new dataset captured specifically for this task.",

keywords = "Datasets, Neural networks, Robotics, Symbol grounding, Unsupervised learning",

author = "Eli Sheppard and Lohan, {Katrin Solveig}",

year = "2020",

month = mar,

day = "23",

doi = "10.1145/3371382.3378265",

language = "English",

pages = "445–446",

booktitle = "HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction",

publisher = "Association for Computing Machinery",

address = "United States",

note = "15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020, HRI 2020 ; Conference date: 23-03-2020 Through 26-03-2020",

url = "https://humanrobotinteraction.org/2020/",

}

Sheppard, E & Lohan, KS 2020, Multimodal Representation Learning for Human Robot Interaction. in HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. Association for Computing Machinery, pp. 445–446, 15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020, Cambridge, United Kingdom, 23/03/20. https://doi.org/10.1145/3371382.3378265

TY - GEN

T1 - Multimodal Representation Learning for Human Robot Interaction

AU - Sheppard, Eli

AU - Lohan, Katrin Solveig

PY - 2020/3/23

Y1 - 2020/3/23

N2 - We present a neural network based system capable of learning a multimodal representation of images and words. This representation allows for bidirectional grounding of the meaning of words and the visual attributes that they represent, such as colour, size and object name. We also present a new dataset captured specifically for this task.

AB - We present a neural network based system capable of learning a multimodal representation of images and words. This representation allows for bidirectional grounding of the meaning of words and the visual attributes that they represent, such as colour, size and object name. We also present a new dataset captured specifically for this task.

KW - Datasets

KW - Neural networks

KW - Robotics

KW - Symbol grounding

KW - Unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=85083256051&partnerID=8YFLogxK

U2 - 10.1145/3371382.3378265

DO - 10.1145/3371382.3378265

M3 - Conference contribution

SP - 445

EP - 446

BT - HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction

PB - Association for Computing Machinery

T2 - 15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020

Y2 - 23 March 2020 through 26 March 2020

ER -

Multimodal Representation Learning for Human Robot Interaction

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this