An empirical study on the generalization power of neural representations learned via visual guessing games

Alessandro Suglia, Yonatan Bisk, Ioannis Konstas, Antonio Vergari, Emanuele Bastianelli, Andrea Vanzo, Oliver Lemon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Guessing games are a prototypical instance of the “learning by interacting” paradigm. This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA). We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL). We evaluate the ability of both procedures to generalise: an in-domain evaluation shows an increased accuracy (+7.79) compared with competitors on the evaluation suite CompGuessWhat?!; a transfer evaluation shows improved performance for VQA on the TDIUC dataset in terms of harmonic average accuracy (+5.31) thanks to more fine-grained object representations learned via SPIEL.

Original languageEnglish
Title of host publicationProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics
PublisherAssociation for Computational Linguistics
Pages2135-2144
Number of pages10
ISBN (Electronic)9781954085022
Publication statusPublished - Apr 2021
Event16th Conference of the European Chapter of the Associationfor Computational Linguistics 2021 - Virtual, Online
Duration: 19 Apr 202123 Apr 2021

Conference

Conference16th Conference of the European Chapter of the Associationfor Computational Linguistics 2021
Abbreviated titleEACL 2021
CityVirtual, Online
Period19/04/2123/04/21

ASJC Scopus subject areas

  • Software
  • Computational Theory and Mathematics
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'An empirical study on the generalization power of neural representations learned via visual guessing games'. Together they form a unique fingerprint.

Cite this