EMit at EVALITA 2023: Overview of the Categorical Emotion Detection in Italian Social Media Task

Oscar Araque; Simona Frenda; Rachele Sprugnoli; Debora Nozza; Viviana Patti

EMit at EVALITA 2023: Overview of the Categorical Emotion Detection in Italian Social Media Task

Oscar Araque^*
, Simona Frenda
, Rachele Sprugnoli
, Debora Nozza
, Viviana Patti

^*Corresponding author for this work

Research output: Contribution to journal › Conference article › peer-review

7 Citations (Scopus)

Abstract

The Emotions in Italian (EMit) task is the first edition of a shared task on emotion analysis and opinion mining in Italian messages at EVALITA 2023. EMit presents two subtasks: (i) Subtask A, that consists in an emotion detection challenge, and (ii) Subtask B, that introduces a novel problem of target detection of the expressed emotion. Additionally, EMit challenges systems with a thorough in-domain and out-of-domain evaluation, probing the generalization capabilities of the submitted solutions. In general, 4 teams have participated in Subtask A, achieving a macro-averaged f-score of 0.6028 and 0.4977 in the in-domain and out-of-domain sets, respectively. In Subtask B a team has participated, obtaining 0.6459 in the in-domain set and 0.3223 in the out-of-domain set as macro-averaged f-scores. The obtained results indicate that further work needs to be done to solve the task, opening new avenues of research.

Original language	English
Journal	CEUR Workshop Proceedings
Volume	3473
Publication status	Published - 7 Sept 2023
Event	8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian 2023 - Parma, Italy Duration: 7 Sept 2023 → 8 Sept 2023 https://www.evalita.it/campaigns/evalita-2023/

Keywords

Emotion detection
Emotion target detection
Sentiment Analysis
User-generated contents

ASJC Scopus subject areas

General Computer Science

Access to Document

Download pdf
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Final published version, 262 KBLicence: CC BY

https://ceur-ws.org/Vol-3473/Licence: CC BY

Cite this

@article{48ca551d710e4e32a96aea4d56ed0db8,

title = "EMit at EVALITA 2023: Overview of the Categorical Emotion Detection in Italian Social Media Task",

abstract = "The Emotions in Italian (EMit) task is the first edition of a shared task on emotion analysis and opinion mining in Italian messages at EVALITA 2023. EMit presents two subtasks: (i) Subtask A, that consists in an emotion detection challenge, and (ii) Subtask B, that introduces a novel problem of target detection of the expressed emotion. Additionally, EMit challenges systems with a thorough in-domain and out-of-domain evaluation, probing the generalization capabilities of the submitted solutions. In general, 4 teams have participated in Subtask A, achieving a macro-averaged f-score of 0.6028 and 0.4977 in the in-domain and out-of-domain sets, respectively. In Subtask B a team has participated, obtaining 0.6459 in the in-domain set and 0.3223 in the out-of-domain set as macro-averaged f-scores. The obtained results indicate that further work needs to be done to solve the task, opening new avenues of research.",

keywords = "Emotion detection, Emotion target detection, Sentiment Analysis, User-generated contents",

author = "Oscar Araque and Simona Frenda and Rachele Sprugnoli and Debora Nozza and Viviana Patti",

note = "Publisher Copyright: {\textcopyright} 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).; 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian 2023, EVALITA 2023 ; Conference date: 07-09-2023 Through 08-09-2023",

year = "2023",

month = sep,

day = "7",

language = "English",

volume = "3473",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

url = "https://www.evalita.it/campaigns/evalita-2023/",

}

TY - JOUR

T1 - EMit at EVALITA 2023: Overview of the Categorical Emotion Detection in Italian Social Media Task

AU - Araque, Oscar

AU - Frenda, Simona

AU - Sprugnoli, Rachele

AU - Nozza, Debora

AU - Patti, Viviana

PY - 2023/9/7

Y1 - 2023/9/7

N2 - The Emotions in Italian (EMit) task is the first edition of a shared task on emotion analysis and opinion mining in Italian messages at EVALITA 2023. EMit presents two subtasks: (i) Subtask A, that consists in an emotion detection challenge, and (ii) Subtask B, that introduces a novel problem of target detection of the expressed emotion. Additionally, EMit challenges systems with a thorough in-domain and out-of-domain evaluation, probing the generalization capabilities of the submitted solutions. In general, 4 teams have participated in Subtask A, achieving a macro-averaged f-score of 0.6028 and 0.4977 in the in-domain and out-of-domain sets, respectively. In Subtask B a team has participated, obtaining 0.6459 in the in-domain set and 0.3223 in the out-of-domain set as macro-averaged f-scores. The obtained results indicate that further work needs to be done to solve the task, opening new avenues of research.

AB - The Emotions in Italian (EMit) task is the first edition of a shared task on emotion analysis and opinion mining in Italian messages at EVALITA 2023. EMit presents two subtasks: (i) Subtask A, that consists in an emotion detection challenge, and (ii) Subtask B, that introduces a novel problem of target detection of the expressed emotion. Additionally, EMit challenges systems with a thorough in-domain and out-of-domain evaluation, probing the generalization capabilities of the submitted solutions. In general, 4 teams have participated in Subtask A, achieving a macro-averaged f-score of 0.6028 and 0.4977 in the in-domain and out-of-domain sets, respectively. In Subtask B a team has participated, obtaining 0.6459 in the in-domain set and 0.3223 in the out-of-domain set as macro-averaged f-scores. The obtained results indicate that further work needs to be done to solve the task, opening new avenues of research.

KW - Emotion detection

KW - Emotion target detection

KW - Sentiment Analysis

KW - User-generated contents

UR - https://www.scopus.com/pages/publications/85173561074

M3 - Conference article

AN - SCOPUS:85173561074

SN - 1613-0073

VL - 3473

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

T2 - 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian 2023

Y2 - 7 September 2023 through 8 September 2023

ER -

EMit at EVALITA 2023: Overview of the Categorical Emotion Detection in Italian Social Media Task

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this