Generating corrupted data sources for the evaluation of matching systems

Fiona McNeill; Diana Bental; Alasdair J. G. Gray; Sabina Jedrzejczyk; Ahmad Alsadeeqi

Generating corrupted data sources for the evaluation of matching systems

Fiona McNeill, Diana Bental, Alasdair J. G. Gray, Sabina Jedrzejczyk, Ahmad Alsadeeqi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

18 Downloads (Pure)

Abstract

One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.

Original language	English
Title of host publication	The Fourteenth International Workshop on Ontology Matching
Publication status	Accepted/In press - 25 Jul 2019
Event	Fourteenth International Workshop on Ontology Matching - Auckland , New Zealand Duration: 26 Oct 2019 → 26 Oct 2019

Workshop

Workshop	Fourteenth International Workshop on Ontology Matching
Abbreviated title	OM-2019
Country	New Zealand
City	Auckland
Period	26/10/19 → 26/10/19

Access to Document

Download pdfAccepted author manuscript, 198 KB

Fingerprint Dive into the research topics of 'Generating corrupted data sources for the evaluation of matching systems'. Together they form a unique fingerprint.

Cite this

@inproceedings{412961f23c7745d7909c9980b7a461b8,

title = "Generating corrupted data sources for the evaluation of matching systems",

abstract = "One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.",

author = "Fiona McNeill and Diana Bental and Gray, {Alasdair J. G.} and Sabina Jedrzejczyk and Ahmad Alsadeeqi",

year = "2019",

month = jul,

day = "25",

language = "English",

booktitle = "The Fourteenth International Workshop on Ontology Matching",

note = "Fourteenth International Workshop on Ontology Matching, OM-2019 ; Conference date: 26-10-2019 Through 26-10-2019",

}

TY - GEN

T1 - Generating corrupted data sources for the evaluation of matching systems

AU - McNeill, Fiona

AU - Bental, Diana

AU - Gray, Alasdair J. G.

AU - Jedrzejczyk, Sabina

AU - Alsadeeqi, Ahmad

PY - 2019/7/25

Y1 - 2019/7/25

N2 - One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.

AB - One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.

M3 - Conference contribution

BT - The Fourteenth International Workshop on Ontology Matching

T2 - Fourteenth International Workshop on Ontology Matching

Y2 - 26 October 2019 through 26 October 2019

ER -