Generating corrupted data sources for the evaluation of matching systems

Fiona McNeill, Diana Bental, Alasdair J. G. Gray, Sabina Jedrzejczyk, Ahmad Alsadeeqi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Downloads (Pure)

Abstract

One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.
Original languageEnglish
Title of host publicationThe Fourteenth International Workshop on Ontology Matching
Publication statusAccepted/In press - 25 Jul 2019
EventFourteenth International Workshop on Ontology Matching - Auckland , New Zealand
Duration: 26 Oct 201926 Oct 2019

Workshop

WorkshopFourteenth International Workshop on Ontology Matching
Abbreviated titleOM-2019
CountryNew Zealand
CityAuckland
Period26/10/1926/10/19

Fingerprint

Ontology
Computer systems

Cite this

McNeill, F., Bental, D., Gray, A. J. G., Jedrzejczyk, S., & Alsadeeqi, A. (Accepted/In press). Generating corrupted data sources for the evaluation of matching systems. In The Fourteenth International Workshop on Ontology Matching
McNeill, Fiona ; Bental, Diana ; Gray, Alasdair J. G. ; Jedrzejczyk, Sabina ; Alsadeeqi, Ahmad. / Generating corrupted data sources for the evaluation of matching systems. The Fourteenth International Workshop on Ontology Matching. 2019.
@inproceedings{412961f23c7745d7909c9980b7a461b8,
title = "Generating corrupted data sources for the evaluation of matching systems",
abstract = "One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.",
author = "Fiona McNeill and Diana Bental and Gray, {Alasdair J. G.} and Sabina Jedrzejczyk and Ahmad Alsadeeqi",
year = "2019",
month = "7",
day = "25",
language = "English",
booktitle = "The Fourteenth International Workshop on Ontology Matching",

}

McNeill, F, Bental, D, Gray, AJG, Jedrzejczyk, S & Alsadeeqi, A 2019, Generating corrupted data sources for the evaluation of matching systems. in The Fourteenth International Workshop on Ontology Matching. Fourteenth International Workshop on Ontology Matching, Auckland , New Zealand, 26/10/19.

Generating corrupted data sources for the evaluation of matching systems. / McNeill, Fiona; Bental, Diana; Gray, Alasdair J. G.; Jedrzejczyk, Sabina; Alsadeeqi, Ahmad.

The Fourteenth International Workshop on Ontology Matching. 2019.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Generating corrupted data sources for the evaluation of matching systems

AU - McNeill, Fiona

AU - Bental, Diana

AU - Gray, Alasdair J. G.

AU - Jedrzejczyk, Sabina

AU - Alsadeeqi, Ahmad

PY - 2019/7/25

Y1 - 2019/7/25

N2 - One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.

AB - One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.

M3 - Conference contribution

BT - The Fourteenth International Workshop on Ontology Matching

ER -

McNeill F, Bental D, Gray AJG, Jedrzejczyk S, Alsadeeqi A. Generating corrupted data sources for the evaluation of matching systems. In The Fourteenth International Workshop on Ontology Matching. 2019