One of the most difficult aspects of developing matching systems – whether for matching ontologies or for other types of mismatched data – is evaluation. The accuracy of ontology matchers are usually evaluated by measuring the results produced by the systems against reference ontologies, but reference ontologies are expensive and difficult to create.In this paper we discuss the use of crptr, a system that corrupts data of different sorts in order to mimic the kind of differences one might expect to find between different data sources on related topics. This automatically creates a map between the original and the corrupted data source, and matching systems can be evaluated by comparing their output to this map. We describe the extension of crptr to ontology-based data and query mismatch, and then discuss how it could be extended to other kinds of matching including ontology matching.
|Title of host publication||The Fourteenth International Workshop on Ontology Matching|
|Publication status||Accepted/In press - 25 Jul 2019|
|Event||Fourteenth International Workshop on Ontology Matching - Auckland , New Zealand|
Duration: 26 Oct 2019 → 26 Oct 2019
|Workshop||Fourteenth International Workshop on Ontology Matching|
|Period||26/10/19 → 26/10/19|
McNeill, F., Bental, D., Gray, A. J. G., Jedrzejczyk, S., & Alsadeeqi, A. (Accepted/In press). Generating corrupted data sources for the evaluation of matching systems. In The Fourteenth International Workshop on Ontology Matching