Predicting Quality of Crowdsourced Annotations Using Graph Kernels

Archana Nottamkandath; Jasper Oosterman; Davide Ceolin; Gerben Klaas Dirk de Vries; Wan Fokkink

doi:10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations Using Graph Kernels

Archana Nottamkandath, Jasper Oosterman, Davide Ceolin, Gerben Klaas Dirk de Vries, Wan Fokkink

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

3 Citations (Scopus)

Abstract

Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler-Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75 %. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

Original language	English
Title of host publication	Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings
Editors	Christian Damsgaard Jensen, Stephen Marsh, Theo Dimitrakos, Yuko Murayama
Publisher	Springer
Pages	134-148
Number of pages	15
Volume	454
ISBN (Print)	978-3-319-18490-6
DOIs	https://doi.org/10.1007/978-3-319-18491-3_10
Publication status	Published - 1 May 2015

Publication series

Name	IFIP Advances in Information and Communication Technology
Publisher	Springer

Access to Document

10.1007/978-3-319-18491-3_10

Cite this

Nottamkandath, A., Oosterman, J., Ceolin, D., Vries, G. K. D. D., & Fokkink, W. (2015). Predicting Quality of Crowdsourced Annotations Using Graph Kernels. In C. D. Jensen, S. Marsh, T. Dimitrakos, & Y. Murayama (Eds.), Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings (Vol. 454, pp. 134-148). (IFIP Advances in Information and Communication Technology). Springer. https://doi.org/10.1007/978-3-319-18491-3_10

Nottamkandath, Archana ; Oosterman, Jasper ; Ceolin, Davide et al. / Predicting Quality of Crowdsourced Annotations Using Graph Kernels. Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings. editor / Christian Damsgaard Jensen ; Stephen Marsh ; Theo Dimitrakos ; Yuko Murayama. Vol. 454 Springer, 2015. pp. 134-148 (IFIP Advances in Information and Communication Technology).

@inproceedings{fff170032ce840458227b96cb2a16afe,

title = "Predicting Quality of Crowdsourced Annotations Using Graph Kernels",

abstract = "Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler-Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75 %. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.",

author = "Archana Nottamkandath and Jasper Oosterman and Davide Ceolin and Vries, {Gerben Klaas Dirk de} and Wan Fokkink",

year = "2015",

month = may,

day = "1",

doi = "10.1007/978-3-319-18491-3_10",

language = "English",

isbn = "978-3-319-18490-6",

volume = "454",

series = "IFIP Advances in Information and Communication Technology",

publisher = "Springer",

pages = "134--148",

editor = "Jensen, {Christian Damsgaard} and Stephen Marsh and Theo Dimitrakos and Yuko Murayama",

booktitle = "Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings",

}

Nottamkandath, A, Oosterman, J, Ceolin, D, Vries, GKDD & Fokkink, W 2015, Predicting Quality of Crowdsourced Annotations Using Graph Kernels. in CD Jensen, S Marsh, T Dimitrakos & Y Murayama (eds), Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings. vol. 454, IFIP Advances in Information and Communication Technology, Springer, pp. 134-148. https://doi.org/10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations Using Graph Kernels. / Nottamkandath, Archana; Oosterman, Jasper; Ceolin, Davide et al.
Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings. ed. / Christian Damsgaard Jensen; Stephen Marsh; Theo Dimitrakos; Yuko Murayama. Vol. 454 Springer, 2015. p. 134-148 (IFIP Advances in Information and Communication Technology).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Predicting Quality of Crowdsourced Annotations Using Graph Kernels

AU - Nottamkandath, Archana

AU - Oosterman, Jasper

AU - Ceolin, Davide

AU - Vries, Gerben Klaas Dirk de

AU - Fokkink, Wan

PY - 2015/5/1

Y1 - 2015/5/1

N2 - Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler-Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75 %. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

AB - Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler-Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75 %. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

U2 - 10.1007/978-3-319-18491-3_10

DO - 10.1007/978-3-319-18491-3_10

M3 - Conference contribution

SN - 978-3-319-18490-6

VL - 454

T3 - IFIP Advances in Information and Communication Technology

SP - 134

EP - 148

BT - Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings

A2 - Jensen, Christian Damsgaard

A2 - Marsh, Stephen

A2 - Dimitrakos, Theo

A2 - Murayama, Yuko

PB - Springer

ER -

Nottamkandath A, Oosterman J, Ceolin D, Vries GKDD, Fokkink W. Predicting Quality of Crowdsourced Annotations Using Graph Kernels. In Jensen CD, Marsh S, Dimitrakos T, Murayama Y, editors, Trust Management IX - 9th IFIP WG 11.11 International Conference, IFIPTM 2015, Hamburg, Germany, May 26-28, 2015, Proceedings. Vol. 454. Springer. 2015. p. 134-148. (IFIP Advances in Information and Communication Technology). doi: 10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations Using Graph Kernels

Abstract

Publication series

Access to Document

Fingerprint

Cite this