ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility

Burak Yildiz; Hayley Hung; Jesse H. Krijthe; Cynthia C.S. Liem; Marco Loog; Gosia Migut; Frans A. Oliehoek; Annibale Panichella; Przemysław Pawełczak; Stjepan Picek; Mathijs de Weerdt; Jan van Gemert

doi:10.1007/978-3-030-76423-4_1

ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility

Burak Yildiz^*, Hayley Hung, Jesse H. Krijthe, Cynthia C.S. Liem, Marco Loog, Gosia Migut, Frans A. Oliehoek, Annibale Panichella, Przemysław Pawełczak, Stjepan Picek, Mathijs de Weerdt, Jan van Gemert

^*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

5 Citations (Scopus)

112 Downloads (Pure)

Abstract

We present ReproducedPapers.org : an open online repository for teaching and structuring machine learning reproducibility. We evaluate doing a reproduction project among students and the added value of an online reproduction repository among AI researchers. We use anonymous self-assessment surveys and obtained 144 responses. Results suggest that students who do a reproduction project place more value on scientific reproductions and become more critical thinkers. Students and AI researchers agree that our online reproduction repository is valuable.

Original language	English
Title of host publication	Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers
Editors	Bertrand Kerautret, Miguel Colom, Adrien Krähenbühl, Daniel Lopresti, Pascal Monasse, Hugues Talbot
Publisher	Springer
Pages	3-11
Number of pages	9
ISBN (Print)	9783030764227
DOIs	https://doi.org/10.1007/978-3-030-76423-4_1
Publication status	Published - 2021
Event	3rd International Workshop on Reproducible Research in Pattern Recognition, RRPR 2021 - Virtual, Online Duration: 11 Jan 2021 → 11 Jan 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12636 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	3rd International Workshop on Reproducible Research in Pattern Recognition, RRPR 2021
City	Virtual, Online
Period	11/01/21 → 11/01/21

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Machine learning
Online repository
Reproducibility

Access to Document

10.1007/978-3-030-76423-4_1

Yildiz2021_Chapter_ReproducedPapersOrgOpenlyTeachFinal published version, 969 KB

Cite this

Yildiz, B., Hung, H., Krijthe, J. H., Liem, C. C. S., Loog, M., Migut, G., Oliehoek, F. A., Panichella, A., Pawełczak, P., Picek, S., de Weerdt, M., & van Gemert, J. (2021). ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility. In B. Kerautret, M. Colom, A. Krähenbühl, D. Lopresti, P. Monasse, & H. Talbot (Eds.), Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers (pp. 3-11). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12636 LNCS). Springer. https://doi.org/10.1007/978-3-030-76423-4_1

Yildiz, Burak ; Hung, Hayley ; Krijthe, Jesse H. et al. / ReproducedPapers.org : Openly Teaching and Structuring Machine Learning Reproducibility. Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers. editor / Bertrand Kerautret ; Miguel Colom ; Adrien Krähenbühl ; Daniel Lopresti ; Pascal Monasse ; Hugues Talbot. Springer, 2021. pp. 3-11 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2dcec1b3db864b44af1ff287cf424b2c,

title = "ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility",

abstract = "We present ReproducedPapers.org : an open online repository for teaching and structuring machine learning reproducibility. We evaluate doing a reproduction project among students and the added value of an online reproduction repository among AI researchers. We use anonymous self-assessment surveys and obtained 144 responses. Results suggest that students who do a reproduction project place more value on scientific reproductions and become more critical thinkers. Students and AI researchers agree that our online reproduction repository is valuable.",

keywords = "Machine learning, Online repository, Reproducibility",

author = "Burak Yildiz and Hayley Hung and Krijthe, {Jesse H.} and Liem, {Cynthia C.S.} and Marco Loog and Gosia Migut and Oliehoek, {Frans A.} and Annibale Panichella and Przemys{\l}aw Pawe{\l}czak and Stjepan Picek and {de Weerdt}, Mathijs and {van Gemert}, Jan",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ; 3rd International Workshop on Reproducible Research in Pattern Recognition, RRPR 2021 ; Conference date: 11-01-2021 Through 11-01-2021",

year = "2021",

doi = "10.1007/978-3-030-76423-4_1",

language = "English",

isbn = "9783030764227",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "3--11",

editor = "Bertrand Kerautret and Miguel Colom and Adrien Kr{\"a}henb{\"u}hl and Daniel Lopresti and Pascal Monasse and Hugues Talbot",

booktitle = "Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers",

}

Yildiz, B, Hung, H , Krijthe, JH , Liem, CCS , Loog, M , Migut, G , Oliehoek, FA , Panichella, A , Pawełczak, P , Picek, S , de Weerdt, M & van Gemert, J 2021, ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility. in B Kerautret, M Colom, A Krähenbühl, D Lopresti, P Monasse & H Talbot (eds), Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12636 LNCS, Springer, pp. 3-11, 3rd International Workshop on Reproducible Research in Pattern Recognition, RRPR 2021, Virtual, Online, 11/01/21. https://doi.org/10.1007/978-3-030-76423-4_1

ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility. / Yildiz, Burak; Hung, Hayley ; Krijthe, Jesse H. et al.
Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers. ed. / Bertrand Kerautret; Miguel Colom; Adrien Krähenbühl; Daniel Lopresti; Pascal Monasse; Hugues Talbot. Springer, 2021. p. 3-11 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12636 LNCS).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - ReproducedPapers.org

T2 - 3rd International Workshop on Reproducible Research in Pattern Recognition, RRPR 2021

AU - Yildiz, Burak

AU - Hung, Hayley

AU - Krijthe, Jesse H.

AU - Liem, Cynthia C.S.

AU - Loog, Marco

AU - Migut, Gosia

AU - Oliehoek, Frans A.

AU - Panichella, Annibale

AU - Pawełczak, Przemysław

AU - Picek, Stjepan

AU - de Weerdt, Mathijs

AU - van Gemert, Jan

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2021

Y1 - 2021

N2 - We present ReproducedPapers.org : an open online repository for teaching and structuring machine learning reproducibility. We evaluate doing a reproduction project among students and the added value of an online reproduction repository among AI researchers. We use anonymous self-assessment surveys and obtained 144 responses. Results suggest that students who do a reproduction project place more value on scientific reproductions and become more critical thinkers. Students and AI researchers agree that our online reproduction repository is valuable.

AB - We present ReproducedPapers.org : an open online repository for teaching and structuring machine learning reproducibility. We evaluate doing a reproduction project among students and the added value of an online reproduction repository among AI researchers. We use anonymous self-assessment surveys and obtained 144 responses. Results suggest that students who do a reproduction project place more value on scientific reproductions and become more critical thinkers. Students and AI researchers agree that our online reproduction repository is valuable.

KW - Machine learning

KW - Online repository

KW - Reproducibility

UR - http://www.scopus.com/inward/record.url?scp=85111162885&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-76423-4_1

DO - 10.1007/978-3-030-76423-4_1

M3 - Conference contribution

AN - SCOPUS:85111162885

SN - 9783030764227

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 11

BT - Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers

A2 - Kerautret, Bertrand

A2 - Colom, Miguel

A2 - Krähenbühl, Adrien

A2 - Lopresti, Daniel

A2 - Monasse, Pascal

A2 - Talbot, Hugues

PB - Springer

Y2 - 11 January 2021 through 11 January 2021

ER -

Yildiz B, Hung H , Krijthe JH , Liem CCS , Loog M , Migut G et al. ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility. In Kerautret B, Colom M, Krähenbühl A, Lopresti D, Monasse P, Talbot H, editors, Reproducible Research in Pattern Recognition - 3rd International Workshop, RRPR 2021, Revised Selected Papers. Springer. 2021. p. 3-11. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-76423-4_1

ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this