The Half-Life of MOOC Knowledge:  A Randomized Trial Evaluating the Testing Effect in MOOCs

Dan Davis; René F. Kizilcec; Claudia Hauff; Geert-Jan Houben

doi:10.1145/3170358.3170383

The Half-Life of MOOC Knowledge: A Randomized Trial Evaluating the Testing Effect in MOOCs

Dan Davis, René F. Kizilcec, Claudia Hauff, Geert-Jan Houben

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

45 Downloads (Pure)

Abstract

Retrieval practice has been established in the learning sciences as one of the most effective strategies to facilitate robust learning in traditional classroom contexts. The cognitive theory underpinning the "testing effect" states that actively recalling information is more effective than passively revisiting materials for storing information in long-term memory. We document the design, deployment, and evaluation of an Adaptive Retrieval Practice System (ARPS) in a MOOC. This push-based system leverages the testing effect to promote learner engagement and achievement by intelligently delivering quiz questions from prior course units to learners throughout the course. We conducted an experiment in which learners were randomized to receive ARPS in a MOOC to track their performance and behavior compared to a control group. In contrast to prior literature, we find no significant effect of retrieval practice in this MOOC environment. In the treatment condition, passing learners engaged more with ARPS but exhibited similar levels of knowledge retention as non-passing learners.

Original language	English
Title of host publication	LAK'18 Proceedings of the 8th International Conference on Learning Analytics and Knowledge
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	1-10
Number of pages	10
ISBN (Print)	978-1-4503-6400-3
DOIs	https://doi.org/10.1145/3170358.3170383
Publication status	Published - 2018
Event	LAK 2018 : The 8th International Conference on Learning Analytics and Knowledge - Sydney, Australia Duration: 7 Mar 2018 → 9 Mar 2018

Conference

Conference	LAK 2018
Country/Territory	Australia
City	Sydney
Period	7/03/18 → 9/03/18

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Retrieval Practice
Testing Effect
Experiment
Knowledge Retention

Access to Document

10.1145/3170358.3170383

p1-davisFinal published version, 1.78 MB

Cite this

@inproceedings{2a955b98f01a41658de6d3b747be05f3,

title = "The Half-Life of MOOC Knowledge: A Randomized Trial Evaluating the Testing Effect in MOOCs",

abstract = "Retrieval practice has been established in the learning sciences as one of the most effective strategies to facilitate robust learning in traditional classroom contexts. The cognitive theory underpinning the {"}testing effect{"} states that actively recalling information is more effective than passively revisiting materials for storing information in long-term memory. We document the design, deployment, and evaluation of an Adaptive Retrieval Practice System (ARPS) in a MOOC. This push-based system leverages the testing effect to promote learner engagement and achievement by intelligently delivering quiz questions from prior course units to learners throughout the course. We conducted an experiment in which learners were randomized to receive ARPS in a MOOC to track their performance and behavior compared to a control group. In contrast to prior literature, we find no significant effect of retrieval practice in this MOOC environment. In the treatment condition, passing learners engaged more with ARPS but exhibited similar levels of knowledge retention as non-passing learners.",

keywords = "Retrieval Practice, Testing Effect, Experiment, Knowledge Retention",

author = "Dan Davis and Kizilcec, {Ren{\'e} F.} and Claudia Hauff and Geert-Jan Houben",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; LAK 2018 : The 8th International Conference on Learning Analytics and Knowledge ; Conference date: 07-03-2018 Through 09-03-2018",

year = "2018",

doi = "10.1145/3170358.3170383",

language = "English",

isbn = "978-1-4503-6400-3 ",

pages = "1--10",

booktitle = "LAK'18 Proceedings of the 8th International Conference on Learning Analytics and Knowledge",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

}

Davis, D, Kizilcec, RF, Hauff, C & Houben, G-J 2018, The Half-Life of MOOC Knowledge: A Randomized Trial Evaluating the Testing Effect in MOOCs. in LAK'18 Proceedings of the 8th International Conference on Learning Analytics and Knowledge. Association for Computing Machinery (ACM), New York, pp. 1-10, LAK 2018 , Sydney, Australia, 7/03/18. https://doi.org/10.1145/3170358.3170383

The Half-Life of MOOC Knowledge: A Randomized Trial Evaluating the Testing Effect in MOOCs. / Davis, Dan; Kizilcec, René F.; Hauff, Claudia et al.
LAK'18 Proceedings of the 8th International Conference on Learning Analytics and Knowledge. New York: Association for Computing Machinery (ACM), 2018. p. 1-10.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - The Half-Life of MOOC Knowledge

T2 - LAK 2018

AU - Davis, Dan

AU - Kizilcec, René F.

AU - Hauff, Claudia

AU - Houben, Geert-Jan

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2018

Y1 - 2018

N2 - Retrieval practice has been established in the learning sciences as one of the most effective strategies to facilitate robust learning in traditional classroom contexts. The cognitive theory underpinning the "testing effect" states that actively recalling information is more effective than passively revisiting materials for storing information in long-term memory. We document the design, deployment, and evaluation of an Adaptive Retrieval Practice System (ARPS) in a MOOC. This push-based system leverages the testing effect to promote learner engagement and achievement by intelligently delivering quiz questions from prior course units to learners throughout the course. We conducted an experiment in which learners were randomized to receive ARPS in a MOOC to track their performance and behavior compared to a control group. In contrast to prior literature, we find no significant effect of retrieval practice in this MOOC environment. In the treatment condition, passing learners engaged more with ARPS but exhibited similar levels of knowledge retention as non-passing learners.

AB - Retrieval practice has been established in the learning sciences as one of the most effective strategies to facilitate robust learning in traditional classroom contexts. The cognitive theory underpinning the "testing effect" states that actively recalling information is more effective than passively revisiting materials for storing information in long-term memory. We document the design, deployment, and evaluation of an Adaptive Retrieval Practice System (ARPS) in a MOOC. This push-based system leverages the testing effect to promote learner engagement and achievement by intelligently delivering quiz questions from prior course units to learners throughout the course. We conducted an experiment in which learners were randomized to receive ARPS in a MOOC to track their performance and behavior compared to a control group. In contrast to prior literature, we find no significant effect of retrieval practice in this MOOC environment. In the treatment condition, passing learners engaged more with ARPS but exhibited similar levels of knowledge retention as non-passing learners.

KW - Retrieval Practice

KW - Testing Effect

KW - Experiment

KW - Knowledge Retention

U2 - 10.1145/3170358.3170383

DO - 10.1145/3170358.3170383

M3 - Conference contribution

SN - 978-1-4503-6400-3

SP - 1

EP - 10

BT - LAK'18 Proceedings of the 8th International Conference on Learning Analytics and Knowledge

PB - Association for Computing Machinery (ACM)

CY - New York

Y2 - 7 March 2018 through 9 March 2018

ER -