Use of AI-driven Code Generation Models in Teaching and Learning Programming: a Systematic Literature Review

Doga Cambaz, Xiaoling Zhang

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

The recent emergence of LLM-based code generation models can potentially transform programming education. To pinpoint the current state of research on using LLM-based code generators to support the teaching and learning of programming, we conducted a systematic literature review of 21 papers published since 2018. The review focuses on (1) the teaching and learning practices in programming education that utilized LLM-based code generation models, (2) characteristics and (3) performance indicators of the models, and (4) aspects to consider when utilizing the models in programming education, including the risks and challenges. We found that the most commonly reported uses of LLM-based code generation models for teachers are generating assignments and evaluating student work, while for students, the models function as virtual tutors. We identified that the models exhibit accuracy limitations; generated content often contains minor errors that are manageable by instructors but pose risks for novice learners. Moreover, risks such as academic misconduct and over-reliance on the models are critical when considering integrating these models into education. Overall, LLM-based code generation models can be an assistive tool for both learners and instructors if the risks are mitigated.

Original languageEnglish
Title of host publicationSIGCSE 2024 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education
PublisherAssociation for Computing Machinery (ACM)
Pages172-178
Number of pages7
ISBN (Electronic)9798400704239
DOIs
Publication statusPublished - 2024
Event55th ACM Technical Symposium on Computer Science Education, SIGCSE 2024 - Portland, United States
Duration: 20 Mar 202423 Mar 2024

Publication series

NameSIGCSE 2024 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education
Volume1

Conference

Conference55th ACM Technical Symposium on Computer Science Education, SIGCSE 2024
Country/TerritoryUnited States
CityPortland
Period20/03/2423/03/24

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • artificial intelligence in education
  • code generation models
  • large language models
  • programming education
  • systematic review

Fingerprint

Dive into the research topics of 'Use of AI-driven Code Generation Models in Teaching and Learning Programming: a Systematic Literature Review'. Together they form a unique fingerprint.

Cite this