What does this Python code do? An exploratory analysis of novice students' code explanations

Motivation. Code reading skills are important for comprehension. Explain-in-plain-English tasks (EiPE) are one type of reading exercises that show promising results on the ability of such exercises to differentiate between particular levels of code comprehension. Code reading/explaining skills also correlate with code writing skills. Objective. This paper aims to provide insight in what novice students express in their explanations after reading a piece of code, and what these insights can tell us about how the students comprehend code. Method. We performed an exploratory analysis on four reading assignments extracted from a university-level beginners course in Python programming. We paid specific attention to 1) the core focus of student answers, 2) elements of the code that are often included or omitted, and 3) errors and misconceptions students may present. Results. We found that students prioritize the output that is generated by print-statements in a program. This is indication that these statements may have the ability to aid students make sense of code. Furthermore, students appear to be selective about which elements they find important in their explanation. Assigning variables and asking input was less often included, whereas control-flow elements, print statements and function definitions were more often included. Finally, students were easily confused or distracted by lines of code that seemed to interfere with the newly learned programming constructs. Also domain knowledge (outside of programming) both positively and negatively interfered with reading and interpreting the code. Discussion. Our results pave the way towards a better understanding of how students understand code by reading and of how an exercise containing self-explanations after reading, as a teaching instrument, may be useful to both teachers and students in programming education.