DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests

Devjeet Roy, Ziyi Zhang, Maggie Ma, Venera Arnaoudova, Annibale Panichella, Sebastiano Panichella, Danielle Gonzalez, Mehdi Mirakhorli

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

24 Citations (Scopus)
137 Downloads (Pure)

Abstract

Automated test case generation tools have been successfully proposed to reduce the amount of human and infrastructure resources required to write and run test cases. However, recent studies demonstrate that the readability of generated tests is very limited due to (i) uninformative identifiers and (ii) lack of proper documentation. Prior studies proposed techniques to improve test readability by either generating natural language summaries or meaningful methods names. While these approaches are shown to improve test readability, they are also affected by two limitations: (1) generated summaries are often perceived as too verbose and redundant by developers, and (2) readable tests require both proper method names but also meaningful identifiers (within-method readability). In this work, we combine template based methods and Deep Learning (DL) approaches to automatically generate test case scenarios (elicited from natural language patterns of test case statements) as well as to train DL models on path-based representations of source code to generate meaningful identifier names. Our approach, called DeepTC-Enhancer, recommends documentation and identifier names with the ultimate goal of enhancing readability of automatically generated test cases. An empirical evaluation with 36 external and internal developers shows that (1) DeepTC-Enhancer outperforms significantly the baseline approach for generating summaries and performs equally with the baseline approach for test case renaming, (2) the transformation proposed by DeepTC-Enhancer results in a significant increase in readability of automatically generated test cases, and (3) there is a significant difference in the feature preferences between external and internal developers.

Original languageEnglish
Title of host publicationProceedings - 2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020
PublisherIEEE / ACM
Pages287-298
Number of pages12
ISBN (Electronic)978-1-4503-6768-4
ISBN (Print)978-1-7281-7281-1
DOIs
Publication statusPublished - 2020
Event35th IEEE/ACM
International Conference on Automated Software Engineering (ASE ’20),
-
Duration: 21 Sept 202025 Sept 2020
Conference number: 35

Publication series

NameProceedings - 2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020

Conference

Conference35th IEEE/ACM
International Conference on Automated Software Engineering (ASE ’20),
Abbreviated titleASE ’20
Period21/09/2025/09/20
OtherVirtual/online event due to COVID-19

Bibliographical note

Accepted author manuscript

Keywords

  • deep learning
  • empirical study
  • evolution
  • maintenance
  • program comprehension
  • software testing
  • test case generation

Fingerprint

Dive into the research topics of 'DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests'. Together they form a unique fingerprint.

Cite this