Shaken, Not Stirred: How Developers Like Their Amplified Tests

Carolin Brandt*, Ali Khatami, Mairieli Wessel, Andy Zaidman

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review


Test amplification makes systematic changes to existing, manually written tests to provide tests complementary to an automated test suite. We consider developer-centric test amplification, where the developer explores, judges and edits the amplified tests before adding them to their maintained test suite. However, it is as yet unclear which kind of selection and editing steps developers take before including an amplified test into the test suite. In this paper we conduct an open source contribution study, amplifying tests of open source Java projects from GitHub. We report which deficiencies we observe in the amplified tests while manually filtering and editing them to open 39 pull requests with amplified tests. We present a detailed analysis of the maintainer's feedback regarding proposed changes, requested information, and expressed judgment. Our observations provide a basis for practitioners to take an informed decision on whether to adopt developer-centric test amplification. As several of the edits we observe are based on the developer's understanding of the amplified test, we conjecture that developer-centric test amplification should invest in supporting the developer to understand the amplified tests.

Original languageEnglish
Pages (from-to)1264-1280
Number of pages17
JournalIEEE Transactions on Software Engineering
Issue number5
Publication statusPublished - 2024

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • automatic test generation
  • developer-centric test amplification
  • software testing


Dive into the research topics of 'Shaken, Not Stirred: How Developers Like Their Amplified Tests'. Together they form a unique fingerprint.

Cite this