Java Unit Testing Tool Competition - Eighth Round

Xavier Devroey, Sebastiano Panichella, Alessio Gambi

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

12 Citations (Scopus)
79 Downloads (Pure)


We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our method- ology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.
Original languageEnglish
Title of host publicationProceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020
PublisherACM DL
Number of pages4
ISBN (Electronic)9781450379632
Publication statusPublished - 2020
Event42nd International Conference on Software Engineering: ICSE 2020 - Seoul, Korea, Republic of
Duration: 27 Jun 202019 Jul 2020


Conference42nd International Conference on Software Engineering
Country/TerritoryKorea, Republic of
OtherVirtual/online event due to COVID-19 online presentations

Bibliographical note

Virtual/online event due to COVID-19


  • tool competition
  • benchmark
  • software testing
  • test case generation
  • unit testing
  • Java
  • JUnit


Dive into the research topics of 'Java Unit Testing Tool Competition - Eighth Round'. Together they form a unique fingerprint.

Cite this