Java Unit Testing Tool Competition - Seventh Round

Fitsum  Kifetew; Xavier Devroey; Urko  Rueda

doi:10.1109/SBST.2019.00014

Java Unit Testing Tool Competition - Seventh Round

Fitsum Kifetew, Xavier Devroey, Urko Rueda

Software Engineering

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

112 Downloads (Pure)

Abstract

We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

Original language	English
Title of host publication	2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST)
Subtitle of host publication	Proceedings
Place of Publication	Piscataway
Publisher	IEEE
Pages	15-20
Number of pages	6
ISBN (Electronic)	978-1-7281-2233-5
ISBN (Print)	978-1-7281-2234-2
DOIs	https://doi.org/10.1109/SBST.2019.00014
Publication status	Published - 2019
Event	SBST '19: 12th International Workshop on Search-Based Software Testing - Montreal, Canada Duration: 27 May 2019 → 27 May 2019 Conference number: 12

Conference

Conference	SBST '19: 12th International Workshop on Search-Based Software Testing
Abbreviated title	SBST '19
Country	Canada
City	Montreal
Period	27/05/19 → 27/05/19

Keywords

Java
automation
benchmark
combined performance
mutation testing
statistical analysis
tool competition
unit testing

Access to Document

10.1109/SBST.2019.00014

sbst-contestAccepted author manuscript, 317 KB

Link to publication in Scopus

Fingerprint Dive into the research topics of 'Java Unit Testing Tool Competition - Seventh Round'. Together they form a unique fingerprint.

Cite this

@inproceedings{e265e18a6dd24f9d8a013229cbc4066f,

title = "Java Unit Testing Tool Competition - Seventh Round",

abstract = "We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.",

keywords = "Java, automation, benchmark, combined performance, mutation testing, statistical analysis, tool competition, unit testing",

author = "Fitsum Kifetew and Xavier Devroey and Urko Rueda",

year = "2019",

doi = "10.1109/SBST.2019.00014",

language = "English",

isbn = "978-1-7281-2234-2",

pages = "15--20",

booktitle = "2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST)",

publisher = "IEEE ",

address = "United States",

note = "SBST '19: 12th International Workshop on Search-Based Software Testing , SBST '19 ; Conference date: 27-05-2019 Through 27-05-2019",

}

Kifetew, F, Devroey, X & Rueda, U 2019, Java Unit Testing Tool Competition - Seventh Round. in 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST): Proceedings., 8812209, IEEE , Piscataway, pp. 15-20, SBST '19: 12th International Workshop on Search-Based Software Testing , Montreal, Canada, 27/05/19. https://doi.org/10.1109/SBST.2019.00014

Java Unit Testing Tool Competition - Seventh Round. / Kifetew, Fitsum ; Devroey, Xavier; Rueda, Urko .

2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST): Proceedings. Piscataway : IEEE , 2019. p. 15-20 8812209.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Java Unit Testing Tool Competition - Seventh Round

AU - Kifetew, Fitsum

AU - Devroey, Xavier

AU - Rueda, Urko

N1 - Conference code: 12

PY - 2019

Y1 - 2019

N2 - We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

AB - We report on the results of the seventh edition of the JUnit tool competition. This year, four tools were executed on a benchmark with (i) new classes, selected from real-world software projects, and (ii) challenging classes from the previous edition. We use Randoop and manual test suites from the projects as baselines. Given the interesting findings of last year, we analyzed the effectiveness of the combined test suites generated by all competing tools and compared; results are confronted with the manual test suites of the projects, as well as those generated by the competing tools. This paper describes our methodology and the results, highlight challenges faced during the contest.

KW - Java

KW - automation

KW - benchmark

KW - combined performance

KW - mutation testing

KW - statistical analysis

KW - tool competition

KW - unit testing

UR - http://www.scopus.com/inward/record.url?scp=85072556942&partnerID=8YFLogxK

U2 - 10.1109/SBST.2019.00014

DO - 10.1109/SBST.2019.00014

M3 - Conference contribution

SN - 978-1-7281-2234-2

SP - 15

EP - 20

BT - 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST)

PB - IEEE

CY - Piscataway

T2 - SBST '19: 12th International Workshop on Search-Based Software Testing

Y2 - 27 May 2019 through 27 May 2019

ER -