Good Things Come In Threes: Improving Search-based Crash Reproduction With Helper Objectives

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

50 Downloads (Pure)

Abstract

Writing a test case reproducing a reported software crash is a common practice to identify the root cause of an anomaly in the software under test. However, this task is usually labor-intensive and time-taking. Hence, evolutionary intelligence approaches have been successfully applied to assist developers during debugging by generating a test case reproducing reported crashes. These approaches use a single fitness function called Crash Distance to guide the search process toward reproducing a target crash. Despite the reported achievements, these approaches do not always successfully reproduce some crashes due to a lack of test diversity (premature convergence). In this study, we introduce a new approach, called MO-HO, that addresses this issue via multi-objectivization. In particular, we introduce two new Helper-Objectives for crash reproduction, namely test length (to minimize) and method sequence diversity (to maximize), in addition to Crash Distance. We assessed MOHO using five multi-objective evolutionary algorithms (NSGA-II, SPEA2, PESA-II, MOEA/D, FEMO) on 124 non-trivial crashes stemming from open-source projects. Our results indicate that SPEA2 is the best-performing multi-objective algorithm for MO-HO. We evaluated this best-performing algorithm for MO-HO against the state-of-the-art: single-objective approach (Single-Objective Search) and decomposition-based multi-objectivization approach (De-MO). Our results show that MO-HO reproduces five crashes that cannot be reproduced by the current state-of-the-art. Besides, MO-HO improves the effectiveness (+10% and +8% in reproduction ratio) and the efficiency in 34.6% and 36% of crashes (i.e., significantly lower running time) compared to Single-Objective Search and De-MO, respectively. For some crashes, the improvements are very large, being up to +93.3% for reproduction ratio and -92% for the required running time.
Original languageEnglish
Title of host publicationASE '20
Subtitle of host publicationProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering
EditorsJohn Grundy, David Lo, Claire Le Goues
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Pages211-223
Number of pages13
ISBN (Electronic)978-1-7281-7280-4
DOIs
Publication statusPublished - 2020
Event35th IEEE/ACM
International Conference on Automated Software Engineering (ASE ’20),
-
Duration: 21 Sep 202025 Sep 2020
Conference number: 35

Conference

Conference35th IEEE/ACM
International Conference on Automated Software Engineering (ASE ’20),
Abbreviated titleASE ’20
Period21/09/2025/09/20
OtherVirtual/online event due to COVID-19

Keywords

  • crash reproduction
  • search-based software testing
  • multi-objective evolutionary algorithms

Fingerprint Dive into the research topics of 'Good Things Come In Threes: Improving Search-based Crash Reproduction With Helper Objectives'. Together they form a unique fingerprint.

Cite this