LogChunks: A Data Set for Build Log Analysis

Carolin E. Brandt; Annibale Panichella; Andy Zaidman; Moritz Beller

doi:10.1145/3379597.3387485

LogChunks: A Data Set for Build Log Analysis

Carolin E. Brandt, Annibale Panichella, Andy Zaidman, Moritz Beller

Software Engineering

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

3 Citations (Scopus)

733 Downloads (Pure)

Abstract

Build logs are textual by-products that a software build process creates, often as part of its Continuous Integration (CI) pipeline. Build logs are a paramount source of information for developers when debugging into and understanding a build failure. Recently, attempts to partly automate this time-consuming, purely manual activity have come up, such as rule- or information-retrieval-based techniques. We believe that having a common data set to compare different build log analysis techniques will advance the research area. It will ultimately increase our understanding of CI build failures. In this paper, we present logchunks, a collection of 797 annotated Travis CI build logs from 80 GitHub repositories in 29 programming languages. For each build log, logchunks contains a manually labeled log part (chunk) describing why the build failed. We externally validated the data set with the developers who caused the original build failure. The width and depth of the logchunks data set are intended to make it the default benchmark for automated build log analysis techniques.

Original language	English
Title of host publication	Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020
Pages	583-587
Number of pages	5
ISBN (Electronic)	9781450379571
DOIs	https://doi.org/10.1145/3379597.3387485
Publication status	Published - 2020
Event	17th International Conference on Mining Software Repositories - Seoul, Korea, Republic of Duration: 5 Oct 2020 → 6 Oct 2020 Conference number: 17

Publication series

Name	Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020

Conference

Conference	17th International Conference on Mining Software Repositories
Abbreviated title	MSR 20
Country/Territory	Korea, Republic of
City	Seoul
Period	5/10/20 → 6/10/20
Other	Virtual/online event due to COVID-19

Bibliographical note

Virtual/online event due to COVID-19
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Continuous Integration
Build Log Analysis
Build Failure
Chunk Retrieval
CI

Access to Document

10.1145/3379597.3387485

3379597.3387485Final published version, 174 KB

Cite this

Brandt, C. E., Panichella, A., Zaidman, A., & Beller, M. (2020). LogChunks: A Data Set for Build Log Analysis. In Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020 (pp. 583-587). (Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020). https://doi.org/10.1145/3379597.3387485

@inproceedings{7f7f9d19339f4f59b6baf92bae1fa447,

title = "LogChunks: A Data Set for Build Log Analysis",

abstract = "Build logs are textual by-products that a software build process creates, often as part of its Continuous Integration (CI) pipeline. Build logs are a paramount source of information for developers when debugging into and understanding a build failure. Recently, attempts to partly automate this time-consuming, purely manual activity have come up, such as rule- or information-retrieval-based techniques. We believe that having a common data set to compare different build log analysis techniques will advance the research area. It will ultimately increase our understanding of CI build failures. In this paper, we present logchunks, a collection of 797 annotated Travis CI build logs from 80 GitHub repositories in 29 programming languages. For each build log, logchunks contains a manually labeled log part (chunk) describing why the build failed. We externally validated the data set with the developers who caused the original build failure. The width and depth of the logchunks data set are intended to make it the default benchmark for automated build log analysis techniques.",

keywords = "Continuous Integration, Build Log Analysis, Build Failure, Chunk Retrieval, CI",

author = "Brandt, {Carolin E.} and Annibale Panichella and Andy Zaidman and Moritz Beller",

note = "Virtual/online event due to COVID-19 Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ; 17th International Conference on Mining Software Repositories, MSR 20 ; Conference date: 05-10-2020 Through 06-10-2020",

year = "2020",

doi = "10.1145/3379597.3387485",

language = "English",

series = "Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020",

pages = "583--587",

booktitle = "Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020",

}

Brandt, CE , Panichella, A , Zaidman, A & Beller, M 2020, LogChunks: A Data Set for Build Log Analysis. in Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020. Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020, pp. 583-587, 17th International Conference on Mining Software Repositories, Seoul, Korea, Republic of, 5/10/20. https://doi.org/10.1145/3379597.3387485

LogChunks: A Data Set for Build Log Analysis. / Brandt, Carolin E.; Panichella, Annibale ; Zaidman, Andy et al.
Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020. 2020. p. 583-587 (Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - LogChunks: A Data Set for Build Log Analysis

AU - Brandt, Carolin E.

AU - Panichella, Annibale

AU - Zaidman, Andy

AU - Beller, Moritz

N1 - Conference code: 17

PY - 2020

Y1 - 2020

N2 - Build logs are textual by-products that a software build process creates, often as part of its Continuous Integration (CI) pipeline. Build logs are a paramount source of information for developers when debugging into and understanding a build failure. Recently, attempts to partly automate this time-consuming, purely manual activity have come up, such as rule- or information-retrieval-based techniques. We believe that having a common data set to compare different build log analysis techniques will advance the research area. It will ultimately increase our understanding of CI build failures. In this paper, we present logchunks, a collection of 797 annotated Travis CI build logs from 80 GitHub repositories in 29 programming languages. For each build log, logchunks contains a manually labeled log part (chunk) describing why the build failed. We externally validated the data set with the developers who caused the original build failure. The width and depth of the logchunks data set are intended to make it the default benchmark for automated build log analysis techniques.

AB - Build logs are textual by-products that a software build process creates, often as part of its Continuous Integration (CI) pipeline. Build logs are a paramount source of information for developers when debugging into and understanding a build failure. Recently, attempts to partly automate this time-consuming, purely manual activity have come up, such as rule- or information-retrieval-based techniques. We believe that having a common data set to compare different build log analysis techniques will advance the research area. It will ultimately increase our understanding of CI build failures. In this paper, we present logchunks, a collection of 797 annotated Travis CI build logs from 80 GitHub repositories in 29 programming languages. For each build log, logchunks contains a manually labeled log part (chunk) describing why the build failed. We externally validated the data set with the developers who caused the original build failure. The width and depth of the logchunks data set are intended to make it the default benchmark for automated build log analysis techniques.

KW - Continuous Integration

KW - Build Log Analysis

KW - Build Failure

KW - Chunk Retrieval

KW - CI

UR - http://www.scopus.com/inward/record.url?scp=85093664154&partnerID=8YFLogxK

U2 - 10.1145/3379597.3387485

DO - 10.1145/3379597.3387485

M3 - Conference contribution

T3 - Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020

SP - 583

EP - 587

BT - Proceedings - 2020 IEEE/ACM 17th International Conference on Mining Software Repositories, MSR 2020

T2 - 17th International Conference on Mining Software Repositories

Y2 - 5 October 2020 through 6 October 2020

ER -

LogChunks: A Data Set for Build Log Analysis

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this