A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective

Carmine Vassallo, Gerald Schermann, Fiorella Zampetti, Daniele Romano, Philipp Leitner, Andy Zaidman, Massimiliano Di Penta, Sebastiano Panichella

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

43 Citations (Scopus)
179 Downloads (Pure)


Continuous Integration (CI) and Continuous Delivery (CD) are widespread in both industrial and open-source software (OSS) projects. Recent research characterized build failures in CI and identified factors potentially correlated to them. However, most observations and findings of previous work are exclusively based on OSS projects or data from a single industrial organization. This paper provides a first attempt to compare the CI processes and occurrences of build failures in 349 Java OSS projects and 418 projects from a financial organization, ING Nederland. Through the analysis of 34,182 failing builds (26% of the total number of observed builds), we derived a taxonomy of failures that affect the observed CI processes. Using cluster analysis, we observed that in some cases OSS and ING projects share similar build failure patterns (e.g., few compilation failures as compared to frequent testing failures), while in other cases completely different patterns emerge. In short, we explain how OSS and ING CI processes exhibit commonalities, yet are substantially different in their design and in the failures they report.
Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Conference on Software Maintenance and Evolution, ICSME 2017
Place of PublicationLos Alamitos, CA
Number of pages11
ISBN (Electronic)978-1-5386-0992-7
Publication statusPublished - 2017
EventICSME 2017: 33rd International Conference on Software Maintenance and Evolution - Shanghai, China
Duration: 17 Sep 201724 Sep 2017
Conference number: 33


ConferenceICSME 2017
Abbreviated titleICSME
Internet address

Bibliographical note

Accepted Author Manuscript


  • Continuous Delivery
  • Continuous Integration
  • Agile development
  • Build failures


Dive into the research topics of 'A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective'. Together they form a unique fingerprint.

Cite this