Scalable Incremental Building with Dynamic Task Dependencies

Gabriël Konat, Sebastian Erdweg, Eelco Visser

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

9 Citations (Scopus)
265 Downloads (Pure)


Incremental build systems are essential for fast, reproducible software builds. Incremental build systems enable short feedback cycles when they capture dependencies precisely and selectively execute build tasks efficiently. A much overlooked feature of build systems is the expressiveness of the scripting language, which directly influences the maintainability of build scripts. In this paper, we present a new incremental build algorithm that allows build engineers to use a full-fledged programming language with explicit task invocation, value and file inspection facilities, and conditional and iterative language constructs. In contrast to prior work on incrementality for such programmable builds, our algorithm scales with the number of tasks affected by a change and is independent of the size of the software project being built. Specifically, our algorithm accepts a set of changed files, transitively detects and re-executes affected build tasks, but also accounts for new task dependencies discovered during building. We have evaluated the performance of our algorithm in a real-world case study and confirm its scalability.
Original languageEnglish
Title of host publicationASE 2018
Subtitle of host publicationProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery (ACM)
Number of pages11
ISBN (Print)978-1-4503-5937-5
Publication statusPublished - 2018
EventASE 2018: 33rd IEEE/ACM International Conference on Automated Software Engineering - Montpellier, France
Duration: 3 Jul 20187 Jul 2018


ConferenceASE 2018
Abbreviated titleASE 2018
Internet address


  • scalable
  • incremental
  • build
  • dynamic task dependency


Dive into the research topics of 'Scalable Incremental Building with Dynamic Task Dependencies'. Together they form a unique fingerprint.

Cite this