Quality Gatekeepers: Investigating the Effects of Code Review Bots on Pull Request Activities

Mairieli Wessel; Alexander Serebrenik; Igor Scaliante Wiese; Igor Steinmacher; Marco Aurélio Gerosa

doi:10.1007/s10664-022-10130-9

Quality Gatekeepers: Investigating the Effects of Code Review Bots on Pull Request Activities

Mairieli Wessel, Alexander Serebrenik, Igor Scaliante Wiese, Igor Steinmacher, Marco Aurélio Gerosa

Software Engineering

Research output: Contribution to journal › Article › Scientific › peer-review

6 Citations (Scopus)

22 Downloads (Pure)

Abstract

Software bots have been facilitating several development activities
in Open Source Software (OSS) projects, including code review. However,
these bots may bring unexpected impacts to group dynamics, as frequently
occurs with new technology adoption. Understanding and anticipating such
effects is important for planning and management. To analyze these effects, we
investigate how several activity indicators change after the adoption of a code
review bot. We employed a regression discontinuity design on 1,194 software
projects from GitHub. We also interviewed 12 practitioners, including open-
source maintainers and contributors. Our results indicate that the adoption
of code review bots increases the number of monthly merged pull requests,
decreases monthly non-merged pull requests, and decreases communication
among developers. From the developers’ perspective, these effects are explained
by the transparency and confidence the bot comments introduce, in addition
to the changes in the discussion focused on pull requests. Practitioners and
maintainers may leverage our results to understand, or even predict, bot effects
on their projects.

Original language	English
Article number	108
Number of pages	36
Journal	Empirical Software Engineering
Volume	27
Issue number	5
DOIs	https://doi.org/10.1007/s10664-022-10130-9
Publication status	Published - 2022

Access to Document

10.1007/s10664-022-10130-9Licence: CC BY

Wessel2022_Article_QualityGatekeepersInvestigatinFinal published version, 2.39 MBLicence: CC BY

Cite this

@article{9cfdac6ae1bb43368abd653e34a607f9,

title = "Quality Gatekeepers: Investigating the Effects of Code Review Bots on Pull Request Activities",

abstract = "Software bots have been facilitating several development activitiesin Open Source Software (OSS) projects, including code review. However,these bots may bring unexpected impacts to group dynamics, as frequentlyoccurs with new technology adoption. Understanding and anticipating sucheffects is important for planning and management. To analyze these effects, weinvestigate how several activity indicators change after the adoption of a codereview bot. We employed a regression discontinuity design on 1,194 softwareprojects from GitHub. We also interviewed 12 practitioners, including open-source maintainers and contributors. Our results indicate that the adoptionof code review bots increases the number of monthly merged pull requests,decreases monthly non-merged pull requests, and decreases communicationamong developers. From the developers{\textquoteright} perspective, these effects are explainedby the transparency and confidence the bot comments introduce, in additionto the changes in the discussion focused on pull requests. Practitioners andmaintainers may leverage our results to understand, or even predict, bot effectson their projects.",

author = "Mairieli Wessel and Alexander Serebrenik and Wiese, {Igor Scaliante} and Igor Steinmacher and Gerosa, {Marco Aur{\'e}lio}",

year = "2022",

doi = "10.1007/s10664-022-10130-9",

language = "English",

volume = "27",

journal = "Empirical Software Engineering",

issn = "1382-3256",

publisher = "Springer",

number = "5",

}

TY - JOUR

T1 - Quality Gatekeepers

T2 - Investigating the Effects of Code Review Bots on Pull Request Activities

AU - Wessel, Mairieli

AU - Serebrenik, Alexander

AU - Wiese, Igor Scaliante

AU - Steinmacher, Igor

AU - Gerosa, Marco Aurélio

PY - 2022

Y1 - 2022

N2 - Software bots have been facilitating several development activitiesin Open Source Software (OSS) projects, including code review. However,these bots may bring unexpected impacts to group dynamics, as frequentlyoccurs with new technology adoption. Understanding and anticipating sucheffects is important for planning and management. To analyze these effects, weinvestigate how several activity indicators change after the adoption of a codereview bot. We employed a regression discontinuity design on 1,194 softwareprojects from GitHub. We also interviewed 12 practitioners, including open-source maintainers and contributors. Our results indicate that the adoptionof code review bots increases the number of monthly merged pull requests,decreases monthly non-merged pull requests, and decreases communicationamong developers. From the developers’ perspective, these effects are explainedby the transparency and confidence the bot comments introduce, in additionto the changes in the discussion focused on pull requests. Practitioners andmaintainers may leverage our results to understand, or even predict, bot effectson their projects.

AB - Software bots have been facilitating several development activitiesin Open Source Software (OSS) projects, including code review. However,these bots may bring unexpected impacts to group dynamics, as frequentlyoccurs with new technology adoption. Understanding and anticipating sucheffects is important for planning and management. To analyze these effects, weinvestigate how several activity indicators change after the adoption of a codereview bot. We employed a regression discontinuity design on 1,194 softwareprojects from GitHub. We also interviewed 12 practitioners, including open-source maintainers and contributors. Our results indicate that the adoptionof code review bots increases the number of monthly merged pull requests,decreases monthly non-merged pull requests, and decreases communicationamong developers. From the developers’ perspective, these effects are explainedby the transparency and confidence the bot comments introduce, in additionto the changes in the discussion focused on pull requests. Practitioners andmaintainers may leverage our results to understand, or even predict, bot effectson their projects.

UR - http://www.scopus.com/inward/record.url?scp=85130914224&partnerID=8YFLogxK

U2 - 10.1007/s10664-022-10130-9

DO - 10.1007/s10664-022-10130-9

M3 - Article

SN - 1382-3256

VL - 27

JO - Empirical Software Engineering

JF - Empirical Software Engineering

IS - 5

M1 - 108

ER -

Quality Gatekeepers: Investigating the Effects of Code Review Bots on Pull Request Activities

Abstract

Access to Document

Other files and links

Fingerprint

Cite this