A security perspective on code review: The case of Chromium

Marco di Biase; Magiel Bruntink; Alberto Bacchelli

doi:10.1109/SCAM.2016.30

A security perspective on code review: The case of Chromium

Marco di Biase, Magiel Bruntink, Alberto Bacchelli

Software Engineering

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

25 Citations (Scopus)

102 Downloads (Pure)

Abstract

Modern Code Review (MCR) is an established software development process that aims to improve software quality. Although evidence showed that higher levels of review coverage relates to less post-release bugs, it remains unknown the effectiveness of MCR at specifically finding security issues. We present a work we conduct aiming to fill that gap by exploring the MCR process in the Chromium open source project. We manually analyzed large sets of registered (114 cases) and missed (71 cases) security issues by backtracking in the project’s issue, review, and code histories. This enabled us to qualify MCR in Chromium from the security perspective from several angles: Are security issues being discussed frequently? What categories of security issues are often missed or found? What characteristics of code reviews appear relevant to the discovery rate?
Within the cases we analyzed, MCR in Chromium addresses security issues at a rate of 1% of reviewers’ comments. Chromium code reviews mostly tend to miss language-specific issues (e.g., C++ issues and buffer overflows) and domain-specific ones (e.g., such as Cross-Site Scripting); when code reviews address issues, mostly they address those that pertain to the latter type. Initial evidence points to reviews conducted by more than 2 reviewers being more successful at finding security issues.

Original language	English
Title of host publication	2016 IEEE 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SACM)
Editors	L. O' Conner
Place of Publication	Los Alamitos
Publisher	IEEE
Pages	21-30
Number of pages	10
ISBN (Print)	978-1-5090-3848-0
DOIs	https://doi.org/10.1109/SCAM.2016.30
Publication status	Published - 2016
Event	2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (Scam 2016) - Raleigh, NC, United States Duration: 2 Oct 2016 → 3 Oct 2016 http://www.ieee-scam.org/2016/

Conference

Conference	2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (Scam 2016)
Abbreviated title	SCAM 2016
Country/Territory	United States
City	Raleigh, NC
Period	2/10/16 → 3/10/16
Internet address	http://www.ieee-scam.org/2016/

Bibliographical note

Acknowledgments: European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 642954

Keywords

Code review
Empirical software engineering
Mining software repositories
Modern code review
Security flaw
Software security

Access to Document

10.1109/SCAM.2016.30

TUD-SERG-2016-019Accepted author manuscript, 892 KB

Cite this

@inproceedings{971dc100f7f94f34abefd3ed7f02b57d,

title = "A security perspective on code review: The case of Chromium",

abstract = "Modern Code Review (MCR) is an established software development process that aims to improve software quality. Although evidence showed that higher levels of review coverage relates to less post-release bugs, it remains unknown the effectiveness of MCR at specifically finding security issues. We present a work we conduct aiming to fill that gap by exploring the MCR process in the Chromium open source project. We manually analyzed large sets of registered (114 cases) and missed (71 cases) security issues by backtracking in the project{\textquoteright}s issue, review, and code histories. This enabled us to qualify MCR in Chromium from the security perspective from several angles: Are security issues being discussed frequently? What categories of security issues are often missed or found? What characteristics of code reviews appear relevant to the discovery rate?Within the cases we analyzed, MCR in Chromium addresses security issues at a rate of 1% of reviewers{\textquoteright} comments. Chromium code reviews mostly tend to miss language-specific issues (e.g., C++ issues and buffer overflows) and domain-specific ones (e.g., such as Cross-Site Scripting); when code reviews address issues, mostly they address those that pertain to the latter type. Initial evidence points to reviews conducted by more than 2 reviewers being more successful at finding security issues.",

keywords = "Code review, Empirical software engineering, Mining software repositories, Modern code review, Security flaw, Software security",

author = "{di Biase}, Marco and Magiel Bruntink and Alberto Bacchelli",

note = "Acknowledgments: European Union{\textquoteright}s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 642954; 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (Scam 2016), SCAM 2016 ; Conference date: 02-10-2016 Through 03-10-2016",

year = "2016",

doi = "10.1109/SCAM.2016.30",

language = "English",

isbn = "978-1-5090-3848-0",

pages = "21--30",

editor = "{O' Conner}, L.",

booktitle = "2016 IEEE 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SACM)",

publisher = "IEEE",

address = "United States",

url = "http://www.ieee-scam.org/2016/",

}

di Biase, M, Bruntink, M & Bacchelli, A 2016, A security perspective on code review: The case of Chromium. in L O' Conner (ed.), 2016 IEEE 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SACM). IEEE, Los Alamitos, pp. 21-30, 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (Scam 2016), Raleigh, NC, United States, 2/10/16. https://doi.org/10.1109/SCAM.2016.30

A security perspective on code review: The case of Chromium. / di Biase, Marco; Bruntink, Magiel; Bacchelli, Alberto.
2016 IEEE 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SACM). ed. / L. O' Conner. Los Alamitos: IEEE, 2016. p. 21-30.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - A security perspective on code review

T2 - 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (Scam 2016)

AU - di Biase, Marco

AU - Bruntink, Magiel

AU - Bacchelli, Alberto

N1 - Acknowledgments: European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 642954

PY - 2016

Y1 - 2016

N2 - Modern Code Review (MCR) is an established software development process that aims to improve software quality. Although evidence showed that higher levels of review coverage relates to less post-release bugs, it remains unknown the effectiveness of MCR at specifically finding security issues. We present a work we conduct aiming to fill that gap by exploring the MCR process in the Chromium open source project. We manually analyzed large sets of registered (114 cases) and missed (71 cases) security issues by backtracking in the project’s issue, review, and code histories. This enabled us to qualify MCR in Chromium from the security perspective from several angles: Are security issues being discussed frequently? What categories of security issues are often missed or found? What characteristics of code reviews appear relevant to the discovery rate?Within the cases we analyzed, MCR in Chromium addresses security issues at a rate of 1% of reviewers’ comments. Chromium code reviews mostly tend to miss language-specific issues (e.g., C++ issues and buffer overflows) and domain-specific ones (e.g., such as Cross-Site Scripting); when code reviews address issues, mostly they address those that pertain to the latter type. Initial evidence points to reviews conducted by more than 2 reviewers being more successful at finding security issues.

AB - Modern Code Review (MCR) is an established software development process that aims to improve software quality. Although evidence showed that higher levels of review coverage relates to less post-release bugs, it remains unknown the effectiveness of MCR at specifically finding security issues. We present a work we conduct aiming to fill that gap by exploring the MCR process in the Chromium open source project. We manually analyzed large sets of registered (114 cases) and missed (71 cases) security issues by backtracking in the project’s issue, review, and code histories. This enabled us to qualify MCR in Chromium from the security perspective from several angles: Are security issues being discussed frequently? What categories of security issues are often missed or found? What characteristics of code reviews appear relevant to the discovery rate?Within the cases we analyzed, MCR in Chromium addresses security issues at a rate of 1% of reviewers’ comments. Chromium code reviews mostly tend to miss language-specific issues (e.g., C++ issues and buffer overflows) and domain-specific ones (e.g., such as Cross-Site Scripting); when code reviews address issues, mostly they address those that pertain to the latter type. Initial evidence points to reviews conducted by more than 2 reviewers being more successful at finding security issues.

KW - Code review

KW - Empirical software engineering

KW - Mining software repositories

KW - Modern code review

KW - Security flaw

KW - Software security

U2 - 10.1109/SCAM.2016.30

DO - 10.1109/SCAM.2016.30

M3 - Conference contribution

SN - 978-1-5090-3848-0

SP - 21

EP - 30

BT - 2016 IEEE 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SACM)

A2 - O' Conner, L.

PB - IEEE

CY - Los Alamitos

Y2 - 2 October 2016 through 3 October 2016

ER -

A security perspective on code review: The case of Chromium

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this