TY - JOUR
T1 - How to Kill Them All
T2 - An Exploratory Study on the Impact of Code Observability on Mutation Testing
AU - Zhu, Qianqian
AU - Zaidman, Andy
AU - Panichella, Annibale
PY - 2021
Y1 - 2021
N2 - Mutation testing is well-known for its efficacy in assessing test quality, and starting to be applied in the industry. However, what should a developer do when confronted with a low mutation score? Should the test suite be plainly reinforced to increase the mutation score, or should the production code be improved as well, to make the creation of better tests possible? In this paper, we aim to provide a new perspective to developers that enables them to understand and reason about the mutation score in the light of testability and observability. First, we investigate whether testability and observability metrics are correlated with the mutation score on six open-source Java projects. We observe a correlation between observability metrics and the mutation score, e.g., test directness, which measures the extent to which the production code is tested directly, seems to be an essential factor. Based on our insights from the correlation study, we propose a number of ”mutation score anti-patterns”, enabling software engineers to refactor their existing code or add tests to improve the mutation score. In doing so, we observe that relatively simple refactoring operations enable an improvement or increase in the mutation score.
AB - Mutation testing is well-known for its efficacy in assessing test quality, and starting to be applied in the industry. However, what should a developer do when confronted with a low mutation score? Should the test suite be plainly reinforced to increase the mutation score, or should the production code be improved as well, to make the creation of better tests possible? In this paper, we aim to provide a new perspective to developers that enables them to understand and reason about the mutation score in the light of testability and observability. First, we investigate whether testability and observability metrics are correlated with the mutation score on six open-source Java projects. We observe a correlation between observability metrics and the mutation score, e.g., test directness, which measures the extent to which the production code is tested directly, seems to be an essential factor. Based on our insights from the correlation study, we propose a number of ”mutation score anti-patterns”, enabling software engineers to refactor their existing code or add tests to improve the mutation score. In doing so, we observe that relatively simple refactoring operations enable an improvement or increase in the mutation score.
KW - Code quality
KW - Code refactoring
KW - Mutation testing
KW - Observability
KW - Testability
UR - http://www.scopus.com/inward/record.url?scp=85097713377&partnerID=8YFLogxK
U2 - 10.1016/j.jss.2020.110864
DO - 10.1016/j.jss.2020.110864
M3 - Article
SN - 0164-1212
VL - 173
SP - 1
EP - 20
JO - Journal of Systems and Software
JF - Journal of Systems and Software
M1 - 110864
ER -