Abstract
Test smells are sub-optimal design choices in the implementation of test code. As reported by recent studies, their presence might not only negatively affect the comprehension of test suites but can also lead to test cases being less effective in finding bugs in production code. Although significant steps toward understanding test smells, there is still a notable absence of studies assessing their association with software quality.
In this paper, we investigate the relationship between the presence of test smells and the change- and defect-proneness of test code, as well as the defect-proneness of the tested production code. To this aim, we collect data on 221 releases of ten software systems and we analyze more than a million test cases to investigate the association of six test smells and their co-occurrence with software quality. Key results of our study include:(i) tests with smells are more change- and defect-prone, (ii) ‘Indirect Testing’, ‘Eager Test’, and ‘Assertion Roulette’ are the most significant smells for change-proneness and, (iii) production code is more defect-prone when tested by smelly tests.
In this paper, we investigate the relationship between the presence of test smells and the change- and defect-proneness of test code, as well as the defect-proneness of the tested production code. To this aim, we collect data on 221 releases of ten software systems and we analyze more than a million test cases to investigate the association of six test smells and their co-occurrence with software quality. Key results of our study include:(i) tests with smells are more change- and defect-prone, (ii) ‘Indirect Testing’, ‘Eager Test’, and ‘Assertion Roulette’ are the most significant smells for change-proneness and, (iii) production code is more defect-prone when tested by smelly tests.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2018 IEEE International Conference on Software Maintenance and Evolution, ICSME 2018 |
| Place of Publication | Los Alamitos, CA |
| Publisher | IEEE |
| Pages | 12-23 |
| Number of pages | 12 |
| ISBN (Electronic) | 978-1-5386-7870-1 |
| DOIs | |
| Publication status | Published - 2018 |
| Event | ICSME 2018: 34th IEEE International Conference on Software Maintenance and Evolution - Madrid, Spain Duration: 23 Sept 2018 → 29 Sept 2018 Conference number: 34 https://icsme2018.github.io/ |
Conference
| Conference | ICSME 2018 |
|---|---|
| Country/Territory | Spain |
| City | Madrid |
| Period | 23/09/18 → 29/09/18 |
| Internet address |
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-careOtherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Keywords
- test smells
- code quality
- automated testing