TY - GEN
T1 - Data-oriented differential testing of object-relational mapping systems
AU - Sotiropoulos, Thodoris
AU - Chaliasos, Stefanos
AU - Atlidakis, Vaggelis
AU - Mitropoulos, Dimitris
AU - Spinellis, Diomidis
PY - 2021
Y1 - 2021
N2 - We introduce, what is to the best of our knowledge, the first approach for systematically testing Object-Relational Mapping (ORM) systems. Our approach leverages differential testing to establish a test oracle for ORM-specific bugs. Specifically, we first generate random relational database schemas, set up the respective databases, and then, we query these databases using the APIs of the ORM systems under test. To tackle the challenge that ORMs lack a common input language, we generate queries written in an abstract query language. These abstract queries are translated into concrete, executable ORM queries, which are ultimately used to differentially test the correctness of target implementations. The effectiveness of our method heavily relies on the data inserted to the underlying databases. Therefore, we employ a solver-based approach for producing targeted database records with respect to the constraints of the generated queries. We implement our approach as a tool, called CYNTHIA, which found 28 bugs in five popular ORM systems. The vast majority of these bugs are confirmed (25 / 28), more than half were fixed (20 / 28), and three were marked as release blockers by the corresponding developers.
AB - We introduce, what is to the best of our knowledge, the first approach for systematically testing Object-Relational Mapping (ORM) systems. Our approach leverages differential testing to establish a test oracle for ORM-specific bugs. Specifically, we first generate random relational database schemas, set up the respective databases, and then, we query these databases using the APIs of the ORM systems under test. To tackle the challenge that ORMs lack a common input language, we generate queries written in an abstract query language. These abstract queries are translated into concrete, executable ORM queries, which are ultimately used to differentially test the correctness of target implementations. The effectiveness of our method heavily relies on the data inserted to the underlying databases. Therefore, we employ a solver-based approach for producing targeted database records with respect to the constraints of the generated queries. We implement our approach as a tool, called CYNTHIA, which found 28 bugs in five popular ORM systems. The vast majority of these bugs are confirmed (25 / 28), more than half were fixed (20 / 28), and three were marked as release blockers by the corresponding developers.
KW - Automated Testing
KW - Differential Testing
KW - Object-Relational Mapping
UR - http://www.scopus.com/inward/record.url?scp=85115693867&partnerID=8YFLogxK
U2 - 10.1109/ICSE43902.2021.00137
DO - 10.1109/ICSE43902.2021.00137
M3 - Conference contribution
AN - SCOPUS:85115693867
T3 - Proceedings - International Conference on Software Engineering
SP - 1535
EP - 1547
BT - Proceedings - 2021 IEEE/ACM 43rd International Conference on Software Engineering, ICSE 2021
PB - IEEE
T2 - 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021
Y2 - 22 May 2021 through 30 May 2021
ER -