An underpinning theory and approach to applicability testing of constructive computational mechanisms

Yongzhe Li*, Imre Horváth, Zoltán Rusák

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

6 Downloads (Pure)

Abstract

Applicability testing of constructive computational mechanisms (CCMs) is a new challenge for both the academia and the industry. The overwhelming majority of the existing validation approaches focuses on the internal validity of CCMs (e.g. consistency, bias), while there is a shortage of efficient approaches for assessing the external validity (e.g. applicability, reusability). The objective of this paper is to clarify the concepts and criteria, and to develop an approach for a systematic evaluation of the applicability of a given CCM to cases that were not considered at design time. The approach is adapted from the validation square approach (VSA). The adapted methodology (A-VSA) makes it possible to evaluate CCMs from (a) theoretical structural, (b) empirical structural, (c) theoretical performance, and (d) empirical performance dimensions. Altogether eight indicators are introduced that support the evaluation process. The effectiveness of the A-VSA was confirmed through a case study, in which a specific CCM is considered and the strategy of the A-VSA was operationalized with three completely different application cases. As evidenced by the results, the proposed A-VSA establishes a tight coupling among the enablers embraced by a CCM and the aspects of theoretical and empirical validation, which approves the approach to be an efficient tool for defining the range and/or the extent of applicability. The advantage of the A-VSA is that it offers a way to transfer qualitative applicability evaluation into quantitative applicability assessment, which allows the use of both subjective statements and mathematical modeling in applicability testing. The results of the assessment can guide the adaptation work of a CCM when applied to an out-of-domain application.

Original languageEnglish
Pages (from-to)213-230
Number of pages18
JournalResearch in Engineering Design
Volume33
Issue number2
DOIs
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Constructive computational mechanism
  • Practical appropriateness measures
  • Structural appropriateness indicators
  • Systematic applicability validation
  • Theoretical utility targets

Fingerprint

Dive into the research topics of 'An underpinning theory and approach to applicability testing of constructive computational mechanisms'. Together they form a unique fingerprint.

Cite this