Abstract
In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 80 researchers worldwide, researching the IVA community interests and practises in evaluating human interaction with an artificial social agent (ASA). The effort is driven by previous IVA workshops and plenary IVA discussions related to the methodological crisis on the evaluation of ASAs. A previous literature review showed a continuous practise of creating new questionnaires instead of reusing validated questionnaires. We address this issue by examining questionnaire measurement constructs used in empirical studies between 2013 to 2018 published in the IVA conference. We identified 189 constructs used in 89 questionnaires that are reported across 81 studies. Although these constructs have different names, they often measure the same thing. In this paper, we, therefore, present a unifying set of 19 constructs that captures more than 80% of the 189 constructs initially identified. We established this set in two steps. First, 49 researchers classified the constructs in broad theoretically based categories. Next, 23 researchers grouped the constructs in each category on their similarity. The resulting 19 groups form a unifying set of constructs, which will be the basis for the future questionnaire instrument of human-ASA interaction.
Original language | English |
---|---|
Title of host publication | Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 |
Publisher | Association for Computing Machinery (ACM) |
Number of pages | 8 |
ISBN (Electronic) | 9781450375863 |
DOIs | |
Publication status | Published - 2020 |
Event | 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 - Virtual, Online, United Kingdom Duration: 20 Oct 2020 → 22 Oct 2020 |
Publication series
Name | Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 |
---|
Conference
Conference | 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 |
---|---|
Country/Territory | United Kingdom |
City | Virtual, Online |
Period | 20/10/20 → 22/10/20 |
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-careOtherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Keywords
- Artificial social agent
- evaluation instrument
- measurement construct
- questionnaire
- user study