What are we measuring anyway? A literature survey of questionnaires used in studies reported in the intelligent virtual agent conferences

Merijn Bruijnes, Siska Fitrianie, Deborah Richards, Amal Abdulrahman, Willem-Paul Brinkman

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

14 Downloads (Pure)

Abstract

Research into artificial social agents aims at constructing these agents and at establishing an empirically grounded understanding of them, their interaction with humans, and how they can ultimately deliver certain outcomes in areas such as health, entertainment, and education. Key for establishing such understanding is the community’s ability to describe and replicate their observations on how users perceive and interact with their agents. In this paper, we address this ability by examining questionnaires and their constructs used in empirical studies reported in the intelligent virtual agent conference proceedings from 2013 to 2018. The literature survey shows the identification of 189 constructs used in 89 questionnaires that were reported across 81 papers. We found unexpectedly little repeated use of questionnaires as the vast majority of questionnaires (more than 76%) were only reported in a single paper. We expect that this finding will motivate joint effort by the IVA community towards creating a unified measurement instrument and in the broader AI community a renewed interest in replicability of our (user) studies.

Original languageEnglish
Title of host publicationBNAIC/BENELEARN 2019
Subtitle of host publicationProceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC 2019) and the 28th Belgian Dutch Conference on Machine Learning (Benelearn 2019)
EditorsKatrien Beuls , Bart Bogaerts , Gianluca Bontempi, Pierre Geurts, Nick Harley, Bertrand Lebichot , Tom Lenaerts, Gilles Louppe, Paul Van Eecke
PublisherCEUR-WS.org
Pages1-2
Number of pages2
Publication statusPublished - 2019
Event31st Benelux Conference on Artificial Intelligence and the 28th Belgian Dutch Conference on Machine Learning, BNAIC/BENELEARN 2019 - Brussels, Belgium
Duration: 6 Nov 20198 Nov 2019

Publication series

NameCEUR Workshop Proceedings
Volume2491
ISSN (Print)1613-0073

Conference

Conference31st Benelux Conference on Artificial Intelligence and the 28th Belgian Dutch Conference on Machine Learning, BNAIC/BENELEARN 2019
CountryBelgium
CityBrussels
Period6/11/198/11/19

Fingerprint Dive into the research topics of 'What are we measuring anyway? A literature survey of questionnaires used in studies reported in the intelligent virtual agent conferences'. Together they form a unique fingerprint.

Cite this