Improving Confidence in the Estimation of Values and Norms

L. Cavalcante Siebert; R.A. Mercuur; M.V. Dignum; M.J. van den Hoven; C.M. Jonker

doi:10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms

L. Cavalcante Siebert, R.A. Mercuur, M.V. Dignum, M.J. van den Hoven, C.M. Jonker

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

23 Downloads (Pure)

Abstract

Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

Original language	English
Title of host publication	Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers
Subtitle of host publication	International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers
Editors	Andrea Aler Tubella, Stephen Cranefield, Christopher Frantz, Felipe Meneguzzi, Wamberto Vasconcelos
Publisher	Cornell University Library - arXiv.org
Pages	98-113
Number of pages	16
ISBN (Electronic)	978-3-030-72376-7
ISBN (Print)	9783030723750
DOIs	https://doi.org/10.1007/978-3-030-72376-7_6
Publication status	Published - 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12298 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Autonomous agents
Norms
Ultimatum game
Values

Access to Document

10.1007/978-3-030-72376-7_6

Siebert2021_Chapter_ImprovingConfidenceInTheEstimaFinal published version, 710 KB

Cite this

Cavalcante Siebert, L., Mercuur, R. A., Dignum, M. V., van den Hoven, M. J., & Jonker, C. M. (2020). Improving Confidence in the Estimation of Values and Norms. In A. Aler Tubella, S. Cranefield, C. Frantz, F. Meneguzzi, & W. Vasconcelos (Eds.), Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers (pp. 98-113). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12298 LNAI). Cornell University Library - arXiv.org. https://doi.org/10.1007/978-3-030-72376-7_6

Cavalcante Siebert, L. ; Mercuur, R.A. ; Dignum, M.V. et al. / Improving Confidence in the Estimation of Values and Norms. Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers. editor / Andrea Aler Tubella ; Stephen Cranefield ; Christopher Frantz ; Felipe Meneguzzi ; Wamberto Vasconcelos. Cornell University Library - arXiv.org, 2020. pp. 98-113 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b1f3f93cff854382b90c89ca8a3eeb72,

title = "Improving Confidence in the Estimation of Values and Norms",

abstract = "Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.",

keywords = "Autonomous agents, Norms, Ultimatum game, Values",

author = "{Cavalcante Siebert}, L. and R.A. Mercuur and M.V. Dignum and {van den Hoven}, M.J. and C.M. Jonker",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ",

year = "2020",

doi = "10.1007/978-3-030-72376-7_6",

language = "English",

isbn = "9783030723750",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Cornell University Library - arXiv.org",

pages = "98--113",

editor = "{Aler Tubella}, Andrea and Stephen Cranefield and Christopher Frantz and Felipe Meneguzzi and Wamberto Vasconcelos",

booktitle = "Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers",

}

Cavalcante Siebert, L, Mercuur, RA, Dignum, MV , van den Hoven, MJ & Jonker, CM 2020, Improving Confidence in the Estimation of Values and Norms. in A Aler Tubella, S Cranefield, C Frantz, F Meneguzzi & W Vasconcelos (eds), Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12298 LNAI, Cornell University Library - arXiv.org, pp. 98-113. https://doi.org/10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms. / Cavalcante Siebert, L.; Mercuur, R.A.; Dignum, M.V. et al.
Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers. ed. / Andrea Aler Tubella; Stephen Cranefield; Christopher Frantz; Felipe Meneguzzi; Wamberto Vasconcelos. Cornell University Library - arXiv.org, 2020. p. 98-113 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12298 LNAI).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Improving Confidence in the Estimation of Values and Norms

AU - Cavalcante Siebert, L.

AU - Mercuur, R.A.

AU - Dignum, M.V.

AU - van den Hoven, M.J.

AU - Jonker, C.M.

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2020

Y1 - 2020

N2 - Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

AB - Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

KW - Autonomous agents

KW - Norms

KW - Ultimatum game

KW - Values

UR - http://www.scopus.com/inward/record.url?scp=85107492756&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-72376-7_6

DO - 10.1007/978-3-030-72376-7_6

M3 - Conference contribution

SN - 9783030723750

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 98

EP - 113

BT - Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers

A2 - Aler Tubella, Andrea

A2 - Cranefield, Stephen

A2 - Frantz, Christopher

A2 - Meneguzzi, Felipe

A2 - Vasconcelos, Wamberto

PB - Cornell University Library - arXiv.org

ER -

Cavalcante Siebert L, Mercuur RA, Dignum MV , van den Hoven MJ , Jonker CM. Improving Confidence in the Estimation of Values and Norms. In Aler Tubella A, Cranefield S, Frantz C, Meneguzzi F, Vasconcelos W, editors, Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers. Cornell University Library - arXiv.org. 2020. p. 98-113. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms

Abstract

Publication series

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this