Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework

Guangliang Li, Shimon Whiteson, Hamdi Dibeklioğlu, H.S. Hung

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

223 Downloads (Pure)

Abstract

Interactive reinforcement learning provides a way for agents to learn to solve tasks from evaluative feedback provided by a human user. Previous research showed that humans give copious feedback early in training but very sparsely thereafter. In this paper, we investigate the potential of agent learning from trainers’ facial expressions via interpreting them as evaluative feedback. To do so, we implemented TAMER which is a popular interactive reinforcement learning method in a reinforcement-learning benchmark problem — Infinite Mario, and conducted the first large-scale study of TAMER involving 561 participants. With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback using facial expressions. In addition, our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible and using strong/effective prediction models or a regression method, facial responses would significantly improve the performance of agents. Furthermore, our experiment supports previous studies demonstrating the importance of bi-directional feedback and competitive elements in the training interface.
Original languageEnglish
Title of host publicationProceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems
Place of PublicationRichland, SC
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages1735-1737
Number of pages3
ISBN (Electronic)9781450383073
Publication statusPublished - 2021
Event20th International Conference on Autonomous Agentsand Multiagent Systems - Virtual/online event due to COVID-19
Duration: 3 May 20217 May 2021
Conference number: 20

Publication series

NameAAMAS '21
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems
ISSN (Electronic)2523-5699

Conference

Conference20th International Conference on Autonomous Agentsand Multiagent Systems
Abbreviated titleAAMAS 2021
Period3/05/217/05/21

Keywords

  • Facial Feedback
  • Implicit Feedback
  • Interactive
  • Reinforcement Learning

Fingerprint

Dive into the research topics of 'Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework'. Together they form a unique fingerprint.

Cite this