Telling autonomous systems what to do

Peter Werkhoven, Leon Kester, Mark Neerincx

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

7 Citations (Scopus)


Recent progress in Artificial Intelligence, sensing and network technology, robotics, and (cloud) computing has enabled the development of intelligent autonomous machine systems. Telling such autonomous systems "what to do" in a responsible way, is a non-trivial task. For intelligent autonomous machines to function in human society and collaborate with humans, we see three challenges ahead affecting meaningful control of autonomous systems. First, autonomous machines are not yet capable of handling failures and unexpected situations. Providing procedures for all possible failures and situations is unfeasible because the state-action space would explode. Machines should therefore become self-aware (self-assessment, self-management) enabling them to handle unexpected situations when they arise. This is a challenge for the computer science community. Second, in order to keep (meaningful) control, humans come into a new role of providing intelligent autonomous machines with objectives or goal functions (including rules, norms, constraints and moral values), specifying the utility of every possible outcome of actions of autonomous machines. Third, in order to be able to collaborate with humans, autonomous systems will require an understanding of (us) humans (i.e., our social, cognitive, affective and physical behaviors) and the ability to engage in partnership interactions (such as explanations of task performances, and the establishment of joint goals and work agreements). These are new challenges for the cognitive ergonomics community.

Original languageEnglish
Title of host publicationProceedings of the 36th European Conference on Cognitive Ergonomics, ECCE 2018
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery (ACM)
Number of pages8
ISBN (Electronic)978-1-4503-6449-2
Publication statusPublished - 2018
Event36th European Conference on Cognitive Ergonomics, ECCE 2018 - Utrecht, Netherlands
Duration: 5 Sep 20187 Sep 2018


Conference36th European Conference on Cognitive Ergonomics, ECCE 2018


  • Autonomous systems
  • Cognitive ergonomics
  • Control


Dive into the research topics of 'Telling autonomous systems what to do'. Together they form a unique fingerprint.

Cite this