A Cross-Field Review of State Abstraction for Markov Decision Processes

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

38 Downloads (Pure)

Abstract

Complex real-world systems pose a significant challenge to decision making: an agent needs to explore a large environment, deal with incomplete or noisy information, generalize the experience and learn from feedback to act optimally. These processes demand vast representation capacity, thus putting a burden on the agent’s limited computational and storage resources. State abstraction enables effective solutions by forming concise representations of the agents world. As such, it has been widely investigated by several research communities which have produced a variety of different approaches. Nonetheless, relations among them still remain unseen or roughly defined. This hampers potential applications of solution methods whose scope remains limited to the specific abstraction context for which they have been designed. To this end, the goal of this paper is to organize the developed approaches and identify connections between abstraction schemes as a fundamental step towards methods generalization. As a second contribution we discuss general abstraction properties with the aim of supporting a unified perspective for state abstraction.
Original languageEnglish
Title of host publication34th Benelux Conference on Artificial Intelligence (BNAIC) and the 30th Belgian Dutch Conference on Machine Learning (Benelearn)
Number of pages15
Publication statusPublished - 2022
Event34th Benelux Conference on Artificial Intelligence (BNAIC) and the 30th Belgian Dutch Conference on Machine Learning (Benelearn) - Mechelen, Belgium
Duration: 7 Nov 20229 Nov 2022

Conference

Conference34th Benelux Conference on Artificial Intelligence (BNAIC) and the 30th Belgian Dutch Conference on Machine Learning (Benelearn)
Abbreviated titleBNAIC/BeNeLearn 2022
Country/TerritoryBelgium
CityMechelen
Period7/11/229/11/22

Keywords

  • State Abstraction
  • Model Irrelevance
  • Robust Reinforcement Learning
  • Bounded Parameters Markov Decision Processes

Fingerprint

Dive into the research topics of 'A Cross-Field Review of State Abstraction for Markov Decision Processes'. Together they form a unique fingerprint.

Cite this