Reinforcement learning with symbolic input-output models

Erik Derner, Jiří Kubalík, Robert Babuska

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

10 Citations (Scopus)

Abstract

It is well known that reinforcement learning (RL) can benefit from the use of a dynamic prediction model which is learned on data samples collected online from the process to be controlled. Most RL algorithms are formulated in the state-space domain and use state-space models. However, learning state-space models is difficult, mainly because in the vast majority of problems the full state cannot be measured on the system or reconstructed from the measurements. To circumvent this limitation, we propose to use input-output models of the NARX (nonlinear autoregressive with exogenous input) type. Symbolic regression is employed to construct parsimonious models and the corresponding value functions. Thanks to this approach, we can learn accurate models and compute optimal policies even from small amounts of training data. We demonstrate the approach on two simulated examples, a hopping robot and a 1-DOF robot arm, and on a real inverted pendulum system. Results show that our proposed method can reliably determine a good control policy based on a symbolic input-output process model and value function.
Original languageEnglish
Title of host publicationTowards a Robotic Society
Subtitle of host publicationProceedings 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Place of PublicationPiscataway, NJ, USA
PublisherIEEE
Pages3004-3009
ISBN (Electronic)978-1-5386-8094-0
ISBN (Print)978-1-5386-8095-7
DOIs
Publication statusPublished - 2018
Event2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 - Madrid, Spain
Duration: 1 Oct 20185 Oct 2018

Conference

Conference2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
Country/TerritorySpain
CityMadrid
Period1/10/185/10/18

Fingerprint

Dive into the research topics of 'Reinforcement learning with symbolic input-output models'. Together they form a unique fingerprint.

Cite this