Abstract
It is well known that reinforcement learning (RL) can benefit from the use of a dynamic prediction model which is learned on data samples collected online from the process to be controlled. Most RL algorithms are formulated in the state-space domain and use state-space models. However, learning state-space models is difficult, mainly because in the vast majority of problems the full state cannot be measured on the system or reconstructed from the measurements. To circumvent this limitation, we propose to use input-output models of the NARX (nonlinear autoregressive with exogenous input) type. Symbolic regression is employed to construct parsimonious models and the corresponding value functions. Thanks to this approach, we can learn accurate models and compute optimal policies even from small amounts of training data. We demonstrate the approach on two simulated examples, a hopping robot and a 1-DOF robot arm, and on a real inverted pendulum system. Results show that our proposed method can reliably determine a good control policy based on a symbolic input-output process model and value function.
Original language | English |
---|---|
Title of host publication | Towards a Robotic Society |
Subtitle of host publication | Proceedings 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) |
Place of Publication | Piscataway, NJ, USA |
Publisher | IEEE |
Pages | 3004-3009 |
ISBN (Electronic) | 978-1-5386-8094-0 |
ISBN (Print) | 978-1-5386-8095-7 |
DOIs | |
Publication status | Published - 2018 |
Event | 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 - Madrid, Spain Duration: 1 Oct 2018 → 5 Oct 2018 |
Conference
Conference | 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 |
---|---|
Country/Territory | Spain |
City | Madrid |
Period | 1/10/18 → 5/10/18 |