Model-plant mismatch compensation using reinforcement learning

Ivan Koryakovskiy, Manuel Kudruss, Heike Vallery, Robert Babuska, Wouter Caarls

Research output: Contribution to journalArticleScientificpeer-review

26 Citations (Scopus)
70 Downloads (Pure)


Learning-based approaches are suitable for the control of systems with unknown dynamics. However, learning from scratch involves many trials with exploratory actions until a good control policy is discovered. Real robots usually cannot withstand the exploratory actions and suffer damage. This problem can be circumvented by combining learning with a model-based control. In this letter, we employ a nominal model-predictive controller that is impeded by the presence of an unknown model-plant mismatch. To compensate for the mismatch, we propose two approaches of combining reinforcement learning with the nominal controller. The first approach learns a compensatory control action that minimizes the same performance measure as is minimized by the nominal controller. The second approach learns a compensatory signal from a difference of a transition predicted by the internal model and an actual transition. We compare the approaches on a robot attached to the ground and performing a setpoint reaching task in simulations. We implement the better approach on the real robot and demonstrate successful learning results.
Original languageEnglish
Pages (from-to)2471 - 2477
JournalIEEE Robotics and Automation Letters
Issue number3
Publication statusPublished - 2018

Bibliographical note

Accepted Author Manuscript


Dive into the research topics of 'Model-plant mismatch compensation using reinforcement learning'. Together they form a unique fingerprint.

Cite this