Reinforcement learning based compensation methods for robot manipulators

Yudha P. Pane, Subramanya P. Nageshrao*, Jens Kober, Robert Babuška

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

57 Citations (Scopus)
55 Downloads (Pure)


Smart robotics will be a core feature while migrating from Industry 3.0 (i.e., mass manufacturing) to Industry 4.0 (i.e., customized or social manufacturing). A key characteristic of a smart system is its ability to learn. For smart manufacturing, this means incorporating learning capabilities into the current fixed, repetitive, task-oriented industrial manipulators, thus rendering them ‘smart’. In this paper we introduce two reinforcement learning (RL) based compensation methods. The learned correction signal, which compensates for unmodeled aberrations, is added to the existing nominal input with an objective to enhance the control performance. The proposed learning algorithms are evaluated on a 6-DoF industrial robotic manipulator arm to follow different kinds of reference paths, such as square or a circular path, or to track a trajectory on a three dimensional surface. In an extensive experimental study we compare the performance of our learning-based methods with well-known tracking controllers, namely, proportional-derivative (PD), model predictive control (MPC), and iterative learning control (ILC). The experimental results show a considerable performance improvement thanks to our RL-based methods when compared to PD, MPC, and ILC.

Original languageEnglish
Pages (from-to)236-247
JournalEngineering Applications of Artificial Intelligence
Publication statusPublished - 2019

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Actor-critic scheme
  • Reinforcement learning
  • Robotics
  • Tracking control


Dive into the research topics of 'Reinforcement learning based compensation methods for robot manipulators'. Together they form a unique fingerprint.

Cite this