Reinforcement learning of potential fields to achieve limit-cycle walking

D.S. Feirstein (student), Ivan Koryakovskiy, Jens Kober, Heike Vallery

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

6 Citations (Scopus)
285 Downloads (Pure)


Reinforcement learning is a powerful tool to derive controllers for systems where no models are available. Particularly policy search algorithms are suitable for complex systems, to keep learning time manageable and account for continuous state and action spaces. However, these algorithms demand more insight into the system to choose a suitable controller parameterization. This paper investigates a type of policy parameterization for impedance control that allows energy input to be implicitly bounded: Potential fields. In this work, a methodology for generating a potential field-constrained impedance controller via approximation of example trajectories, and subsequently improving the control policy using Reinforcement Learning, is presented. The potential field-const rained approximation is used as a policy parameterization for policy search reinforcement learning and is compared to its unconstrained counterpart. Simulations on a simple biped walking model show the learned controllers are able to surpass the potential field of gravity by generating a stable limit-cycle gait on flat ground for both parameterizations. The potential field-constrained controller provides safety with a known energy bound while performing equally well as the unconstrained policy.

Original languageEnglish
Title of host publicationProceedings of the 6th IFAC Workshop on Periodic Control Systems (PSYCO 2016)
EditorsHenk Nijmeijer
Publication statusPublished - 2016
EventPSYCO 2016: 6th IFAC Workshop on Periodic Control System - Eindhoven, Netherlands
Duration: 29 Jun 20161 Jul 2016

Publication series

ISSN (Electronic)2405-8963


WorkshopPSYCO 2016: 6th IFAC Workshop on Periodic Control System


  • Energy Control
  • Limit cycles
  • Machine learning
  • Robot control
  • Walking


Dive into the research topics of 'Reinforcement learning of potential fields to achieve limit-cycle walking'. Together they form a unique fingerprint.

Cite this