Integrating state representation learning into deep reinforcement learning

Tim de Bruin, Jens Kober, Karl Tuyls, Robert Babuska

Research output: Contribution to journalArticleScientificpeer-review

86 Citations (Scopus)
81 Downloads (Pure)


Most deep reinforcement learning techniques are unsuitable for robotics, as they require too much interaction time to learn useful, general control policies. This problem can be largely attributed to the fact that a state representation needs to be learned as a part of learning control policies, which can only be done through fitting expected returns based on observed rewards. While the reward function provides information on the desirability of the state of the world, it does not necessarily provide information on how to distill a good, general representation of that state from the sensory observations. State representation learning objectives can be used to help learn such a representation. While many of these objectives have been proposed, they are typically not directly combined with reinforcement learning algorithms. We investigate several methods for integrating state representation learning into reinforcement learning. In these methods, the state representation learning objectives help regularize the state representation during the reinforcement learning, and the reinforcement learning itself is viewed as a crucial state representation learning objective and allowed to help shape the representation. Using autonomous racing tests in the TORCS simulator, we show how the integrated methods quickly learn policies that generalize to new environments much better than deep reinforcement learning without state representation learning.
Original languageEnglish
Pages (from-to)1394-1401
JournalIEEE Robotics and Automation Letters
Issue number3
Publication statusPublished - 2018

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Deep learning in robotics and automation
  • learning and adaptive systems
  • sensor fusion


Dive into the research topics of 'Integrating state representation learning into deep reinforcement learning'. Together they form a unique fingerprint.

Cite this