Spiking Neural-Networks-Based Data-Driven Control

Y. Liu, W. Pan

Research output: Contribution to journalArticleScientificpeer-review

143 Downloads (Pure)

Abstract

Machine learning can be effectively applied in control loops to make optimal control decisions robustly. There is increasing interest in using spiking neural networks (SNNs) as the apparatus for machine learning in control engineering because SNNs can potentially offer high energy efficiency, and new SNN-enabling neuromorphic hardware is being rapidly developed. A defining characteristic of control problems is that environmental reactions and delayed rewards must be considered. Although reinforcement learning (RL) provides the fundamental mechanisms to address such problems, implementing these mechanisms in SNN learning has been underexplored. Previously, spike-timing-dependent plasticity learning schemes (STDP) modulated by factors of temporal difference (TD-STDP) or reward (R-STDP) have been proposed for RL with SNN. Here, we designed and implemented an SNN controller to explore and compare these two schemes by considering cart-pole balancing as a representative example. Although the TD-based learning rules are very general, the resulting model exhibits rather slow convergence, producing noisy and imperfect results even after prolonged training. We show that by integrating the understanding of the dynamics of the environment into the reward function of R-STDP, a robust SNN-based controller can be learned much more efficiently than TD-STDP.
Original languageEnglish
Article number310
Number of pages17
JournalElectronics (Switzerland)
Volume12
Issue number2
DOIs
Publication statusPublished - 2023

Keywords

  • spiking neural network
  • reinforcement learning
  • control

Fingerprint

Dive into the research topics of 'Spiking Neural-Networks-Based Data-Driven Control'. Together they form a unique fingerprint.

Cite this