Risk-sensitive Distributional Reinforcement Learning for Flight Control

Peter Seres; Cheng Liu; Erik Jan van Kampen

doi:10.1016/j.ifacol.2023.10.1097

Risk-sensitive Distributional Reinforcement Learning for Flight Control

Peter Seres^*, Cheng Liu^*, Erik Jan van Kampen^*

^*Corresponding author for this work

Control & Simulation

Research output: Contribution to journal › Conference article › Scientific › peer-review

3 Citations (SciVal)

40 Downloads (Pure)

Abstract

Recent aerospace systems increasingly demand model-free controller synthesis, and autonomous operations require adaptability to uncertainties in partially observable environments. This paper applies distributional reinforcement learning to synthesize risk-sensitive, robust model-free policies for aerospace control. We investigate the use of distributional soft actor-critic (DSAC) agents for flight control and compare their learning characteristics and tracking performance with the soft actor-critic (SAC) algorithm. The results show that (1) the addition of distributional critics significantly improves learning consistency, (2) risk-averse agents increase flight safety by avoiding uncertainties in the environment.

Original language	English
Pages (from-to)	2013-2018
Number of pages	6
Journal	IFAC-PapersOnLine
Volume	56
Issue number	2
DOIs	https://doi.org/10.1016/j.ifacol.2023.10.1097
Publication status	Published - 2023
Event	22nd IFAC World Congress - Yokohama, Japan Duration: 9 Jul 2023 → 14 Jul 2023

Keywords

Distributional reinforcement learning
Guidance
navigation and control of vehicles
Reinforcement learning control
Risk-sensitive learning

Access to Document

10.1016/j.ifacol.2023.10.1097

1-s2.0-S2405896323015008-mainFinal published version, 989 KBLicence: CC BY-NC-ND

Cite this

@article{15883e3a79ff42ac9152bcaa390f2dcf,

title = "Risk-sensitive Distributional Reinforcement Learning for Flight Control",

abstract = "Recent aerospace systems increasingly demand model-free controller synthesis, and autonomous operations require adaptability to uncertainties in partially observable environments. This paper applies distributional reinforcement learning to synthesize risk-sensitive, robust model-free policies for aerospace control. We investigate the use of distributional soft actor-critic (DSAC) agents for flight control and compare their learning characteristics and tracking performance with the soft actor-critic (SAC) algorithm. The results show that (1) the addition of distributional critics significantly improves learning consistency, (2) risk-averse agents increase flight safety by avoiding uncertainties in the environment.",

keywords = "Distributional reinforcement learning, Guidance, navigation and control of vehicles, Reinforcement learning control, Risk-sensitive learning",

author = "Peter Seres and Cheng Liu and {van Kampen}, {Erik Jan}",

year = "2023",

doi = "10.1016/j.ifacol.2023.10.1097",

language = "English",

volume = "56",

pages = "2013--2018",

journal = "IFAC-PapersOnLine",

issn = "2405-8963",

publisher = "Elsevier",

number = "2",

note = "22nd IFAC World Congress ; Conference date: 09-07-2023 Through 14-07-2023",

}

TY - JOUR

T1 - Risk-sensitive Distributional Reinforcement Learning for Flight Control

AU - Seres, Peter

AU - Liu, Cheng

AU - van Kampen, Erik Jan

PY - 2023

Y1 - 2023

N2 - Recent aerospace systems increasingly demand model-free controller synthesis, and autonomous operations require adaptability to uncertainties in partially observable environments. This paper applies distributional reinforcement learning to synthesize risk-sensitive, robust model-free policies for aerospace control. We investigate the use of distributional soft actor-critic (DSAC) agents for flight control and compare their learning characteristics and tracking performance with the soft actor-critic (SAC) algorithm. The results show that (1) the addition of distributional critics significantly improves learning consistency, (2) risk-averse agents increase flight safety by avoiding uncertainties in the environment.

AB - Recent aerospace systems increasingly demand model-free controller synthesis, and autonomous operations require adaptability to uncertainties in partially observable environments. This paper applies distributional reinforcement learning to synthesize risk-sensitive, robust model-free policies for aerospace control. We investigate the use of distributional soft actor-critic (DSAC) agents for flight control and compare their learning characteristics and tracking performance with the soft actor-critic (SAC) algorithm. The results show that (1) the addition of distributional critics significantly improves learning consistency, (2) risk-averse agents increase flight safety by avoiding uncertainties in the environment.

KW - Distributional reinforcement learning

KW - Guidance

KW - navigation and control of vehicles

KW - Reinforcement learning control

KW - Risk-sensitive learning

UR - http://www.scopus.com/inward/record.url?scp=85184963093&partnerID=8YFLogxK

U2 - 10.1016/j.ifacol.2023.10.1097

DO - 10.1016/j.ifacol.2023.10.1097

M3 - Conference article

AN - SCOPUS:85184963093

SN - 2405-8963

VL - 56

SP - 2013

EP - 2018

JO - IFAC-PapersOnLine

JF - IFAC-PapersOnLine

IS - 2

T2 - 22nd IFAC World Congress

Y2 - 9 July 2023 through 14 July 2023

ER -

Risk-sensitive Distributional Reinforcement Learning for Flight Control

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this