Conflict Resolution at High Traffic Densities with Reinforcement Learning

Increasing delays and congestion reported in many aviation sectors indicate that the current centralised operational model is rapidly approaching saturation levels. Air Traffic Control (ATC) system is not expected to keep pace with the ever-increasing demand for air transportation. Its capacity is still limited by the available controllers, and the number of aircraft that each controller can manage. This system cannot be stretched any further under its current conditions. However, it is expected that the number of aircraft operating simultaneously will continue to increase. On top of this, new unmanned aviation operations promise traffic densities never seen before. The expected future increase in traffic demand has redirected focus into automated tools and alternative approaches. This research has been primarily characterised by a change in the degree of centralisation, more specifically by exploring distributed options, where control is transferred from ground-based Air Traffic Controllers (ATCOs) to each individual aircraft. As each aircraft only takes into account its neighbouring aircraft when resolving conflicts, each distributed resolution system is expected to have only a fraction of the computational strain that a centralised system would have. Nevertheless, a distributed approach has its own challenges. A crucial disadvantage is the lack of global coordination from surrounding traffic, which can affect safety. Without knowledge of the movement of intruders, decentralised solutions cannot guarantee globally optimal solutions when more than two aircraft are involved. Conflict resolution (CR) methods based on geometric solutions have proven to be very successful in achieving a high level of safety for one-to-one conflicts, where a set of rules can be defined which leads to implicitly coordinated optimal behaviour. However, at higher traffic densities, when individual conflict situations can no longer be considered isolated events, successive CR manoeuvres can lead to traffic patterns with a negative effect on the global safety. Knock-on effects of intruders avoiding each othermay result in unforeseen trajectory changes. The latter increases uncertainty regarding intruders’ future movements, decreasing the efficacy of conflict resolution manoeuvres. The goal of this research is to improve upon aircraft self-separation efficacy at higher traffic densities, with an emphasis on employing airspace designs and approaches applicable to future unmanned operations. To do so, we look at a scenario with multi aircraft interacting as a multi-agent problem. Analysis and understanding of emergent behaviour in a multi-agent environment is often almost impossible to the human eye. However, reinforcement learning (RL) techniques are often capable of identifying emerging patterns through training in the environment. We translate successful applications of RL techniques in other areas (e.g., carmobility, lane changing, freeways) to aircraft operational scenarios to mitigate the negative effect on safety...