Sensors, algorithms, and representations for efficient environment perception

that do not require any action from the drivers for a short period of time. Although these systems are still limited and only reliable in certain situations, it shows the general trend: cars will become more and more autonomous. The reasons why people and companies are eagerly anticipating fully autonomous cars are manifold: self-driving vehicles could provide mobility to people unable to drive themselves, they could reduce the need for parking spaces in inner cities, they could decrease traffic jams, and of course, they let passengers spend their time on something else than actively driving. Self-driving vehicles also have the potential to eliminate human error as a cause of traffic accidents and thereby increase traffic safety. Thus, the presence of driver assistance systems and self-driving vehicles in traffic will inevitably increase, and it is crucial to make the technology as safe as possible.

An essential building block to achieving safe autonomous driving is the efficient perception and representation of the vehicle’s environment. The perception and representation need to be as accurate as possible, but at the same time, as efficient as possible, to increase the time in which the vehicle can react to the evolving traffic situation. This thesis discusses various ways to increase the efficiency of perception systems of autonomous vehicles by showing: how a novel acoustic sensor detects traffic before it becomes visible, how to combine traditional machine learning algorithms with deep neural networks for faster inference, how a compact representation for images of traffic scenes can be enriched with object instance information, and how different modalities, such as images and point clouds, contribute to deep representation learning.

To detect vehicles ahead of commonly used sensors in autonomous vehicles, this thesis introduces a passive acoustic perception approach. This acoustic perception system can detect approaching vehicles behind blind corners by sound before such vehicles enter in line-of-sight. A research vehicle equipped with a roof-mounted microphone array is used to collect data and serves as a demonstrator platform. The data shows that wall reflections provide information on the presence and direction of occluded approaching vehicles. In test scenarios with a static ego-vehicle, a novel data-driven approach achieves an accuracy of 0.92 on the hidden vehicle classification task. Compared to a state-of-the-art visual detector, Faster R-CNN, the acoustic system achieves the same accuracy more than one second ahead, providing crucial reaction time for the situations studied in this work. While the ego-vehicle is driving, acoustic detection shows encouraging results, still achieving an accuracy of 0.84 within one environment type. Further, failure cases are studied across environments to identify future research directions...