Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision

Fangqiang Ding, Andras Palffy, Dariu Gavrila, Chris Xiaoxuan Lu*

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

21 Downloads (Pure)

Abstract

This work proposes a novel approach to 4D radar-based scene flow estimation via cross-modal learning. Our approach is motivated by the co-located sensing redundancy in modern autonomous vehicles. Such redundancy implicitly provides various forms of supervision cues to the radar scene flow estimation. Specifically, we introduce a multi-task model architecture for the identified cross-modal learning problem and propose loss functions to opportunistically engage scene flow estimation using multiple cross-modal constraints for effective model training. Extensive experiments show the state-of-the-art performance of our method and demonstrate the effectiveness of cross-modal super-vised learning to infer more accurate 4D radar scene flow. We also show its usefulness to two subtasks - motion segmentation and ego-motion estimation. Our source code will be available on https://github.com/Toytiny/CMFlow.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
EditorsLisa O'Conner
Place of PublicationPiscataway
PublisherIEEE
Pages9340-9349
Number of pages10
ISBN (Electronic)979-8-3503-0129-8
ISBN (Print)979-8-3503-0130-4
DOIs
Publication statusPublished - 2023
Event2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) - Vancouver, Canada
Duration: 17 Jun 202324 Jun 2023

Conference

Conference2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Country/TerritoryCanada
CityVancouver
Period17/06/2324/06/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Fingerprint

Dive into the research topics of 'Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision'. Together they form a unique fingerprint.

Cite this