Push-the-Boundary: Boundary-Aware Feature Propagation for Semantic Segmentation of 3D Point Clouds

Shenglan Du*, Nail İbrahimli, Jantien Stoter, Julian Kooij, Liangliang Nan

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientific

2 Citations (Scopus)
14 Downloads (Pure)

Abstract

Feedforward fully convolutional neural networks currently dominate in semantic segmentation of 3D point clouds. Despite their great success, they suffer from the loss of local information at low-level layers, posing significant challenges to accurate scene segmentation and precise object boundary delineation. Prior works either address this issue by post-processing or jointly learn object boundaries to implicitly improve feature encoding of the networks. These approaches often require additional modules which are difficult to integrate into the original architecture. To improve the segmentation near object boundaries, we propose a boundary-aware feature propagation mechanism. This mechanism is achieved by exploiting a multitask learning framework that aims to explicitly guide the boundaries to their original locations. With one shared encoder, our network outputs (i) boundary localization, (ii) prediction of directions pointing to the object's interior, and (iii) semantic segmentation, in three parallel streams. The predicted boundaries and directions are fused to propagate the learned features to refine the segmentation. We conduct extensive experiments on the S3DIS and SensatUrban datasets against various baseline methods, demonstrating that our proposed approach yields consistent improvements by reducing boundary errors. Our code is available at https://github.com/shenglandu/PushBoundary.

Original languageEnglish
Title of host publicationProceedings - 2022 International Conference on 3D Vision, 3DV 2022
EditorsCristina Ceballos
Place of PublicationPrague
PublisherIEEE
Pages124-133
Number of pages10
ISBN (Electronic)9781665456708
ISBN (Print)978-1-6654-5670-8
DOIs
Publication statusPublished - 2022
Event2022 International Conference on 3D Vision (3DV) - Czech Technical University, Prague, Czech Republic
Duration: 12 Sept 202215 Sept 2022
https://3dvconf.github.io/2022/

Conference

Conference2022 International Conference on 3D Vision (3DV)
Abbreviated title3DV2022
Country/TerritoryCzech Republic
CityPrague
Period12/09/2215/09/22
Internet address

Bibliographical note

This work was supported by the 3D Urban Understanding Lab funded by the TU Delft AI Initiative.

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • point cloud compression
  • location awareness
  • three-dimensional displays
  • semantic segmentation
  • semantics
  • self-supervised learning
  • encoding

Fingerprint

Dive into the research topics of 'Push-the-Boundary: Boundary-Aware Feature Propagation for Semantic Segmentation of 3D Point Clouds'. Together they form a unique fingerprint.

Cite this