Learning with Noisy Labels for Robust Point Cloud Segmentation

Robust Point Cloud Segmentation with Noisy Annotations


1City University of Hong Kong, 2Microsoft Cloud AI, 3University of California San Diego

ICCV 2021, Oral Presentation

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2022

Illustration of the label noise concept in point cloud segmentation. From left to right are the input (noisy instances highlighted boxes), the manual annotation given by the real-world and popular dataset ScanNetV2, and the prediction of our framework (more in line with the real category and the real boundary). It is noticeable that this popular dataset suffers from label noise, even though it is already a re-labeled version of ScanNet. In the first row, the GT label noise is at instance-level, where the floor was mislabeled as a chair. In the second row, we show boundary-level label noise, where we can find inaccurate GT boundaries of door and photo. Our framework is trained on this noisy dataset but still achieves correct predictions.

Abstract

Point cloud segmentation is a fundamental task in 3D. Despite recent progress on point cloud segmentation with the power of deep networks, current learning methods based on the clean label assumptions may fail with noisy labels. Yet, class labels are often mislabeled at both instance-level and boundary-level in real-world datasets.

In this work, we take the lead in solving the instance-level label noise by proposing a Point Noise-Adaptive Learning (PNAL) framework. Compared to noise-robust methods on image tasks, our framework is noise-rate blind, to cope with the spatially variant noise rate problem specific to point clouds. Specifically, we propose a point-wise confidence selection to obtain reliable labels from the historical predictions of each point. A cluster-wise label correction is proposed with a voting strategy to generate the best possible label by considering the neighbor correlations.

To handle boundary-level label noise, we also propose a variant "PNAL-boundary" with a progressive boundary label cleaning strategy.

Extensive experiments demonstrate its effectiveness on both synthetic and real-world noisy datasets. Even with $60\%$ symmetric noise and high-level boundary noise, our framework significantly outperforms its baselines, and is comparable to the upper bound trained on completely clean data. Moreover, we cleaned the popular real-world dataset ScanNetV2 for rigorous experiment. Our code and data is available at https://github.com/pleaseconnectwifi/PNAL.

Acknowledgement

Particularly thank Jiaying Lin for providing the initial idea, constructive suggestions, and generous support to this project.

BibTeX

@article{pnal2021,
  author    = {Ye, Shuquan and Chen, Dongdong and Han, Songfang and Liao, Jing},
  title     = {Learning with Noisy Labels for Robust Point Cloud Segmentation},
  journal   = {International Conference on Computer Vision},
  year      = {2021},
}
@ARTICLE {pnalpami2022,
author = {Ye, Shuquan and Chen, Dongdong and Han, Songfang and Liao, Jing},
journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence},
title = {Robust Point Cloud Segmentation with Noisy Annotations},
issn = {1939-3539},
pages = {1-14},
doi = {10.1109/TPAMI.2022.3225323},
}