Learning with Noisy Labels for Robust Point Cloud Segmentation

1City University of Hong Kong, 2Microsoft Cloud AI, 3University of California San Diego

ICCV 2021, Oral Presentation

Illustration of the instance-level label noise concept in point cloud segmentation. From left to right are the input (noisy instances highlighted red boxes), the manual annotation given by the real-world dataset ScanNetV2, and the prediction of the proposed Point Noise-Adaptive Learning (PNAL) framework which is more in line with the real category. It is noticeable that this popular dataset suffers from label noise, such as mislabeling the floor as a chair, even that it is already a re-labeled version of ScanNet. Our PNAL framework is trained on this noisy dataset but still achieves correct predictions.


Point cloud segmentation is a fundamental task in 3D. Despite recent progress on point cloud segmentation with the power of deep networks, current deep learning methods based on the clean label assumptions may fail with noisy labels. Yet, object class labels are often mislabeled in real-world point cloud datasets.

In this work, we take the lead in solving this issue by proposing a novel Point Noise-Adaptive Learning (PNAL) framework.

Compared to existing noise-robust methods on image tasks, our PNAL is noise-rate blind, to cope with the spatially variant noise rate problem specific to point clouds. Specifically, we propose a novel point-wise confidence selection to obtain reliable labels based on the historical predictions of each point. A novel cluster-wise label correction is proposed with a voting strategy to generate the best possible label taking the neighbor point correlations into consideration. We conduct extensive experiments to demonstrate the effectiveness of PNAL on both synthetic and real-world noisy datasets. In particular, even with 60% symmetric noisy labels, our proposed method produces much better results than its baseline counterpart without PNAL and is comparable to the ideal upper bound trained on a completely clean dataset.

In addition, we re-labeled the validation set of 312 scenes of a popular but noisy real-world scene dataset ScanNetV2 to make it clean, for rigorous experiment and for future research. Our code and data will be released.


I would like to give my particular thanks to Jiaying Lin, my special friend, for his constructive suggestions, generous support to this project, as well as the tremendous love given to me.


  author    = {Ye, Shuquan and Chen, Dongdong and Han, Songfang and Liao, Jing},
  title     = {Learning with Noisy Labels for Robust Point Cloud Segmentation},
  journal   = {International Conference on Computer Vision},
  year      = {2021},