Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor Point Clouds


Yuyang Zhao (National University of Singapore), Na Zhao (Singapore University of Technology and Design), Gim Hee Lee (National University of Singapore)
The 35th British Machine Vision Conference
PDFPoster

Abstract

Recent advancements in semantic segmentation for 3D indoor scenes have yielded impressive results using large-scale annotated data. However, existing methods operate under the assumption that training and testing data share the same distribution, resulting in performance degradation when evaluated on out-of-distribution scenes. To address the high annotation cost and performance degradation, we introduce a synthetic-to-real domain generalization setting for this task, which trains a robust model on synthetic domains and evaluates its performance on unseen real-world target domains. The domain shift between synthetic and real-world point cloud data mainly lies in the different layouts and point patterns. To address these problems, we first propose a clustering instance mix (CINMix) augmentation technique to diversify the layouts of the source data. In addition, we augment the point patterns of the source data and introduce non-parametric multi-prototypes to ameliorate the intra-class variance enlarged by the augmented point patterns. The multi-prototypes can model the intra-class variance and rectify the global classifier in both training and inference stages. Experiments on the synthetic-to-real benchmark demonstrate that both CINMix and multi-prototypes can narrow the distribution gap and thus improve the generalization ability on real-world datasets.

Citation

@inproceedings{Zhao_2024_BMVC,
author    = {Yuyang Zhao and Na Zhao and Gim Hee Lee},
title     = {Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor Point Clouds},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0164.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection