PG: Pacific Graphics Short Papers

Permanent URI for this community

Now showing 1 - 1 of 1

Label Name is Mantra: Unifying Point Cloud Segmentation across Heterogeneous Datasets
(The Eurographics Association, 2024) Liang, Yixun; He, Hao; Xiao, Shishi; Lu, Hao; Chen, Yingcong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Point cloud segmentation is a fundamental task in 3D vision that serves a wide range of applications. Despite recent advancements, its practical usability is still limited by the availability of training data. The prevalent methodologies cannot optimally exploit multiple datasets due to the inconsistency of labels across datasets. In this work, we introduce a robust method that accommodates learning from diverse datasets with variant label sets. We leverage a pre-trained language model to map discrete labels into a continuous latent space using their semantic names. This harmonizes labels across datasets, facilitating concurrent training. Contrarily, when classifying points within the continuous 3D space via their linguistic tokens, our model exhibits superior generalizability compared to extant methods with fixed decoder structures. Further, our approach assimilates prompt learning to alleviate data shifts across sources. Comprehensive evaluations attest that our model markedly surpasses current benchmarks.