OpenDistill3D: Continual Learning and Unknown Object Discovery in 3D Scenes via Self-Distillation
ECCV 2024

Abstract

Open-world 3D instance segmentation is a recently intro- duced problem with diverse applications, notably in continually learning embodied agents. This task involves segmenting unknown instances and learning new instances when their labels are introduced. However, prior research in the open-world domain has traditionally addressed the two sub-problems, namely continual learning and unknown object identifi- cation, separately. This approach has resulted in limited performance on unknown instances and cannot effectively mitigate catastrophic for- getting. Additionally, these methods bypass the utilization of the infor- mation stored in the previous version of the continual learning model, instead relying on a dedicated memory to store historical data sam- ples, which inevitably leads to an expansion of the memory budget. In this paper, we argue that continual learning and unknown object identification are desired to be tackled in conjunction. To this end, we propose a new exemplar-free approach for 3D continual learning and unknown object discovery through continual self-distillation. Our ap- proach, named OpenDistill3D, leverages the pseudo-labels generated by the model from the preceding task to improve the unknown predic- tions during training while simultaneously mitigating catastrophic for- getting. By integrating these pseudo-labels into the continual learning process, we achieve enhanced performance in handling unknown ob- jects. We validate the efficacy of the proposed approach via compre- hensive experiments on various splits of the ScanNet200 dataset, show- casing superior performance in continual learning and unknown object retrieval compared to the state-of-the-art.