Beyond Known Objects: Prompting 6D Pose Estimator to Learn Novel Objects Continually

Robust 6D object pose estimation is essential for enabling intelligent systems to interact with the physical world. While existing methods have achieved notable progress, they remain constrained by two major limitations: poor scalability when new objects are introduced and severe performance degradation caused by catastrophic forgetting in sequential learning settings. To address these challenges, we propose a novel prompt-based continual learning framework for 6D pose estimation. Our approach leverages frozen foundation features for robust appearance and geometric representation, while introducing a Multi-Head pose predictor to isolate task-specific regressors and a Prompt pool that serves as a rehearsal-free knowledge repository. The proposed framework enables task-agnostic inference by retrieving relevant prompts and activating the appropriate predictor without requiring explicit task labels. We conduct comprehensive experiments on the Linemod, YCB-Video, and Occlusion Linemod datasets under various settings within continual learning. Results show that our method achieves competitive performance, while exhibiting strong robustness to task order, adaptability to arbitrary object groupings, and high scalability through lightweight predictor expansion. To the best of our knowledge, this work represents the first systematic study of continual learning for 6D pose estimation, providing a foundation toward scalable and adaptive open-world perception. The project website is available at: https://lyhellopython.github.io/CLPose/