Point'n Move: Interactive scene object manipulation on Gaussian splatting radiance fields

Abstract

Abstract The authors propose Point'n Move, a method that achieves interactive scene object manipulation with exposed region inpainting. Interactivity here further comes from intuitive object selection and real‐time editing. To achieve this, Gaussian Splatting Radiance Field is adopted as the scene representation and its explicit nature and speed advantage are fully leveraged. Its explicit representation formulation allows to devise a 2D prompt points to 3D masks dual‐stage self‐prompting segmentation algorithm, perform mask refinement and merging, minimize changes, and provide good initialization for scene inpainting and perform editing in real‐time without per‐editing training; all lead to superior quality and performance. The method was tested by editing both forward‐facing and 360 scenes. The method is also compared against existing methods, showing superior quality despite being more capable and having a speed advantage.