GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization

Simple item page
dc.contributor.authorSun, Yanhaoen_US
dc.contributor.authorTian, Runzeen_US
dc.contributor.authorHan, Xiaoen_US
dc.contributor.authorLiu, Xinyaoen_US
dc.contributor.authorZhang, Yanen_US
dc.contributor.authorXu, Kaien_US
dc.contributor.editorChen, Renjieen_US
dc.contributor.editorRitschel, Tobiasen_US
dc.contributor.editorWhiting, Emilyen_US
dc.date.accessioned2024-10-13T18:07:47Z
dc.date.available2024-10-13T18:07:47Z
dc.date.issued2024
dc.description.abstractWith the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real-time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.en_US
dc.description.number7
dc.description.sectionheaders3D Reconstruction and Novel View Synthesis II
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume43
dc.identifier.doi10.1111/cgf.15215
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.15215
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf15215
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Rendering; Point-based models; Computer vision representations
dc.subjectComputing methodologies → Rendering
dc.subjectPoint
dc.subjectbased models
dc.subjectComputer vision representations
dc.titleGSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localizationen_US
Files
Original bundle
Now showing 1 - 3 of 3
No Thumbnail Available
Name:
cgf15215.pdf
Size:
3.73 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
supplementary_pg2024.pdf
Size:
284.11 KB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
video.mp4
Size:
28.44 MB
Format:
Video MP4
Download
Collections
43-Issue 7