D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video

Simple item page
dc.contributor.authorKappel, Moritzen_US
dc.contributor.authorHahlbohm, Florianen_US
dc.contributor.authorScholz, Timonen_US
dc.contributor.authorCastillo, Susanaen_US
dc.contributor.authorTheobalt, Christianen_US
dc.contributor.authorEisemann, Martinen_US
dc.contributor.authorGolyanik, Vladislaven_US
dc.contributor.authorMagnor, Marcusen_US
dc.contributor.editorBousseau, Adrienen_US
dc.contributor.editorDay, Angelaen_US
dc.date.accessioned2025-05-09T09:12:40Z
dc.date.available2025-05-09T09:12:40Z
dc.date.issued2025
dc.description.abstractDynamic reconstruction and spatiotemporal novel-view synthesis of non-rigidly deforming scenes recently gained increased attention. While existing work achieves impressive quality and performance on multi-view or teleporting camera setups, most methods fail to efficiently and faithfully recover motion and appearance from casual monocular captures. This paper contributes to the field by introducing a new method for dynamic novel view synthesis from monocular video, such as casual smartphone captures. Our approach represents the scene as a dynamic neural point cloud, an implicit time-conditioned point distribution that encodes local geometry and appearance in separate hash-encoded neural feature grids for static and dynamic regions. By sampling a discrete point cloud from our model, we can efficiently render high-quality novel views using a fast differentiable rasterizer and neural rendering network. Similar to recent work, we leverage advances in neural scene analysis by incorporating data-driven priors like monocular depth estimation and object segmentation to resolve motion and depth ambiguities originating from the monocular captures. In addition to guiding the optimization process, we show that these priors can be exploited to explicitly initialize our scene representation to drastically improve optimization speed and final image quality. As evidenced by our experimental evaluation, our dynamic point cloud model not only enables fast optimization and real-time frame rates for interactive applications, but also achieves competitive image quality on monocular benchmark sequences. Our code and data are available online https://moritzkappel.github.io/projects/dnpc/.en_US
dc.description.number2
dc.description.sectionheadersFix it in Post: Image and Video Synthesis and Analysis
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume44
dc.identifier.doi10.1111/cgf.70038
dc.identifier.issn1467-8659
dc.identifier.pages13 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.70038
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf70038
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Image-based rendering; Point-based models; Reconstruction; Rasterization
dc.subjectComputing methodologies → Image
dc.subjectbased rendering
dc.subjectPoint
dc.subjectbased models
dc.subjectReconstruction
dc.subjectRasterization
dc.titleD-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Videoen_US
Files
Original bundle
Now showing 1 - 5 of 8
Thumbnail Image
Name:
cgf70038.pdf
Size:
54.12 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
comparisons.mp4
Size:
45.42 MB
Format:
Video MP4
Download
No Thumbnail Available
Name:
gui_live_capture.zip
Size:
54.19 MB
Format:
Zip file
Download
No Thumbnail Available
Name:
paper1095_2.mp4
Size:
122.17 MB
Format:
Video MP4
Download
No Thumbnail Available
Name:
readme.txt
Size:
1.69 KB
Format:
Plain Text
Download
Collections
EG 2025 - Full Papers - CGF 44-Issue 2
44-Issue 2