State of the Art on Neural Rendering

Simple item page
dc.contributor.authorTewari, Ayushen_US
dc.contributor.authorFried, Ohaden_US
dc.contributor.authorThies, Justusen_US
dc.contributor.authorSitzmann, Vincenten_US
dc.contributor.authorLombardi, Stephenen_US
dc.contributor.authorSunkavalli, Kalyanen_US
dc.contributor.authorMartin-Brualla, Ricardoen_US
dc.contributor.authorSimon, Tomasen_US
dc.contributor.authorSaragih, Jasonen_US
dc.contributor.authorNießner, Matthiasen_US
dc.contributor.authorPandey, Rohiten_US
dc.contributor.authorFanello, Seanen_US
dc.contributor.authorWetzstein, Gordonen_US
dc.contributor.authorZhu, Jun-Yanen_US
dc.contributor.authorTheobalt, Christianen_US
dc.contributor.authorAgrawala, Maneeshen_US
dc.contributor.authorShechtman, Elien_US
dc.contributor.authorGoldman, Dan B.en_US
dc.contributor.authorZollhöfer, Michaelen_US
dc.contributor.editorMantiuk, Rafal and Sundstedt, Veronicaen_US
dc.date.accessioned2020-05-24T13:45:13Z
dc.date.available2020-05-24T13:45:13Z
dc.date.issued2020
dc.description.abstractEfficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on the type of control, i.e., how the control is provided, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. The second half of this state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.en_US
dc.description.documenttypestar
dc.description.number2
dc.description.sectionheadersState of the Art Reports
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume39
dc.identifier.doi10.1111/cgf.14022
dc.identifier.issn1467-8659
dc.identifier.pages701-727
dc.identifier.urihttps://doi.org/10.1111/cgf.14022
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14022
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/]
dc.titleState of the Art on Neural Renderingen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v39i2pp701-727.pdf
Size:
26.89 MB
Format:
Adobe Portable Document Format
Download
Collections
EG 2020 - STARs (CGF 39-2)