39-Issue 4

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2632924
London, UK | 29 June - 3 July 2020
(Rendering - DL only track and Industry track are available here.)
Table of Contents
Denoising and Filtering
Neural Denoising with Layer Embeddings
[full paper Image] [meta data Image]
Jacob Munkberg and Jon Hasselgren
Natural Appearance
A Scalable and Production Ready Sky and Atmosphere Rendering Technique
[full paper Image] [meta data Image]
Sébastien Hillaire
Path Guiding
Practical Product Path Guiding Using Linearly Transformed Cosines
[full paper Image] [meta data Image]
Stavros Diolatzis, Adrien Gruson, Wenzel Jakob, Derek Nowrouzezahrai, and George Drettakis
Global Illumination
Deep Kernel Density Estimation for Photon Mapping
[full paper Image] [meta data Image]
Shilin Zhu, Zexiang Xu, Henrik Wann Jensen, Hao Su, and Ravi Ramamoorthi
Adaptive Matrix Completion for Fast Visibility Computations with Many Lights Rendering
[full paper Image] [meta data Image]
Sunrise Wang and Nicolas Holzschuch
BxDFs
An Adaptive BRDF Fitting Metric
[full paper Image] [meta data Image]
James Bieron and Pieter Peers
Practical Measurement and Reconstruction of Spectral Skin Reflectance
[full paper Image] [meta data Image]
Yuliya Gitlina, Giuseppe Claudio Guarnera, Daljit Singh Dhillon, Jan Hansen, Alexandros Lattas, Dinesh Pai, and Abhijeet Ghosh
Materials
Guided Fine-Tuning for Large-Scale Material Transfer
[full paper Image] [meta data Image]
Valentin Deschaintre, George Drettakis, and Adrien Bousseau
Photorealistic Material Editing Through Direct Image Manipulation
[full paper Image] [meta data Image]
Károly Zsolnai-Fehér, Peter Wonka, and Michael Wimmer
Sampling
Can't Invert the CDF? The Triangle-Cut Parameterization of the Region under the Curve
[full paper Image] [meta data Image]
Eric Heitz
A Comprehensive Theory and Variational Framework for Anti-aliasing Sampling Patterns
[full paper Image] [meta data Image]
A. Cengiz Öztireli
Practical Product Sampling by Fitting and Composing Warps
[full paper Image] [meta data Image]
David Hart, Matt Pharr, Thomas Müller, Ward Lopes, Morgan McGuire, and Peter Shirley
Images and Textures
Semi-Procedural Textures Using Point Process Texture Basis Functions
[full paper Image] [meta data Image]
Pascal Guehl, Remi Allègre, Jean-Michel Dischler, Bedrich Benes, and Eric Galin
High-Resolution Neural Face Swapping for Visual Effects
[full paper Image] [meta data Image]
Jacek Naruniec, Leonhard Helminger, Christopher Schroers, Romann M. Weber

BibTeX (39-Issue 4)
                
@article{
10.1111:cgf.14049,

journal = {Computer Graphics Forum},
title = {{
Neural Denoising with Layer Embeddings}},

author = {
Munkberg, Jacob
 and 
Hasselgren, Jon
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14049}

}
                
@article{
10.1111:cgf.14050,

journal = {Computer Graphics Forum},
title = {{
A Scalable and Production Ready Sky and Atmosphere Rendering Technique}},

author = {
Hillaire, Sébastien
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14050}

}
                
@article{
10.1111:cgf.14051,

journal = {Computer Graphics Forum},
title = {{
Practical Product Path Guiding Using Linearly Transformed Cosines}},

author = {
Diolatzis, Stavros
 and 
Gruson, Adrien
 and 
Jakob, Wenzel
 and 
Nowrouzezahrai, Derek
 and 
Drettakis, George
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14051}

}
                
@article{
10.1111:cgf.14052,

journal = {Computer Graphics Forum},
title = {{
Deep Kernel Density Estimation for Photon Mapping}},

author = {
Zhu, Shilin
 and 
Xu, Zexiang
 and 
Jensen, Henrik Wann
 and 
Su, Hao
 and 
Ramamoorthi, Ravi
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14052}

}
                
@article{
10.1111:cgf.14053,

journal = {Computer Graphics Forum},
title = {{
Adaptive Matrix Completion for Fast Visibility Computations with Many Lights Rendering}},

author = {
Wang, Sunrise
 and 
Holzschuch, Nicolas
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14053}

}
                
@article{
10.1111:cgf.14054,

journal = {Computer Graphics Forum},
title = {{
An Adaptive BRDF Fitting Metric}},

author = {
Bieron, James
 and 
Peers, Pieter
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14054}

}
                
@article{
10.1111:cgf.14055,

journal = {Computer Graphics Forum},
title = {{
Practical Measurement and Reconstruction of Spectral Skin Reflectance}},

author = {
Gitlina, Yuliya
 and 
Guarnera, Giuseppe Claudio
 and 
Dhillon, Daljit Singh
 and 
Hansen, Jan
 and 
Lattas, Alexandros
 and 
Pai, Dinesh
 and 
Ghosh, Abhijeet
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14055}

}
                
@article{
10.1111:cgf.14056,

journal = {Computer Graphics Forum},
title = {{
Guided Fine-Tuning for Large-Scale Material Transfer}},

author = {
Deschaintre, Valentin
 and 
Drettakis, George
 and 
Bousseau, Adrien
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14056}

}
                
@article{
10.1111:cgf.14058,

journal = {Computer Graphics Forum},
title = {{
Can't Invert the CDF? The Triangle-Cut Parameterization of the Region under the Curve}},

author = {
Heitz, Eric
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14058}

}
                
@article{
10.1111:cgf.14057,

journal = {Computer Graphics Forum},
title = {{
Photorealistic Material Editing Through Direct Image Manipulation}},

author = {
Zsolnai-Fehér, Károly
 and 
Wonka, Peter
 and 
Wimmer, Michael
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14057}

}
                
@article{
10.1111:cgf.14059,

journal = {Computer Graphics Forum},
title = {{
A Comprehensive Theory and Variational Framework for Anti-aliasing Sampling Patterns}},

author = {
Öztireli, A. Cengiz
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14059}

}
                
@article{
10.1111:cgf.14060,

journal = {Computer Graphics Forum},
title = {{
Practical Product Sampling by Fitting and Composing Warps}},

author = {
Hart, David
 and 
Pharr, Matt
 and 
Müller, Thomas
 and 
Lopes, Ward
 and 
McGuire, Morgan
 and 
Shirley, Peter
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14060}

}
                
@article{
10.1111:cgf.14061,

journal = {Computer Graphics Forum},
title = {{
Semi-Procedural Textures Using Point Process Texture Basis Functions}},

author = {
Guehl, Pascal
 and 
Allègre, Remi
 and 
Dischler, Jean-Michel
 and 
Benes, Bedrich
 and 
Galin, Eric
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14061}

}
                
@article{
10.1111:cgf.14062,

journal = {Computer Graphics Forum},
title = {{
High-Resolution Neural Face Swapping for Visual Effects}},

author = {
Naruniec, Jacek
 and 
Helminger, Leonhard
 and 
Schroers, Christopher
 and 
Weber, Romann M.
},
year = {
2020},

publisher = {
The Eurographics Association and John Wiley & Sons Ltd.},


ISSN = {1467-8659},

DOI = {
10.1111/cgf.14062}

}

Browse

Recent Submissions

Now showing 1 - 15 of 15
  • Thumbnail Image
    Item
    Rendering 2020 CGF 39-4: Frontmatter
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Dachsbacher, Carsten; Pharr, Matt; Dachsbacher, Carsten and Pharr, Matt
  • Thumbnail Image
    Item
    Neural Denoising with Layer Embeddings
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Munkberg, Jacob; Hasselgren, Jon; Dachsbacher, Carsten and Pharr, Matt
    We propose a novel approach for denoising Monte Carlo path traced images, which uses data from individual samples rather than relying on pixel aggregates. Samples are partitioned into layers, which are filtered separately, giving the network more freedom to handle outliers and complex visibility. Finally the layers are composited front-to-back using alpha blending. The system is trained end-to-end, with learned layer partitioning, filter kernels, and compositing. We obtain similar image quality as recent state-of-the-art sample based denoisers at a fraction of the computational cost and memory requirements.
  • Thumbnail Image
    Item
    A Scalable and Production Ready Sky and Atmosphere Rendering Technique
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Hillaire, Sébastien; Dachsbacher, Carsten and Pharr, Matt
    We present a physically based method to render the atmosphere of a planet from ground to space views. Our method is cheap to compute and, as compared to previous successful methods, does not require any high dimensional Lookup Tables (LUTs) and thus does not suffer from visual artifacts associated with them. We also propose a new approximation to evaluate light multiple scattering within the atmosphere in real time. We take a new look at what it means to render natural atmospheric effects, and propose a set of simple look up tables and parameterizations to render a sky and its aerial perspective. The atmosphere composition can change dynamically to match artistic visions and weather changes without requiring heavy LUT update. The complete technique can be used in real-time applications such as games, simulators or architecture pre-visualizations. The technique also scales from power-efficient mobile platforms up to PCs with high-end GPUs, and is also useful for accelerating path tracing.
  • Thumbnail Image
    Item
    Practical Product Path Guiding Using Linearly Transformed Cosines
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Diolatzis, Stavros; Gruson, Adrien; Jakob, Wenzel; Nowrouzezahrai, Derek; Drettakis, George; Dachsbacher, Carsten and Pharr, Matt
    Path tracing is now the standard method used to generate realistic imagery in many domains, e.g., film, special effects, architecture etc. Path guiding has recently emerged as a powerful strategy to counter the notoriously long computation times required to render such images. We present a practical path guiding algorithm that performs product sampling, i.e., samples proportional to the product of the bidirectional scattering distribution function (BSDF) and incoming radiance. We use a spatial-directional subdivision to represent incoming radiance, and introduce the use of Linearly Transformed Cosines (LTCs) to represent the BSDF during path guiding, thus enabling efficient product sampling. Despite the computational efficiency of LTCs, several optimizations are needed to make our method cost effective. In particular, we show how we can use vectorization, precomputation, as well as strategies to optimize multiple importance sampling and Russian roulette to improve performance. We evaluate our method on several scenes, demonstrating consistent improvement in efficiency compared to previous work, especially in scenes with significant glossy inter-reflection.
  • Thumbnail Image
    Item
    Deep Kernel Density Estimation for Photon Mapping
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhu, Shilin; Xu, Zexiang; Jensen, Henrik Wann; Su, Hao; Ramamoorthi, Ravi; Dachsbacher, Carsten and Pharr, Matt
    Recently, deep learning-based denoising approaches have led to dramatic improvements in low sample-count Monte Carlo rendering. These approaches are aimed at path tracing, which is not ideal for simulating challenging light transport effects like caustics, where photon mapping is the method of choice. However, photon mapping requires very large numbers of traced photons to achieve high-quality reconstructions. In this paper, we develop the first deep learning-based method for particlebased rendering, and specifically focus on photon density estimation, the core of all particle-based methods. We train a novel deep neural network to predict a kernel function to aggregate photon contributions at shading points. Our network encodes individual photons into per-photon features, aggregates them in the neighborhood of a shading point to construct a photon local context vector, and infers a kernel function from the per-photon and photon local context features. This network is easy to incorporate in many previous photon mapping methods (by simply swapping the kernel density estimator) and can produce high-quality reconstructions of complex global illumination effects like caustics with an order of magnitude fewer photons compared to previous photon mapping methods. Our approach largely reduces the required number of photons, significantly advancing the computational efficiency in photon mapping.
  • Thumbnail Image
    Item
    Adaptive Matrix Completion for Fast Visibility Computations with Many Lights Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Wang, Sunrise; Holzschuch, Nicolas; Dachsbacher, Carsten and Pharr, Matt
    Several fast global illumination algorithms rely on the Virtual Point Lights framework. This framework separates illumination into two steps: first, propagate radiance in the scene and store it in virtual lights, then gather illumination from these virtual lights. To accelerate the second step, virtual lights and receiving points are grouped hierarchically, for example using Multi- Dimensional Lightcuts. Computing visibility between clusters of virtual lights and receiving points is a bottleneck. Separately, matrix completion algorithms reconstruct completely a low-rank matrix from an incomplete set of sampled elements. In this paper, we use adaptive matrix completion to approximate visibility information after an initial clustering step. We reconstruct visibility information using as little as 10%to 20%samples for most scenes, and combine it with shading information computed separately, in parallel on the GPU. Overall, our method computes global illumination 3 or more times faster than previous stateof- the-art methods.
  • Thumbnail Image
    Item
    An Adaptive BRDF Fitting Metric
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Bieron, James; Peers, Pieter; Dachsbacher, Carsten and Pharr, Matt
    We propose a novel image-driven fitting strategy for isotropic BRDFs. Whereas existing BRDF fitting methods minimize a cost function directly on the error between the fitted analytical BRDF and the measured isotropic BRDF samples, we also take into account the resulting material appearance in visualizations of the BRDF. This change of fitting paradigm improves the appearance reproduction fidelity, especially for analytical BRDF models that lack the expressiveness to reproduce the measured surface reflectance. We formulate BRDF fitting as a two-stage process that first generates a series of candidate BRDF fits based only on the BRDF error with measured BRDF samples. Next, from these candidates, we select the BRDF fit that minimizes the visual error. We demonstrate qualitatively and quantitatively improved fits for the Cook-Torrance and GGX microfacet BRDF models. Furthermore, we present an analysis of the BRDF fitting results, and show that the image-driven isotropic BRDF fits generalize well to other light conditions, and that depending on the measured material, a different weighting of errors with respect to the measured BRDF is necessary.
  • Thumbnail Image
    Item
    Practical Measurement and Reconstruction of Spectral Skin Reflectance
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Gitlina, Yuliya; Guarnera, Giuseppe Claudio; Dhillon, Daljit Singh; Hansen, Jan; Lattas, Alexandros; Pai, Dinesh; Ghosh, Abhijeet; Dachsbacher, Carsten and Pharr, Matt
    We present two practical methods for measurement of spectral skin reflectance suited for live subjects, and drive a spectral BSSRDF model with appropriate complexity to match skin appearance in photographs, including human faces. Our primary measurement method employs illuminating a subject with two complementary uniform spectral illumination conditions using a multispectral LED sphere to estimate spatially varying parameters of chromophore concentrations including melanin and hemoglobin concentration, melanin blend-type fraction, and epidermal hemoglobin fraction. We demonstrate that our proposed complementary measurements enable higher-quality estimate of chromophores than those obtained using standard broadband illumination, while being suitable for integration with multiview facial capture using regular color cameras. Besides novel optimal measurements under controlled illumination, we also demonstrate how to adapt practical skin patch measurements using a hand-held dermatological skin measurement device, a Miravex Antera 3D camera, for skin appearance reconstruction and rendering. Furthermore, we introduce a novel approach for parameter estimation given the measurements using neural networks which is significantly faster than a lookup table search and avoids parameter quantization. We demonstrate high quality matches of skin appearance with photographs for a variety of skin types with our proposed practical measurement procedures, including photorealistic spectral reproduction and renderings of facial appearance.
  • Thumbnail Image
    Item
    Guided Fine-Tuning for Large-Scale Material Transfer
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Deschaintre, Valentin; Drettakis, George; Bousseau, Adrien; Dachsbacher, Carsten and Pharr, Matt
    We present a method to transfer the appearance of one or a few exemplar SVBRDFs to a target image representing similar materials. Our solution is extremely simple: we fine-tune a deep appearance-capture network on the provided exemplars, such that it learns to extract similar SVBRDF values from the target image. We introduce two novel material capture and design workflows that demonstrate the strength of this simple approach. Our first workflow allows to produce plausible SVBRDFs of large-scale objects from only a few pictures. Specifically, users only need take a single picture of a large surface and a few close-up flash pictures of some of its details.We use existing methods to extract SVBRDF parameters from the close-ups, and our method to transfer these parameters to the entire surface, enabling the lightweight capture of surfaces several meters wide such as murals, floors and furniture. In our second workflow, we provide a powerful way for users to create large SVBRDFs from internet pictures by transferring the appearance of existing, pre-designed SVBRDFs. By selecting different exemplars, users can control the materials assigned to the target image, greatly enhancing the creative possibilities offered by deep appearance capture.
  • Thumbnail Image
    Item
    Can't Invert the CDF? The Triangle-Cut Parameterization of the Region under the Curve
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Heitz, Eric; Dachsbacher, Carsten and Pharr, Matt
    We present an exact, analytic and deterministic method for sampling densities whose Cumulative Distribution Functions (CDFs) cannot be inverted analytically. Indeed, the inverse-CDF method is often considered the way to go for sampling non-uniform densities. If the CDF is not analytically invertible, the typical fallback solutions are either approximate, numerical, or nondeterministic such as acceptance-rejection. To overcome this problem, we show how to compute an analytic area-preserving parameterization of the region under the curve of the target density. We use it to generate random points uniformly distributed under the curve of the target density and their abscissae are thus distributed with the target density. Technically, our idea is to use an approximate analytic parameterization whose error can be represented geometrically as a triangle that is simple to cut out. This triangle-cut parameterization yields exact and analytic solutions to sampling problems that were presumably not analytically resolvable.
  • Thumbnail Image
    Item
    Photorealistic Material Editing Through Direct Image Manipulation
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Zsolnai-Fehér, Károly; Wonka, Peter; Wimmer, Michael; Dachsbacher, Carsten and Pharr, Matt
    Creating photorealistic materials for light transport algorithms requires carefully fine-tuning a set of material properties to achieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge. In this work, we present a technique that aims to empower novice and intermediate-level users to synthesize high-quality photorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our method combines the advantages of a neural network-augmented optimizer and an encoder neural network to produce high-quality output results within 30 seconds. We also demonstrate that it is resilient against poorly-edited target images and propose a simple extension to predict image sequences with a strict time budget of 1-2 seconds per image.
  • Thumbnail Image
    Item
    A Comprehensive Theory and Variational Framework for Anti-aliasing Sampling Patterns
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Öztireli, A. Cengiz; Dachsbacher, Carsten and Pharr, Matt
    In this paper, we provide a comprehensive theory of anti-aliasing sampling patterns that explains and revises known results, and introduce a variational optimization framework to generate point patterns with any desired power spectra and anti-aliasing properties. We start by deriving the exact spectral expression for expected error in reconstructing a function in terms of power spectra of sampling patterns, and analyzing how the shape of power spectra is related to anti-aliasing properties. Based on this analysis, we then formulate the problem of generating anti-aliasing sampling patterns as constrained variational optimization on power spectra. This allows us to not rely on any parametric form, and thus explore the whole space of realizable spectra. We show that the resulting optimized sampling patterns lead to reconstructions with less visible aliasing artifacts, while keeping low frequencies as clean as possible. Although we focus on image plane sampling, our theory and algorithms apply in any dimensions, and the variational optimization framework can be utilized in all problems where point pattern characteristics are given or optimized.
  • Thumbnail Image
    Item
    Practical Product Sampling by Fitting and Composing Warps
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Hart, David; Pharr, Matt; Müller, Thomas; Lopes, Ward; McGuire, Morgan; Shirley, Peter; Dachsbacher, Carsten and Pharr, Matt
    We introduce a Monte Carlo importance sampling method for integrands composed of products and show its application to rendering where direct sampling of the product is often difficult. Our method is based on warp functions that operate on the primary samples in [0;1)^n, where each warp approximates sampling a single factor of the product distribution. Our key insight is that individual factors are often well-behaved and inexpensive to fit and sample in primary sample space, which leads to a practical, efficient sampling algorithm. Our sampling approach is unbiased, easy to implement, and compatible with multiple importance sampling. We show the results of applying our warps to projected solid angle sampling of spherical triangles, to sampling bilinear patch light sources, and to sampling glossy BSDFs and area light sources, with efficiency improvements of over 1.6 x on real-world scenes.
  • Thumbnail Image
    Item
    Semi-Procedural Textures Using Point Process Texture Basis Functions
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Guehl, Pascal; Allègre, Remi; Dischler, Jean-Michel; Benes, Bedrich; Galin, Eric; Dachsbacher, Carsten and Pharr, Matt
    We introduce a novel semi-procedural approach that avoids drawbacks of procedural textures and leverages advantages of datadriven texture synthesis. We split synthesis in two parts: 1) structure synthesis, based on a procedural parametric model and 2) color details synthesis, being data-driven. The procedural model consists of a generic Point Process Texture Basis Function (PPTBF), which extends sparse convolution noises by defining rich convolution kernels. They consist of a window function multiplied with a correlated statistical mixture of Gabor functions, both designed to encapsulate a large span of common spatial stochastic structures, including cells, cracks, grains, scratches, spots, stains, and waves. Parameters can be prescribed automatically by supplying binary structure exemplars. As for noise-based Gaussian textures, the PPTBF is used as stand-alone function, avoiding classification tasks that occur when handling multiple procedural assets. Because the PPTBF is based on a single set of parameters it allows for continuous transitions between different visual structures and an easy control over its visual characteristics. Color is consistently synthesized from the exemplar using a multiscale parallel texture synthesis by numbers, constrained by the PPTBF. The generated textures are parametric, infinite and avoid repetition. The data-driven part is automatic and guarantees strong visual resemblance with inputs.
  • Thumbnail Image
    Item
    High-Resolution Neural Face Swapping for Visual Effects
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Naruniec, Jacek; Helminger, Leonhard; Schroers, Christopher; Weber, Romann M.; Dachsbacher, Carsten and Pharr, Matt
    In this paper, we propose an algorithm for fully automatic neural face swapping in images and videos. To the best of our knowledge, this is the first method capable of rendering photo-realistic and temporally coherent results at megapixel resolution. To this end, we introduce a progressively trained multi-way comb network and a light- and contrast-preserving blending method. We also show that while progressive training enables generation of high-resolution images, extending the architecture and training data beyond two people allows us to achieve higher fidelity in generated expressions. When compositing the generated expression onto the target face, we show how to adapt the blending strategy to preserve contrast and low-frequency lighting. Finally, we incorporate a refinement strategy into the face landmark stabilization algorithm to achieve temporal stability, which is crucial for working with high-resolution videos. We conduct an extensive ablation study to show the influence of our design choices on the quality of the swap and compare our work with popular state-of-the-art methods.