Volume 41 (2022)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/2633151

Browse

Now showing 1 - 20 of 267

Progressive Denoising of Monte Carlo Rendered Images
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Firmino, Arthur; Frisvad, Jeppe Revall; Jensen, Henrik Wann; Chaine, Raphaëlle; Kim, Min H.
Image denoising based on deep learning has become a powerful tool to accelerate Monte Carlo rendering. Deep learning techniques can produce smooth images using a low sample count. Unfortunately, existing deep learning methods are biased and do not converge to the correct solution as the number of samples increase. In this paper, we propose a progressive denoising technique that aims to use denoising only when it is beneficial and to reduce its impact at high sample counts. We use Stein's unbiased risk estimate (SURE) to estimate the error in the denoised image, and we combine this with a neural network to infer a per-pixel mixing parameter. We further augment this network with confidence intervals based on classical statistics to ensure consistency and convergence of the final denoised image. Our results demonstrate that our method is consistent and that it improves existing denoising techniques. Furthermore, it can be used in combination with existing high quality denoisers to ensure consistency. In addition to being asymptotically unbiased, progressive denoising is particularly good at preserving fine details that would otherwise be lost with existing denoisers.
Fabricable Multi-Scale Wang Tiles
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Liu, Xiaokang; Li, Chenran; Lu, Lin; Deussen, Oliver; Tu, Changhe; Campen, Marcel; Spagnuolo, Michela
Wang tiles, also known as Wang dominoes, are a jigsaw puzzle system with matching edges. Due to their compactness and expressiveness in representing variations, they have become a popular tool in the procedural synthesis of textures, height fields, 3D printing and representing other large and non-repetitive data. Multi-scale tiles created from low-level tiles allow for a higher tiling efficiency, although they face the problem of combinatorial explosion. In this paper, we propose a generation method for multi-scale Wang tiles that aims at minimizing the amount of needed tiles while still resembling a tiling appearance similar to low-level tiles. Based on a set of representative multi-scale Wang tiles, we use a dynamic generation algorithm for this purpose. Our method can be used for rapid texture synthesis and image halftoning. Respecting physical constraints, our tiles are connected, lightweight, independent of the fabrication scale, able to tile larger areas with image contents and contribute to "mass customization".
Facial Animation with Disentangled Identity and Motion using Transformers
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chandran, Prashanth; Zoss, Gaspard; Gross, Markus; Gotardo, Paulo; Bradley, Derek; Dominik L. Michels; Soeren Pirk
We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non-rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer-based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame-to-frame correlations required to represent the motion manifold, via the internal self-attention mechanism. Furthermore, our method disentangles the constant facial identity from the time-varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity-agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity-modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key-frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling.
Local Offset Point Cloud Transformer Based Implicit Surface Reconstruction
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Yang, Yan Xin; Zhang, San Guo; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
Implicit neural representations, such as MLP, can well recover the topology of watertight object. However, MLP fails to recover geometric details of watertight object and complicated topology due to dealing with point cloud in a point-wise manner. In this paper, we propose a point cloud transformer called local offset point cloud transformer (LOPCT) as a feature fusion module. Before using MLP to learn the implicit function, the input point cloud is first fed into the local offset transformer, which adaptively learns the dependency of the local point cloud and obtains the enhanced features of each point. The feature-enhanced point cloud is then fed into the MLP to recover the geometric details and sharp features of watertight object and complex topology. Extensive reconstruction experiments of watertight object and complex topology demonstrate that our method achieves comparable or better results than others in terms of recovering sharp features and geometric details. In addition, experiments on watertight objects demonstrate the robustness of our method in terms of average result.
Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Users
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Ye, Yongjing; Liu, Libin; Hu, Lei; Xia, Shihong; Dominik L. Michels; Soeren Pirk
Animating an avatar that reflects a user's action in the VR world enables natural interactions with the virtual environment. It has the potential to allow remote users to communicate and collaborate in a way as if they met in person. However, a typical VR system provides only a very sparse set of up to three positional sensors, including a head-mounted display (HMD) and optionally two hand-held controllers, making the estimation of the user's full-body movement a difficult problem. In this work, we present a data-driven physics-based method for predicting the realistic full-body movement of the user according to the transformations of these VR trackers and simulating an avatar character to mimic such user actions in the virtual world in realtime. We train our system using reinforcement learning with carefully designed pretraining processes to ensure the success of the training and the quality of the simulation. We demonstrate the effectiveness of the method with an extensive set of examples.
Coverage Axis: Inner Point Selection for 3D Shape Skeletonization
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Dou, Zhiyang; Lin, Cheng; Xu, Rui; Yang, Lei; Xin, Shiqing; Komura, Taku; Wang, Wenping; Chaine, Raphaëlle; Kim, Min H.
In this paper, we present a simple yet effective formulation called Coverage Axis for 3D shape skeletonization. Inspired by the set cover problem, our key idea is to cover all the surface points using as few inside medial balls as possible. This formulation inherently induces a compact and expressive approximation of the Medial Axis Transform (MAT) of a given shape. Different from previous methods that rely on local approximation error, our method allows a global consideration of the overall shape structure, leading to an efficient high-level abstraction and superior robustness to noise. Another appealing aspect of our method is its capability to handle more generalized input such as point clouds and poor-quality meshes. Extensive comparisons and evaluations demonstrate the remarkable effectiveness of our method for generating compact and expressive skeletal representation to approximate the MAT.
WTFM Layer: An Effective Map Extractor for Unsupervised Shape Correspondence
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Liu, Shengjun; Xu, Haojun; Yan, Dong-Ming; Hu, Ling; Liu, Xinru; Li, Qinsong; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
We propose a novel unsupervised learning approach for computing correspondences between non-rigid 3D shapes. The core idea is that we integrate a novel structural constraint into the deep functional map pipeline, a recently dominant learning framework for shape correspondence, via a powerful spectral manifold wavelet transform (SMWT). As SMWT is isometrically invariant operator and can analyze features from multiple frequency bands, we use the multiscale SMWT results of the learned features as function preservation constraints to optimize the functional map by assuming each frequency-band information of the descriptors should be correspondingly preserved by the functional map. Such a strategy allows extracting significantly more deep feature information than existing approaches which only use the learned descriptors to estimate the functional map. And our formula strongly ensure the isometric properties of the underlying map. We also prove that our computation of the functional map amounts to filtering processes only referring to matrix multiplication. Then, we leverage the alignment errors of intrinsic embedding between shapes as a loss function and solve it in an unsupervised way using the Sinkhorn algorithm. Finally, we utilize DiffusionNet as a feature extractor to ensure that discretization-resistant and directional shape features are produced. Experiments on multiple challenging datasets prove that our method can achieve state-of-the-art correspondence quality. Furthermore, our method yields significant improvements in robustness to shape discretization and generalization across the different datasets. The source code and trained models will be available at https://github.com/HJ-Xu/ WTFM-Layer.
A Second Order Cone Programming Approach for Simulating Biphasic Materials
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Tang, Pengbin; Coros, Stelian; Thomaszewski, Bernhard; Dominik L. Michels; Soeren Pirk
Strain limiting is a widely used approach for simulating biphasic materials such as woven textiles and biological tissue that exhibit a soft elastic regime followed by a hard deformation limit. However, existing methods are either based on slowly converging local iterations, or offer no guarantees on convergence. In this work, we propose a new approach to strain limiting based on second order cone programming (SOCP). Our work is based on the key insight that upper bounds on per-triangle deformations lead to convex quadratic inequality constraints. Though nonlinear, these constraints can be reformulated as inclusion conditions on convex sets, leading to a second order cone programming problem-a convex optimization problem that a) is guaranteed to have a unique solution and b) allows us to leverage efficient conic programming solvers. We first cast strain limiting with anisotropic bounds on stretching as a quadratically constrained quadratic program (QCQP), then show how this QCQP can be mapped to a second order cone programming problem. We further propose a constraint reflection scheme and empirically show that it exhibits superior energy-preservation properties compared to conventional end-of-step projection methods. Finally, we demonstrate our prototype implementation on a set of examples and illustrate how different deformation limits can be used to model a wide range of material behaviors.
N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Li, Yu Di; Tang, Min; Yang, Yun; Huang, Zi; Tong, Ruo Feng; Yang, Shuang Cai; Li, Yao; Manocha, Dinesh; Chaine, Raphaëlle; Kim, Min H.
We present a novel mesh-based learning approach (N-Cloth) for plausible 3D cloth deformation prediction. Our approach is general and can handle cloth or obstacles represented by triangle meshes with arbitrary topologies.We use graph convolution to transform the cloth and object meshes into a latent space to reduce the non-linearity in the mesh space. Our network can predict the target 3D cloth mesh deformation based on the initial state of the cloth mesh template and the target obstacle mesh. Our approach can handle complex cloth meshes with up to 100K triangles and scenes with various objects corresponding to SMPL humans, non-SMPL humans or rigid bodies. In practice, our approach can be used to generate plausible cloth simulation at 30??45 fps on an NVIDIA GeForce RTX 3090 GPU. We highlight its benefits over prior learning-based methods and physicallybased cloth simulators.
Pixel Art Adaptation for Handicraft Fabrication
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Igarashi, Yuki; Igarashi, Takeo; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
Knitting and weaving patterns can be visually represented as pixel art. With hand knitting and weaving, human error (shifting, duplicating, or skipping pixels) can occur during manual fabrication. It is too costly to change already-fabricated pixels, so experts often adapt pixels that have not yet been fabricated to make the errors less visible. This paper proposes an automatic adaptation process to minimize visual artifacts. The system presents multiple adaptation possibilities to the user, who can choose the proposed adaptation or untie and re-fabricate their work. In typical handicraft fabrication, the design is complete before the start of fabrication and remains fixed during fabrication. Our system keeps updating the design during fabrication to tolerate human errors in the process. We implemented the proposed algorithm in a system that visualizes the knitting pattern, cross-stitching and bead weaving processes.
Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Wang, Ziyu; Deng, Yu; Yang, Jiaolong; Yu, Jingyi; Tong, Xin; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D-aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high-quality texture and shape editing results.
Sketching Vocabulary for Crowd Motion
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Mathew, C. D. Tharindu; Benes, Bedrich; Aliaga, Daniel; Dominik L. Michels; Soeren Pirk
This paper proposes and evaluates a sketching language to author crowd motion. It focuses on the path, speed, thickness, and density parameters of crowd motion. A sketch-based vocabulary is proposed for each parameter and evaluated in a user study against complex crowd scenes. A sketch recognition pipeline converts the sketches into a crowd simulation. The user study results show that 1) participants at various skill levels and can draw accurate crowd motion through sketching, 2) certain sketch styles lead to a more accurate representation of crowd parameters, and 3) sketching allows to produce complex crowd motions in a few seconds. The results show that some styles although accurate actually are less preferred over less accurate ones.
NSTO: Neural Synthesizing Topology Optimization for Modulated Structure Generation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Zhong, Shengze; Punpongsanon, Parinya; Iwai, Daisuke; Sato, Kosuke; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
Nature evolves structures like honeycombs at optimized performance with limited material. These efficient structures can be artificially created with the collaboration of structural topology optimization and additive manufacturing. However, the extensive computation cost of topology optimization causes low mesh resolution, long solving time, and rough boundaries that fail to match the requirements for meeting the growing personal fabrication demands and printing capability. Therefore, we propose the neural synthesizing topology optimization that leverages a self-supervised coordinate-based network to optimize structures with significantly shorter computation time, where the network encodes the structural material layout as an implicit function of coordinates. Continuous solution space is further generated from optimization tasks under varying boundary conditions or constraints for users' instant inference of novel solutions. We demonstrate the system's efficacy for a broad usage scenario through numerical experiments and 3D printing.
Automatic Differentiable Procedural Modeling
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Gaillard, Mathieu; Krs, Vojtech; Gori, Giorgio; Mech, Radomír; Benes, Bedrich; Chaine, Raphaëlle; Kim, Min H.
Procedural modeling allows for an automatic generation of large amounts of similar assets, but there is limited control over the generated output. We address this problem by introducing Automatic Differentiable Procedural Modeling (ADPM). The forward procedural model generates a final editable model. The user modifies the output interactively, and the modifications are transferred back to the procedural model as its parameters by solving an inverse procedural modeling problem. We present an auto-differentiable representation of the procedural model that significantly accelerates optimization. In ADPM the procedural model is always available, all changes are non-destructive, and the user can interactively model the 3D object while keeping the procedural representation. ADPM provides the user with precise control over the resulting model comparable to non-procedural interactive modeling. ADPM is node-based, and it generates hierarchical 3D scene geometry converted to a differentiable computational graph. Our formulation focuses on the differentiability of high-level primitives and bounding volumes of components of the procedural model rather than the detailed mesh geometry. Although this high-level formulation limits the expressiveness of user edits, it allows for efficient derivative computation and enables interactivity. We designed a new optimizer to solve for inverse procedural modeling. It can detect that an edit is under-determined and has degrees of freedom. Leveraging cheap derivative evaluation, it can explore the region of optimality of edits and suggest various configurations, all of which achieve the requested edit differently. We show our system's efficiency on several examples, and we validate it by a user study.
EUROGRAPHICS 2022: CGF 41-2 Frontmatter
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chaine, Raphaëlle; Kim, Min H.; Chaine, Raphaëlle; Kim, Min H.
Efficient Texture Parameterization Driven by Perceptual-Loss-on-Screen
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Sun, Haoran; Wang, Shiyi; Wu, Wenhai; Jin, Yao; Bao, Hujun; Huang, Jin; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
Texture mapping is a ubiquitous technique to enrich the visual effect of a mesh, which represents the desired signal (e.g. diffuse color) on the mesh to a texture image discretized by pixels through a bijective parameterization. To achieve high visual quality, large number of pixels are generally required, which brings big burden in storage, memory and transmission. We propose to use a perceptual model and a rendering procedure to measure the loss coming from the discretization, then optimize a parameterization to improve the efficiency, i.e. using fewer pixels under a comparable perceptual loss. The general perceptual model and rendering procedure can be very complicated, and non-isotropic property rooted in the square shape of pixels make the problem more difficult to solve. We adopt a two-stage strategy and use the Bayesian optimization in the triangle-wise stage. With our carefully designed weighting scheme, the mesh-wise optimization can take the triangle-wise perceptual loss into consideration under a global conforming requirement. Comparing with many parameterizations manually designed, driven by interpolation error, or driven by isotropic energy, ours can use significantly fewer pixels with comparable perception loss or vise vesa.
Color-mapped Noise Vector Fields for Generating Procedural Micro-patterns
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Grenier, Charline; Sauvage, Basile; Dischler, Jean-Michel; Thery, Sylvain; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
Stochastic micro-patterns successfully enhance the realism of virtual scenes. Procedural models using noise combined with transfer functions are extremely efficient. However, most patterns produced today employ 1D transfer functions, which assign color, transparency, or other material attributes, based solely on the single scalar quantity of noise. Multi-dimensional transfer functions have received widespread attention in other fields, such as scientific volume rendering. But their potential has not yet been well explored for modeling micro-patterns in the field of procedural texturing. We propose a new procedural model for stochastic patterns, defined as the composition of a bi-dimensional transfer function (a.k.a. color-map) with a stochastic vector field. Our model is versatile, as it encompasses several existing procedural noises, including Gaussian noise and phasor noise. It also generates a much larger gamut of patterns, including locally structured patterns which are notoriously difficult to reproduce. We leverage the Gaussian assumption and a tiling and blending algorithm to provide real-time generation and filtering. A key contribution is a real-time approximation of the second order statistics over an arbitrary pixel footprint, which enables, in addition, the filtering of procedural normal maps. We exhibit a wide variety of results, including Gaussian patterns, profiled waves, concentric and non-concentric patterns.
Computational Design of Self-Actuated Surfaces by Printing Plastic Ribbons on Stretched Fabric
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Jourdan, David; Skouras, Mélina; Vouga, Etienne; Bousseau, Adrien; Chaine, Raphaëlle; Kim, Min H.
We introduce a new mechanism for self-actuating deployable structures, based on printing a dense pattern of closely-spaced plastic ribbons on sheets of pre-stretched elastic fabric. We leverage two shape-changing effects that occur when such an assembly is printed and allowed to relax: first, the incompressible plastic ribbons frustrate the contraction of the fabric back to its rest state, forcing residual strain in the fabric and creating intrinsic curvature. Second, the differential compression at the interface between the plastic and fabric layers yields a bilayer effect in the direction of the ribbons, making each ribbon buckle into an arc at equilibrium state and creating extrinsic curvature. We describe an inverse design tool to fabricate lowcost, lightweight prototypes of freeform surfaces using the controllable directional distortion and curvature offered by this mechanism. The core of our method is a parameterization algorithm that bounds surface distortions along and across principal curvature directions, along with a pattern synthesis algorithm that covers a surface with ribbons to match the target distortions and curvature given by the aforementioned parameterization. We demonstrate the flexibility and accuracy of our method by fabricating and measuring a variety of surfaces, including nearly-developable surfaces as well as surfaces with positive and negative mean curvature, which we achieve thanks to a simple hardware setup that allows printing on both sides of the fabric.
Pose Representations for Deep Skeletal Animation
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Andreou, Nefeli; Aristidou, Andreas; Chrysanthou, Yiorgos; Dominik L. Michels; Soeren Pirk
Data-driven skeletal animation relies on the existence of a suitable learning scheme, which can capture the rich context of motion. However, commonly used motion representations often fail to accurately encode the full articulation of motion, or present artifacts. In this work, we address the fundamental problem of finding a robust pose representation for motion, suitable for deep skeletal animation, one that can better constrain poses and faithfully capture nuances correlated with skeletal characteristics. Our representation is based on dual quaternions, the mathematical abstractions with well-defined operations, which simultaneously encode rotational and positional orientation, enabling a rich encoding, centered around the root. We demonstrate that our representation overcomes common motion artifacts, and assess its performance compared to other popular representations. We conduct an ablation study to evaluate the impact of various losses that can be incorporated during learning. Leveraging the fact that our representation implicitly encodes skeletal motion attributes, we train a network on a dataset comprising of skeletons with different proportions, without the need to retarget them first to a universal skeleton, which causes subtle motion elements to be missed. Qualitative results demonstrate the usefulness of the parameterization in skeleton-specific synthesis.
Learning 3D Shape Aesthetics Globally and Locally
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Chen, Minchan; Lau, Manfred; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
There exist previous works in computing the visual aesthetics of 3D shapes ''globally'', where the term global means that shape aesthetics data are collected for whole 3D shapes and then used to compute the aesthetics of whole 3D shapes. In this paper, we introduce a novel method that takes such ''global'' shape aesthetics data, and learn both a ''global'' shape aesthetics measure that computes aesthetics scores for whole 3D shapes, and a ''local'' shape aesthetics measure that computes to what extent a local region on the 3D shape surface contributes to the whole shape's aesthetics. These aesthetics measures are learned, and hence do not consider existing handcrafted notions of what makes a 3D shape aesthetic. We take a dataset of global pairwise shape aesthetics, where humans compares between pairs of shapes and say which shape from each pair is more aesthetic. Our solution proposes a point-based neural network that takes a 3D shape represented by surface patches as input and jointly outputs its global aesthetics score and a local aesthetics map. To build connections between global and local aesthetics, we embed the global and local features into the same latent space and then output scores with the weights-shared aesthetics predictors. Furthermore, we designed three loss functions to supervise the training jointly. We demonstrate the shape aesthetics results globally and locally to show that our framework can make good global aesthetics predictions while the predicted aesthetics maps are consistent with human perception. In addition, we present several applications enabled by our local aesthetics metric.

Browse

Browsing Volume 41 (2022) by Issue Date

Results Per Page

Sort Options