Volume 39 (2020)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/2632879

Browse

Now showing 1 - 20 of 297

Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Alexanderson, Simon; Henter, Gustav Eje; Kucherenko, Taras; Beskow, Jonas; Panozzo, Daniele and Assarsson, Ulf
Automatic synthesis of realistic gestures promises to transform the fields of animation, avatars and communicative agents. In off-line applications, novel tools can alter the role of an animator to that of a director, who provides only high-level input for the desired animation; a learned network then translates these instructions into an appropriate sequence of body poses. In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters. In this paper we address some of the core issues towards these ends. By adapting a deep learning-based motion synthesis method called MoGlow, we propose a new generative model for generating state-of-the-art realistic speech-driven gesticulation. Owing to the probabilistic nature of the approach, our model can produce a battery of different, yet plausible, gestures given the same input speech signal. Just like humans, this gives a rich natural variation of motion. We additionally demonstrate the ability to exert directorial control over the output style, such as gesture level, speed, symmetry and spacial extent. Such control can be leveraged to convey a desired character personality or mood. We achieve all this without any manual annotation of the data. User studies evaluating upper-body gesticulation confirm that the generated motions are natural and well match the input speech. Our method scores above all prior systems and baselines on these measures, and comes close to the ratings of the original recorded motions. We furthermore find that we can accurately control gesticulation styles without unnecessarily compromising perceived naturalness. Finally, we also demonstrate an application of the same method to full-body gesticulation, including the synthesis of stepping motion and stance.
Interactive Modeling of Cellular Structures on Surfaces with Application to Additive Manufacturing
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Stadlbauer, Pascal; Mlakar, Daniel; Seidel, Hans-Peter; Steinberger, Markus; Zayer, Rhaleb; Panozzo, Daniele and Assarsson, Ulf
The rich and evocative patterns of natural tessellations endow them with an unmistakable artistic appeal and structural properties which are echoed across design, production, and manufacturing. Unfortunately, interactive control of such patterns-as modeled by Voronoi diagrams, is limited to the simple two dimensional case and does not extend well to freeform surfaces. We present an approach for direct modeling and editing of such cellular structures on surface meshes. The overall modeling experience is driven by a set of editing primitives which are efficiently implemented on graphics hardware. We feature a novel application for 3D printing on modern support-free additive manufacturing platforms. Our method decomposes the input surface into a cellular skeletal structure which hosts a set of overlay shells. In this way, material saving can be channeled to the shells while structural stability is channeled to the skeleton. To accommodate the available printer build volume, the cellular structure can be further split into moderately sized parts. Together with shells, they can be conveniently packed to save on production time. The assembly of the printed parts is streamlined by a part numbering scheme which respects the geometric layout of the input model.
Image-Driven Furniture Style for Interactive 3D Scene Modeling
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Weiss, Tomer; Yildiz, Ilkay; Agarwal, Nitin; Ataer-Cansizoglu, Esra; Choi, Jae-Woo; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Creating realistic styled spaces is a complex task, which involves design know-how for what furniture pieces go well together. Interior style follows abstract rules involving color, geometry and other visual elements. Following such rules, users manually select similar-style items from large repositories of 3D furniture models, a process which is both laborious and time-consuming. We propose a method for fast-tracking style-similarity tasks, by learning a furniture's style-compatibility from interior scene images. Such images contain more style information than images depicting single furniture. To understand style, we train a deep learning network on a classification task. Based on image embeddings extracted from our network, we measure stylistic compatibility of furniture. We demonstrate our method with several 3D model style-compatibility results, and with an interactive system for modeling style-consistent scenes.
Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Wen, Quan; Bradley, Derek; Beeler, Thabo; Park, Seonwook; Hilliges, Otmar; Yong, Jun-Hai; Xu, Feng; Panozzo, Daniele and Assarsson, Ulf
3D gaze tracking from a single RGB camera is very challenging due to the lack of information in determining the accurate gaze target from a monocular RGB sequence. The eyes tend to occupy only a small portion of the video, and even small errors in estimated eye orientations can lead to very large errors in the triangulated gaze target. We overcome these difficulties with a novel lightweight eyeball calibration scheme that determines the user-specific visual axis, eyeball size and position in the head. Unlike the previous calibration techniques, we do not need the ground truth positions of the gaze points. In the online stage, gaze is tracked by a new gaze fitting algorithm, and refined by a 3D gaze regression method to correct for bias errors. Our regression is pre-trained on several individuals and works well for novel users. After the lightweight one-time user calibration, our method operates in real time. Experiments show that our technique achieves state-of-the-art accuracy in gaze angle estimation, and we demonstrate applications of 3D gaze target tracking and gaze retargeting to an animated 3D character.
Accelerating Liquid Simulation With an Improved Data‐Driven Method
(© 2020 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2020) Gao, Yang; Zhang, Quancheng; Li, Shuai; Hao, Aimin; Qin, Hong; Benes, Bedrich and Hauser, Helwig
In physics‐based liquid simulation for graphics applications, pressure projection consumes a significant amount of computational time and is frequently the bottleneck of the computational efficiency. How to rapidly apply the pressure projection and at the same time how to accurately capture the liquid geometry are always among the most popular topics in the current research trend in liquid simulations. In this paper, we incorporate an artificial neural network into the simulation pipeline for handling the tricky projection step for liquid animation. Compared with the previous neural‐network‐based works for gas flows, this paper advocates new advances in the composition of representative features as well as the loss functions in order to facilitate fluid simulation with free‐surface boundary. Specifically, we choose both the velocity and the level‐set function as the additional representation of the fluid states, which allows not only the motion but also the boundary position to be considered in the neural network solver. Meanwhile, we use the divergence error in the loss function to further emulate the lifelike behaviours of liquid. With these arrangements, our method could greatly accelerate the pressure projection step in liquid simulation, while maintaining fairly convincing visual results. Additionally, our neutral network performs well when being applied to new scene synthesis even with varied boundaries or scales.
VisuaLint: Sketchy In Situ Annotations of Chart Construction Errors
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Hopkins, Aspen K.; Correll, Michael; Satyanarayan, Arvind; Viola, Ivan and Gleicher, Michael and Landesberger von Antburg, Tatiana
Chart construction errors, such as truncated axes or inexpressive visual encodings, can hinder reading a visualization, or worse, imply misleading facts about the underlying data. These errors can be caught by critical readings of visualizations, but readers must have a high level of data and design literacy and must be paying close attention. To address this issue, we introduce VisuaLint: a technique for surfacing chart construction errors in situ. Inspired by the ubiquitous red wavy underline that indicates spelling mistakes, visualization elements that contain errors (e.g., axes and legends) are sketchily rendered and accompanied by a concise annotation. VisuaLint is unobtrusive-it does not interfere with reading a visualization-and its direct display establishes a close mapping between erroneous elements and the expression of error. We demonstrate five examples of VisualLint and present the results of a crowdsourced evaluation (N = 62) of its efficacy. These results contribute an empirical baseline proficiency for recognizing chart construction errors, and indicate near-universal difficulty in error identification. We find that people more reliably identify chart construction errors after being shown examples of VisuaLint, and prefer more verbose explanations for unfamiliar or less obvious flaws.
Statistics-based Motion Synthesis for Social Conversations
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Yang, Yanzhe; Yang, Jimei; Hodgins, Jessica; Bender, Jan and Popa, Tiberiu
Plausible conversations among characters are required to generate the ambiance of social settings such as a restaurant, hotel lobby, or cocktail party. In this paper, we propose a motion synthesis technique that can rapidly generate animated motion for characters engaged in two-party conversations. Our system synthesizes gestures and other body motions for dyadic conversations that synchronize with novel input audio clips. Human conversations feature many different forms of coordination and synchronization. For example, speakers use hand gestures to emphasize important points, and listeners often nod in agreement or acknowledgment. To achieve the desired degree of realism, our method first constructs a motion graph that preserves the statistics of a database of recorded conversations performed by a pair of actors. This graph is then used to search for a motion sequence that respects three forms of audio-motion coordination in human conversations: coordination to phonemic clause, listener response, and partner's hesitation pause. We assess the quality of the generated animations through a user study that compares them to the originally recorded motion and evaluate the effects of each type of audio-motion coordination via ablation studies.
A Laplacian for Nonmanifold Triangle Meshes
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Sharp, Nicholas; Crane, Keenan; Jacobson, Alec and Huang, Qixing
We describe a discrete Laplacian suitable for any triangle mesh, including those that are nonmanifold or nonorientable (with or without boundary). Our Laplacian is a robust drop-in replacement for the usual cotan matrix, and is guaranteed to have nonnegative edge weights on both interior and boundary edges, even for extremely poor-quality meshes. The key idea is to build what we call a ''tufted cover'' over the input domain, which has nonmanifold vertices but manifold edges. Since all edges are manifold, we can flip to an intrinsic Delaunay triangulation; our Laplacian is then the cotan Laplacian of this new triangulation. This construction also provides a high-quality point cloud Laplacian, via a nonmanifold triangulation of the point set. We validate our Laplacian on a variety of challenging examples (including all models from Thingi10k), and a variety of standard tasks including geodesic distance computation, surface deformation, parameterization, and computing minimal surfaces.
Generating High-quality Superpixels in Textured Images
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Zhang, Zhe; Xu, Panpan; Chang, Jian; Wang, Wencheng; Zhao, Chong; Zhang, Jian Jun; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Superpixel segmentation is important for promoting various image processing tasks. However, existing methods still have difficulties in generating high-quality superpixels in textured images, because they cannot separate textures from structures well. Though texture filtering can be adopted for smoothing textures before superpixel segmentation, the filtering would also smooth the object boundaries, and thus weaken the quality of generated superpixels. In this paper, we propose to use the adaptive scale box smoothing instead of the texture filtering to obtain more high-quality texture and boundary information. Based on this, we design a novel distance metric to measure the distance between different pixels, which considers boundary, color and Euclidean distance simultaneously. As a result, our method can achieve high-quality superpixel segmentation in textured images without texture filtering. The experimental results demonstrate the superiority of our method over existing methods, even the learning-based methods. Benefited from using boundaries to guide superpixel segmentation, our method can also suppress noise to generate high-quality superpixels in non-textured images.
Feature Driven Combination of Animated Vector Field Visualizations
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Lobo, María Jesús; Telea, Alexandru; Hurter, Christophe; Viola, Ivan and Gleicher, Michael and Landesberger von Antburg, Tatiana
Animated visualizations are one of the methods for finding and understanding complex structures of time-dependent vector fields. Many visualization designs can be used to this end, such as streamlines, vector glyphs, and image-based techniques. While all such designs can depict any vector field, their effectiveness in highlighting particular field aspects has not been fully explored. To fill this gap, we compare three animated vector field visualization techniques, OLIC, IBFV, and particles, for a critical point detection-and-classification task through a user study. Our results show that the effectiveness of the studied techniques depends on the nature of the critical points. We use these results to design a new flow visualization technique that combines all studied techniques in a single view by locally using the most effective technique for the patterns present in the flow data at that location. A second user study shows that our technique is more efficient and less error prone than the three other techniques used individually for the critical point detection task.
v-plots: Designing Hybrid Charts for the Comparative Analysis of Data Distributions
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Blumenschein, Michael; Debbeler, Luka J.; Lages, Nadine C.; Renner, Britta; Keim, Daniel A.; El-Assady, Mennatallah; Viola, Ivan and Gleicher, Michael and Landesberger von Antburg, Tatiana
Comparing data distributions is a core focus in descriptive statistics, and part of most data analysis processes across disciplines. In particular, comparing distributions entails numerous tasks, ranging from identifying global distribution properties, comparing aggregated statistics (e.g., mean values), to the local inspection of single cases. While various specialized visualizations have been proposed (e.g., box plots, histograms, or violin plots), they are not usually designed to support more than a few tasks, unless they are combined. In this paper, we present the v-plot designer; a technique for authoring custom hybrid charts, combining mirrored bar charts, difference encodings, and violin-style plots. v-plots are customizable and enable the simultaneous comparison of data distributions on global, local, and aggregation levels. Our system design is grounded in an expert survey that compares and evaluates 20 common visualization techniques to derive guidelines for the task-driven selection of appropriate visualizations. This knowledge externalization step allowed us to develop a guiding wizard that can tailor v-plots to individual tasks and particular distribution properties. Finally, we confirm the usefulness of our system design and the userguiding process by measuring the fitness for purpose and applicability in a second study with four domain and statistic experts.
A Pixel-Based Framework for Data-Driven Clothing
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Jin, Ning; Zhu, Yilin; Geng, Zhenglin; Fedkiw, Ron; Bender, Jan and Popa, Tiberiu
We propose a novel approach to learning cloth deformation as a function of body pose, recasting the graph-like triangle mesh data structure into image-based data in order to leverage popular and well-developed convolutional neural networks (CNNs) in a two-dimensional Euclidean domain. Then, a three-dimensional animation of clothing is equivalent to a sequence of twodimensional RGB images driven/choreographed by time dependent joint angles. In order to reduce nonlinearity demands on the neural network, we utilize procedural skinning of the body surface to capture much of the rotation/deformation so that the RGB images only contain textures of displacement offsets from skin to clothing. Notably, we illustrate that our approach does not require accurate unclothed body shapes or robust skinning techniques. Additionally, we discuss how standard image based techniques such as image partitioning for higher resolution can readily be incorporated into our framework.
PointSkelCNN: Deep Learning-Based 3D Human Skeleton Extraction from Point Clouds
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Qin, Hongxing; Zhang, Songshan; Liu, Qihuang; Chen, Li; Chen, Baoquan; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
A 3D human skeleton plays important roles in human shape reconstruction and human animation. Remarkable advances have been achieved recently in 3D human skeleton estimation from color and depth images via a powerful deep convolutional neural network. However, applying deep learning frameworks to 3D human skeleton extraction from point clouds remains challenging because of the sparsity of point clouds and the high nonlinearity of human skeleton regression. In this study, we develop a deep learning-based approach for 3D human skeleton extraction from point clouds. We convert 3D human skeleton extraction into offset vector regression and human body segmentation via deep learning-based point cloud contraction. Furthermore, a disambiguation strategy is adopted to improve the robustness of joint points regression. Experiments on the public human pose dataset UBC3V and the human point cloud skeleton dataset 3DHumanSkeleton compiled by the authors show that the proposed approach outperforms the state-of-the-art methods.
Generating Adversarial Surfaces via Band-Limited Perturbations
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Mariani, Giorgio; Cosmo, Luca; Bronstein, Alex M.; Rodolà, Emanuele; Jacobson, Alec and Huang, Qixing
Adversarial attacks have demonstrated remarkable efficacy in altering the output of a learning model by applying a minimal perturbation to the input data. While increasing attention has been placed on the image domain, however, the study of adversarial perturbations for geometric data has been notably lagging behind. In this paper, we show that effective adversarial attacks can be concocted for surfaces embedded in 3D, under weak smoothness assumptions on the perceptibility of the attack. We address the case of deformable 3D shapes in particular, and introduce a general model that is not tailored to any specific surface representation, nor does it assume access to a parametric description of the 3D object. In this context, we consider targeted and untargeted variants of the attack, demonstrating compelling results in either case. We further show how discovering adversarial examples, and then using them for adversarial training, leads to an increase in both robustness and accuracy. Our findings are confirmed empirically over multiple datasets spanning different semantic classes and deformations.
ZerNet: Convolutional Neural Networks on Arbitrary Surfaces Via Zernike Local Tangent Space Estimation
(© 2020 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2020) Sun, Zhiyu; Rooke, Ethan; Charton, Jerome; He, Yusen; Lu, Jia; Baek, Stephen; Benes, Bedrich and Hauser, Helwig
In this paper, we propose a novel formulation extending convolutional neural networks (CNN) to arbitrary two‐dimensional manifolds using orthogonal basis functions called Zernike polynomials. In many areas, geometric features play a key role in understanding scientific trends and phenomena, where accurate numerical quantification of geometric features is critical. Recently, CNNs have demonstrated a substantial improvement in extracting and codifying geometric features. However, the progress is mostly centred around computer vision and its applications where an inherent grid‐like data representation is naturally present. In contrast, many geometry processing problems deal with curved surfaces and the application of CNNs is not trivial due to the lack of canonical grid‐like representation, the absence of globally consistent orientation and the incompatible local discretizations. In this paper, we show that the Zernike polynomials allow rigourous yet practical mathematical generalization of CNNs to arbitrary surfaces. We prove that the convolution of two functions can be represented as a simple dot product between Zernike coefficients and the rotation of a convolution kernel is essentially a set of 2 × 2 rotation matrices applied to the coefficients. The key contribution of this work is in such a computationally efficient but rigorous generalization of the major CNN building blocks.
Fast Out-of-Core Octree Generation for Massive Point Clouds
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Schütz, Markus; Ohrhallinger, Stefan; Wimmer, Michael; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
We propose an efficient out-of-core octree generation method for arbitrarily large point clouds. It utilizes a hierarchical counting sort to quickly split the point cloud into small chunks, which are then processed in parallel. Levels of detail are generated by subsampling the full data set bottom up using one of multiple exchangeable sampling strategies.We introduce a fast hierarchical approximate blue-noise strategy and compare it to a uniform random sampling strategy. The throughput, including out-of-core access to disk, generating the octree, and writing the final result to disk, is about an order of magnitude faster than the state of the art, and reaches up to around 6 million points per second for the blue-noise approach and up to around 9 million points per second for the uniform random approach on modern SSDs.
Visual Analytics in Dental Aesthetics
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Amirkhanov, Aleksandr; Bernhard, Matthias; Karimov, Alexey; Stiller, Sabine; Geier, Andreas; Gröller, Eduard; Mistelbauer, Gabriel; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
Dental healthcare increasingly employs computer-aided design software, to provide patients with high-quality dental prosthetic devices. In modern dental reconstruction, dental technicians address the unique anatomy of each patient individually, by capturing the dental impression and measuring the mandibular movements. Subsequently, dental technicians design a custom denture that fits the patient from a functional point of view. The current workflow does not include a systematic analysis of aesthetics, and dental technicians rely only on an aesthetically pleasing mock-up that they discuss with the patient, and on their experience. Therefore, the final denture aesthetics remain unknown until the dental technicians incorporate the denture into the patient. In this work, we present a solution that integrates aesthetics analysis into the functional workflow of dental technicians. Our solution uses a video recording of the patient, to preview the denture design at any stage of the denture design process. We present a teeth pose estimation technique that enables denture preview and a set of linked visualizations that support dental technicians in the aesthetic design of dentures. These visualizations assist dental technicians in choosing the most aesthetically fitting preset from a library of dentures, in identifying the suitable denture size, and in adjusting the denture position. We demonstrate the utility of our system with four use cases, explored by a dental technician. Also, we performed a quantitative evaluation for teeth pose estimation, and an informal usability evaluation, with positive outcomes concerning the integration of aesthetics analysis into the functional workflow.
Memory-Efficient Bijective Parameterizations of Very-Large-Scale Models
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Ye, Chunyang; Su, Jian-Ping; Liu, Ligang; Fu, Xiao-Ming; Eisemann, Elmar and Jacobson, Alec and Zhang, Fang-Lue
As high-precision 3D scanners become more and more widespread, it is easy to obtain very-large-scale meshes containing at least millions of vertices. However, processing these very-large-scale meshes is still a very challenging task due to memory limitations. This paper focuses on a fundamental geometric processing task, i.e., bijective parameterization construction. To this end, we present a spline-enhanced method to compute bijective and low distortion parameterizations for very-large-scale disk topology meshes. Instead of computing descent directions using the mesh vertices as variables, we estimate descent directions for each vertex by optimizing a proxy energy defined in spline spaces. Since the spline functions contain a small set of control points, it significantly decreases memory requirement. Besides, a divide-and-conquer method is proposed to obtain bijective initializations, and a submesh-based optimization strategy is developed to reduce distortion further. The capability and feasibility of our method are demonstrated over various complex models. Compared to the existing methods for bijective parameterizations of very-large-scale meshes, our method exhibits better scalability and requires much less memory.
VA-TRAC: Geospatial Trajectory Analysis for Monitoring, Identification, and Verification in Fishing Vessel Operations
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Storm-Furru, Syver; Bruckner, Stefan; Viola, Ivan and Gleicher, Michael and Landesberger von Antburg, Tatiana
In order to ensure sustainability, fishing operations are governed by many rules and regulations that restrict the use of certain techniques and equipment, specify the species and size of fish that can be harvested, and regulate commercial activities based on licensing schemes. As the world's second largest exporter of fish and seafood products, Norway invests a significant amount of effort into maintaining natural ecosystem dynamics by ensuring compliance with its constantly evolving sciencebased regulatory body. This paper introduces VA-TRAC, a geovisual analytics application developed in collaboration with the Norwegian Directorate of Fisheries in order to address this complex task. Our approach uses automatic methods to identify possible catch operations based on fishing vessel trajectories, embedded in an interactive web-based visual interface used to explore the results, compare them with licensing information, and incorporate the analysts' domain knowledge into the decision making process. We present a data and task analysis based on a close collaboration with domain experts, and the design and implementation of VA-TRAC to address the identified requirements.
Infomages: Embedding Data into Thematic Images
(The Eurographics Association and John Wiley & Sons Ltd., 2020) Coelho, Darius; Mueller, Klaus; Viola, Ivan and Gleicher, Michael and Landesberger von Antburg, Tatiana
Recent studies have indicated that visually embellished charts such as infographics have the ability to engage viewers and positively affect memorability. Fueled by these findings, researchers have proposed a variety of infographic design tools. However, these tools do not cover the entire design space. In this work, we identify a subset of infographics that we call infomages. Infomages are casual visuals of data in which a data chart is embedded into a thematic image such that the content of the image reflects the subject and the designer's interpretation of the data. Creating an effective infomage, however, can require a fair amount of design expertise and is thus out of reach for most people. In order to also afford non-artists with the means to design convincing infomages, we first study the principled design of existing infomages and identify a set of key chart embedding techniques. Informed by these findings we build a design tool that links web-scale image search with a set of interactive image processing tools to empower novice users with the ability to design a wide variety of infomages. As the embedding process might introduce some amount of visual distortion of the data our tool also aids users to gauge the amount of this distortion, if any. We experimentally demonstrate the usability of our tool and conclude with a discussion of infomages and our design tool.

Browse

Browsing Volume 39 (2020) by Issue Date

Results Per Page

Sort Options