Volume 43 (2024)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/3606941

Browse

Now showing 1 - 20 of 252

ShellNeRF: Learning a Controllable High-resolution Model of the Eye and Periocular Region
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Gengyan; Sarkar, Kripasindhu; Meka, Abhimitra; Buehler, Marcel; Mueller, Franziska; Gotardo, Paulo; Hilliges, Otmar; Beeler, Thabo; Bermano, Amit H.; Kalogerakis, Evangelos
Eye gaze and expressions are crucial non-verbal signals in face-to-face communication. Visual effects and telepresence demand significant improvements in personalized tracking, animation, and synthesis of the eye region to achieve true immersion. Morphable face models, in combination with coordinate-based neural volumetric representations, show promise in solving the difficult problem of reconstructing intricate geometry (eyelashes) and synthesizing photorealistic appearance variations (wrinkles and specularities) of eye performances. We propose a novel hybrid representation - ShellNeRF - that builds a discretized volume around a 3DMM face mesh using concentric surfaces to model the deformable 'periocular' region. We define a canonical space using the UV layout of the shells that constrains the space of dense correspondence search. Combined with an explicit eyeball mesh for modeling corneal light-transport, our model allows for animatable photorealistic 3D synthesis of the whole eye region. Using multi-view video input, we demonstrate significant improvements over state-of-the-art in expression re-enactment and transfer for high-resolution close-up views of the eye region.
Sketch Video Synthesis
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zheng, Yudian; Cun, Xiaodong; Xia, Menghan; Pun, Chi-Man; Bermano, Amit H.; Kalogerakis, Evangelos
Understanding semantic intricacies and high-level concepts is essential in image sketch generation, and this challenge becomes even more formidable when applied to the domain of videos. To address this, we propose a novel optimization-based framework for sketching videos represented by the frame-wise Bézier Curves. In detail, we first propose a cross-frame stroke initialization approach to warm up the location and the width of each curve. Then, we optimize the locations of these curves by utilizing a semantic loss based on CLIP features and a newly designed consistency loss using the self-decomposed 2D atlas network. Built upon these design elements, the resulting sketch video showcases notable visual abstraction and temporal coherence. Furthermore, by transforming a video into vector lines through the sketching process, our method unlocks applications in sketch-based video editing and video doodling, enabled through video composition.
MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Jin, Ge; Jung, Younhyun; Bi, Lei; Kim, Jinman; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Three-dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applications including pre / post-surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time-consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these limitations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh reconstruction accuracy when compared to segmentation-based methods. In this study, we address these limitations by combining the implicit function representation with a multi-level deep learning architecture. We introduce a novel multi-level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi-level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z-axis resolutions. We show that our method minimised the staircase artefacts while achieving comparable results in surface accuracy when compared to the state-of-the-art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.
SOD-diffusion: Salient Object Detection via Diffusion-Based Image Generators
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Shuo; Huang, Jiaming; Chen, Shizhe; Wu, Yan; Hu, Tao; Liu, Jing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Salient Object Detection (SOD) is a challenging task that aims to precisely identify and segment the salient objects. However, existing SOD methods still face challenges in making explicit predictions near the edges and often lack end-to-end training capabilities. To alleviate these problems, we propose SOD-diffusion, a novel framework that formulates salient object detection as a denoising diffusion process from noisy masks to object masks. Specifically, object masks diffuse from ground-truth masks to random distribution in latent space, and the model learns to reverse this noising process to reconstruct object masks. To enhance the denoising learning process, we design an attention feature interaction module (AFIM) and a specific fine-tuning protocol to integrate conditional semantic features from the input image with diffusion noise embedding. Extensive experiments on five widely used SOD benchmark datasets demonstrate that our proposed SOD-diffusion achieves favorable performance compared to previous well-established methods. Furthermore, leveraging the outstanding generalization capability of SOD-diffusion, we applied it to publicly available images, generating high-quality masks that serve as an additional SOD benchmark testset.
Learned Inference of Annual Ring Pattern of Solid Wood
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Larsson, Maria; Ijiri, Takashi; Shen, I‐Chao; Yoshida, Hironori; Shamir, Ariel; Igarashi, Takeo; Alliez, Pierre; Wimmer, Michael
We propose a method for inferring the internal anisotropic volumetric texture of a given wood block from annotated photographs of its external surfaces. The global structure of the annual ring pattern is represented using a continuous spatial scalar field referred to as the growth time field (GTF). First, we train a generic neural model that can represent various GTFs using procedurally generated training data. Next, we fit the generic model to the GTF of a given wood block based on surface annotations. Finally, we convert the GTF to an annual ring field (ARF) revealing the layered pattern and apply neural style transfer to render orientation‐dependent small‐scale features and colors on a cut surface. We show rendered results of various physically cut real wood samples. Our method has physical and virtual applications such as cut‐preview before subtractive fabricating solid wood artifacts and simulating object breaking.
A Surface-based Appearance Model for Pennaceous Feathers
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Padrón-Griffe, Juan Raúl; Lanza, Dario; Jarabo, Adrian; Muñoz, Adolfo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
The appearance of a real-world feather results from the complex interaction of light with its multi-scale biological structure, including the central shaft, branching barbs, and interlocking barbules on those barbs. In this work, we propose a practical surface-based appearance model for feathers. We represent the far-field appearance of feathers using a BSDF that implicitly represents the light scattering from the main biological structures of a feather, such as the shaft, barb and barbules. Our model accounts for the particular characteristics of feather barbs such as the non-cylindrical cross-sections and the scattering media via a numerically-based BCSDF. To model the relative visibility between barbs and barbules, we derive a masking term for the differential projected areas of the different components of the feather's microgeometry, which allows us to analytically compute the masking between barbs and barbules. As opposed to previous works, our model uses a lightweight representation of the geometry based on a 2D texture, and does not require explicitly representing the barbs as curves. We show the flexibility and potential of our appearance model approach to represent the most important visual features of several pennaceous feathers.
A Hierarchical Architecture for Neural Materials
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Xue, Bowen; Zhao, Shuang; Jensen, Henrik Wann; Montazeri, Zahra; Alliez, Pierre; Wimmer, Michael
Neural reflectance models are capable of reproducing the spatially‐varying appearance of many real‐world materials at different scales. Unfortunately, existing techniques such as NeuMIP have difficulties handling materials with strong shadowing effects or detailed specular highlights. In this paper, we introduce a neural appearance model that offers a new level of accuracy. Central to our model is an inception‐based core network structure that captures material appearances at multiple scales using parallel‐operating kernels and ensures multi‐stage features through specialized convolution layers. Furthermore, we encode the inputs into frequency space, introduce a gradient‐based loss, and employ it adaptive to the progress of the learning phase. We demonstrate the effectiveness of our method using a variety of synthetic and real examples.
Residual Path Integrals for Re-rendering
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Xu, Bing; Li, Tzu-Mao; Georgiev, Iliyan; Hedstrom, Trevor; Ramamoorthi, Ravi; Garces, Elena; Haines, Eric
Conventional rendering techniques are primarily designed and optimized for single-frame rendering. In practical applications, such as scene editing and animation rendering, users frequently encounter scenes where only a small portion is modified between consecutive frames. In this paper, we develop a novel approach to incremental re-rendering of scenes with dynamic objects, where only a small part of a scene moves from one frame to the next. We formulate the difference (or residual) in the image between two frames as a (correlated) light-transport integral which we call the residual path integral. Efficient numerical solution of this integral then involves (1) devising importance sampling strategies to focus on paths with non-zero residual-transport contributions and (2) choosing appropriate mappings between the native path spaces of the two frames. We introduce a set of path importance sampling strategies that trace from the moving object(s) which are the sources of residual energy. We explore path mapping strategies that generalize those from gradient-domain path tracing to our importance sampling techniques specially for dynamic scenes. Additionally, our formulation can be applied to material editing as a simpler special case. We demonstrate speed-ups over previous correlated sampling of path differences and over rendering the new frame independently. Our formulation brings new insights into the re-rendering problem and paves the way for devising new types of sampling techniques and path mappings with different trade-offs.
Identifying and Visualizing Terrestrial Magnetospheric Topology using Geodesic Level Set Method
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Xiong, Peikun; Fujita, Shigeru; Watanabe, Masakazu; Tanaka, Takashi; Cai, Dongsheng; Alliez, Pierre; Wimmer, Michael
This study introduces a novel numerical method for identifying and visualizing the terrestrial magnetic field topology in a large‐scale three‐dimensional global MHD (Magneto‐Hydro‐Dynamic) simulation. The (un)stable two‐dimensional manifolds are generated from critical points (CPs) located north and south of the magnetosphere using an improved geodesic level set method. A boundary value problem is solved numerically using a shooting method to forward a new geodesic level set from the previous set. These sets are generated starting from a small circle whose centre is a CP. The level sets are the sets of mesh points that form the magnetic manifold, which determines the magnetic field topology. In this study, a consistent method is proposed to determine the magnetospheric topology. Using this scheme, we successfully visualize a terrestrial magnetospheric field topology and identify its two neutral lines using the global MHD simulation. Our results present a terrestrial topology that agrees well with the recent magnetospheric physics and can help us understand various nonlinear magnetospheric dynamics and phenomena. Our visualization enables us to fill the gaps between current magnetospheric physics that can be observed via satellites and nonlinear dynamics, particularly, the bifurcation theory, in the future.
Instantaneous Visual Analysis of Blood Flow in Stenoses Using Morphological Similarity
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Eulzer, Pepe; Richter, Kevin; Hundertmark, Anna; Wickenhoefer, Ralph; Klingner, Carsten; Lawonn, Kai; Aigner, Wolfgang; Archambault, Daniel; Bujack, Roxana
The emergence of computational fluid dynamics (CFD) enabled the simulation of intricate transport processes, including flow in physiological structures, such as blood vessels. While these so-called hemodynamic simulations offer groundbreaking opportunities to solve problems at the clinical forefront, a successful translation of CFD to clinical decision-making is challenging. Hemodynamic simulations are intrinsically complex, time-consuming, and resource-intensive, which conflicts with the timesensitive nature of clinical workflows and the fact that hospitals usually do not have the necessary resources or infrastructure to support CFD simulations. To address these transfer challenges, we propose a novel visualization system which enables instant flow exploration without performing on-site simulation. To gain insights into the viability of the approach, we focus on hemodynamic simulations of the carotid bifurcation, which is a highly relevant arterial subtree in stroke diagnostics and prevention. We created an initial database of 120 high-resolution carotid bifurcation flow models and developed a set of similarity metrics used to place a new carotid surface model into a neighborhood of simulated cases with the highest geometric similarity. The neighborhood can be immediately explored and the flow fields analyzed.We found that if the artery models are similar enough in the regions of interest, a new simulation leads to coinciding results, allowing the user to circumvent individual flow simulations. We conclude that similarity-based visual analysis is a promising approach toward the usability of CFD in medical practice.
DSGI-Net: Density-based Selective Grouping Point Cloud Learning Network for Indoor Scene
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Wen, Xin; Duan, Yao; Xu, Kai; Zhu, Chenyang; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Indoor scene point clouds exhibit diverse distributions and varying levels of sparsity, characterized by more intricate geometry and occlusion compared to outdoor scenes or individual objects. Despite recent advancements in 3D point cloud analysis introducing various network architectures, there remains a lack of frameworks tailored to the unique attributes of indoor scenarios. To address this, we propose DSGI-Net, a novel indoor scene point cloud learning network that can be integrated into existing models. The key innovation of this work is selectively grouping more informative neighbor points in sparse regions and promoting semantic consistency of the local area where different instances are in proximity but belong to distinct categories. Furthermore, our method encodes both semantic and spatial relationships between points in local regions to reduce the loss of local geometric details. Extensive experiments on the ScanNetv2, SUN RGB-D, and S3DIS indoor scene benchmarks demonstrate that our method is straightforward yet effective.
Enhancing Spatiotemporal Resampling with a Novel MIS Weight
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Pan, Xingyue; Zhang, Jiaxuan; Huang, Jiancong; Liu, Ligang; Bermano, Amit H.; Kalogerakis, Evangelos
In real-time rendering, optimizing the sampling of large-scale candidates is crucial. The spatiotemporal reservoir resampling (ReSTIR) method provides an effective approach for handling large candidate samples, while the Generalized Resampled Importance Sampling (GRIS) theory provides a general framework for resampling algorithms. However, we have observed that when using the generalized multiple importance sampling (MIS) weight in previous work during spatiotemporal reuse, variances gradually amplify in the candidate domain when there are significant differences. To address this issue, we propose a new MIS weight suitable for resampling that blends samples from different sampling domains, ensuring convergence of results as the proportion of non-canonical samples increases. Additionally, we apply this weight to temporal resampling to reduce noise caused by scene changes or jitter. Our method effectively reduces energy loss in the biased version of ReSTIR DI while incurring no additional overhead, and it also suppresses artifacts caused by a high proportion of temporal samples. As a result, our approach leads to lower variance in the sampling results.
Real‐Time Polygonal Lighting of Iridescence Effect using Precomputed Monomial‐Gaussians
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Liu, Zhengze; Huo, Yuchi; Yang, Yinhui; Chen, Jie; Wang, Rui; Alliez, Pierre; Wimmer, Michael
The real world consists of mass phenomena, such as iridescence on thin film and metal oxide layers, that is only explicable by wave optics. Existing research can reproduce such effects with simple point lights or low‐frequency environmental lighting. However, it remains a difficult task to efficiently rendering these effects when near‐field, high‐frequency area lights are involved. This paper presents a high‐fidelity, real‐time rendering algorithm for the iridescence effect under polygonal lights. We introduce a novel set of spherical functions, Monomial‐Gaussians, to accurately fit iridescent materials' reflectance. With a precomputed lookup table, the Monomial‐Gaussians are easily integrated over spherical polygons in linear time. Importance sampling of Monomial‐Gaussians is also supported to efficiently reduce Monte‐Carlo error. Our approach produces accurate renderings of the iridescence effect while still preserving high frame rates.
A High‐Scalability Graph Modification System for Large‐Scale Networks
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Xu, Shaobin; Sun, Minghui; Qin, Jun; Alliez, Pierre; Wimmer, Michael
Modifying network results is the most intuitive way to inject domain knowledge into network detection algorithms to improve their performance. While advances in computation scalability have made detecting large‐scale networks possible, the human ability to modify such networks has not scaled accordingly, resulting in a huge ‘interaction gap’. Most existing works only support navigating and modifying edges one by one in a graph visualization, which causes a significant interaction burden when faced with large‐scale networks. In this work, we propose a novel graph pattern mining algorithm based on the minimum description length (MDL) principle to partition and summarize multi‐feature and isomorphic sub‐graph matches. The mined sub‐graph patterns can be utilized as mediums for modifying large‐scale networks. Combining two traditional approaches, we introduce a new coarse‐middle‐fine graph modification paradigm (. query graph‐based modification sub‐graph pattern‐based modification raw edge‐based modification). We further present a graph modification system that supports the graph modification paradigm for improving the scalability of modifying detected large‐scale networks. We evaluate the performance of our graph pattern mining algorithm through an experimental study, demonstrate the usefulness of our system through a case study, and illustrate the efficiency of our graph modification paradigm through a user study.
ProtEGOnist: Visual Analysis of Interactions in Small World Networks Using Ego-graphs
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Brich, Nicolas; Harbig, Theresa A.; Witte Paz, Mathias; Nieselt, Kay; Krone, Michael; Aigner, Wolfgang; Archambault, Daniel; Bujack, Roxana
Visualizing small-world networks such as protein-protein interaction networks or social networks often leads to visual clutter and limited interpretability. To overcome these problems, we present ProtEGOnist, a visualization approach designed to explore small-world networks. ProtEGOnist visualizes networks using ego-graphs that represent local neighborhoods. Egographs are visualized in an aggregated state as a glyph where the size encodes the size of the neighborhood and in a detailed version where the original network nodes can be explored. The ego-graphs are arranged in an ego-graph network, where edges encode similarity using the Jaccard index. Our design aims to reduce visual complexity and clutter while enabling detailed exploration and facilitating the discovery of meaningful patterns. To achieve this, our approach offers a network overview using ego-graphs, a radar chart for a one-to-many ego-graph comparison and meta-data integration, and detailed ego-graph subnetworks for interactive exploration. We demonstrate the applicability of our approach on a co-author network and two different protein-protein interaction networks. A web-based prototype of ProtEGOnist can be accessed online at https://protegonist-tuevis.cs.uni-tuebingen.de/.
Generating Flight Summaries Conforming to Cinematographic Principles
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Lino, Christophe; Cani, Marie-Paule; Skouras, Melina; Wang, He
We propose an automatic method for generating flight summaries of prescribed duration, given any planed 3D trajectory of a flying object. The challenge is to select relevant time-ellipses, while keeping and adequately framing the most interesting parts of the trajectory, and enforcing cinematographic rules between the selected shots. Our solution optimizes the visual quality of the output video both in terms of camera view and film editing choices, thanks to a new optimization technique, designed to jointly optimize the selection of the interesting parts of a flight, and the camera animation parameters over time. To our best knowledge, this solution is the first one to address camera control, film editing, and trajectory summarizing at once. Ablation studies demonstrate the visual quality of the flights summaries we generate compared to alternative methods.
ChoreoVis: Planning and Assessing Formations in Dance Choreographies
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Beck, Samuel; Doerr, Nina; Kurzhals, Kuno; Riedlinger, Alexander; Schmierer, Fabian; Sedlmair, Michael; Koch, Steffen; Aigner, Wolfgang; Archambault, Daniel; Bujack, Roxana
Sports visualization has developed into an active research field over the last decades. Many approaches focus on analyzing movement data recorded from unstructured situations, such as soccer. For the analysis of choreographed activities like formation dancing, however, the goal differs, as dancers follow specific formations in coordinated movement trajectories. To date, little work exists on how visual analytics methods can support such choreographed performances. To fill this gap, we introduce a new visual approach for planning and assessing dance choreographies. In terms of planning choreographies, we contribute a web application with interactive authoring tools and views for the dancers' positions and orientations, movement trajectories, poses, dance floor utilization, and movement distances. For assessing dancers' real-world movement trajectories, extracted by manual bounding box annotations, we developed a timeline showing aggregated trajectory deviations and a dance floor view for detailed trajectory comparison. Our approach was developed and evaluated in collaboration with dance instructors, showing that introducing visual analytics into this domain promises improvements in training efficiency for the future.
Mesh Parameterization Meets Intrinsic Triangulations
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Akalin, Koray; Finnendahl, Ugo; Sorkine-Hornung, Olga; Alexa, Marc; Hu, Ruizhen; Lefebvre, Sylvain
A parameterization of a triangle mesh is a realization in the plane so that all triangles have positive signed area. Triangle mesh parameterizations are commonly computed by minimizing a distortion energy, measuring the distortions of the triangles as they are mapped into the parameter domain. It is assumed that the triangulation is fixed and the triangles are mapped affinely. We consider a more general setup and additionally optimize among the intrinsic triangulations of the piecewise linear input geometry. This means the distortion energy is computed for the same geometry, yet the space of possible parameterizations is enlarged. For minimizing the distortion energy, we suggest alternating between varying the parameter locations of the vertices and intrinsic flipping. We show that this process improves the mapping for different distortion energies at moderate additional cost. We also find intrinsic triangulations that are better starting points for the optimization of positions, offering a compromise between the full optimization approach and exploiting the additional freedom of intrinsic triangulations.
Row–Column Separated Attention Based Low‐Light Image/Video Enhancement
(© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2024) Dong, Chengqi; Cao, Zhiyuan; Qi, Tuoshi; Wu, Kexin; Gao, Yixing; Tang, Fan; Alliez, Pierre; Wimmer, Michael
U‐Net structure is widely used for low‐light image/video enhancement. The enhanced images result in areas with large local noise and loss of more details without proper guidance for global information. Attention mechanisms can better focus on and use global information. However, attention to images could significantly increase the number of parameters and computations. We propose a Row–Column Separated Attention module (RCSA) inserted after an improved U‐Net. The RCSA module's input is the mean and maximum of the row and column of the feature map, which utilizes global information to guide local information with fewer parameters. We propose two temporal loss functions to apply the method to low‐light video enhancement and maintain temporal consistency. Extensive experiments on the LOL, MIT Adobe FiveK image, and SDSD video datasets demonstrate the effectiveness of our approach.
Eurographics/ ACM SIGGRAPH Symposium on Computer Animation 2024 - CGF 43-8: Frontmatter
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Skouras, Melina; Wang, He; Skouras, Melina; Wang, He

Browse

Browsing Volume 43 (2024) by Issue Date

Results Per Page

Sort Options