Volume 42 (2023)

Permanent URI for this community

https://diglib.eg.org/handle/10.2312/2633290

Browse

Now showing 1 - 20 of 243

Markov Chain Mixture Models for Real-Time Direct Illumination
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Dittebrandt, Addis; Schüßler, Vincent; Hanika, Johannes; Herholz, Sebastian; Dachsbacher, Carsten; Ritschel, Tobias; Weidlich, Andrea
We present a novel technique to efficiently render complex direct illumination in real-time. It is based on a spatio-temporal randomized mixture model of von Mises-Fisher (vMF) distributions in screen space. For every pixel we determine the vMF distribution to sample from using a Markov chain process which is targeted to capture important features of the integrand. By this we avoid the storage overhead of finite-component deterministic mixture models, for which, in addition, determining the optimal component count is challenging. We use stochastic multiple importance sampling (SMIS) to be independent of the equilibrium distribution of our Markov chain process, since it cancels out in the estimator. Further, we use the same sample to advance the Markov chain and to construct the SMIS estimator and local Markov chain state permutations avoid the resulting bias due to dependent sampling. As a consequence we require one ray per sample and pixel only. We evaluate our technique using implementations in a research renderer as well as a classic game engine with highly dynamic content. Our results show that it is efficient and quickly readapts to dynamic conditions. We compare to spatio-temporal resampling (ReSTIR), which can suffer from correlation artifacts due to its non-adapting candidate distributions that can deviate strongly from the integrand.While we focus on direct illumination, our approach is more widely applicable and we exemplarily show the rendering of caustics.
A Fully Integrated Pipeline for Visual Carotid Morphology Analysis
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Eulzer, Pepe; Deylen, Fabienne von; Hsu, Wei-Chan; Wickenhöfer, Ralph; Klingner, Carsten M.; Lawonn, Kai; Bujack, Roxana; Archambault, Daniel; Schreck, Tobias
Analyzing stenoses of the internal carotids - local constrictions of the artery - is a critical clinical task in cardiovascular disease treatment and prevention. For this purpose, we propose a self-contained pipeline for the visual analysis of carotid artery geometries. The only inputs are computed tomography angiography (CTA) scans, which are already recorded in clinical routine. We show how integrated model extraction and visualization can help to efficiently detect stenoses and we provide means for automatic, highly accurate stenosis degree computation. We directly connect multiple sophisticated processing stages, including a neural prediction network for lumen and plaque segmentation and automatic global diameter computation. We enable interactive and retrospective user control over the processing stages. Our aims are to increase user trust by making the underlying data validatable on the fly, to decrease adoption costs by minimizing external dependencies, and to optimize scalability by streamlining the data processing. We use interactive visualizations for data inspection and adaption to guide the user through the processing stages. The framework was developed and evaluated in close collaboration with radiologists and neurologists. It has been used to extract and analyze over 100 carotid bifurcation geometries and is built with a modular architecture, available as an extendable open-source platform.
VOLMAP: a Large Scale Benchmark for Volume Mappings to Simple Base Domains
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Cherchi, Gianmarco; Livesu, Marco; Memari, Pooran; Solomon, Justin
Correspondences between geometric domains (mappings) are ubiquitous in computer graphics and engineering, both for a variety of downstream applications and as core building blocks for higher level algorithms. In particular, mapping a shape to a convex or star-shaped domain with simple geometry is a fundamental module in existing pipelines for mesh generation, solid texturing, generation of shape correspondences, advanced manufacturing etc. For the case of surfaces, computing such a mapping with guarantees of injectivity is a solved problem. Conversely, robust algorithms for the generation of injective volume mappings to simple polytopes are yet to be found, making this a fundamental open problem in volume mesh processing. VOLMAP is a large scale benchmark aimed to support ongoing research in volume mapping algorithms. The dataset contains 4.7K tetrahedral meshes, whose boundary vertices are mapped to a variety of simple domains, either convex or star-shaped. This data constitutes the input for candidate algorithms, which are then required to position interior vertices in the domain to obtain a volume map. Overall, this yields more than 22K alternative test cases. VOLMAP also comprises tools to process this data, analyze the resulting maps, and extend the dataset with new meshes, boundary maps and base domains. This article provides a brief overview of the field, discussing its importance and the lack of effective techniques. We then introduce both the dataset and its major features. An example of comparative analysis between two existing methods is also present.
Stochastic Subsets for BVH Construction
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Tessari, Lorenzo; Dittebrandt, Addis; Doyle, Michael J.; Benthin, Carsten; Myszkowski, Karol; Niessner, Matthias
BVH construction is a critical component of real-time and interactive ray-tracing systems. However, BVH construction can be both compute and bandwidth intensive, especially when a large degree of dynamic geometry is present. Different build algorithms vary substantially in the traversal performance that they produce, making high quality construction algorithms desirable. However, high quality algorithms, such as top-down construction, are typically more expensive, limiting their benefit in real-time and interactive contexts. One particular challenge of high quality top-down construction algorithms is that the large working set at the top of the tree can make constructing these levels bandwidth-intensive, due to O(nlog(n)) complexity, limited cache locality, and less dense compute at these levels. To address this limitation, we propose a novel stochastic approach to GPU BVH construction that selects a representative subset to build the upper levels of the tree. As a second pass, the remaining primitives are clustered around the BVH leaves and further processed into a complete BVH. We show that our novel approach significantly reduces the construction time of top-down GPU BVH builders by a factor up to 1.8x, while achieving competitive rendering performance in most cases, and exceeding the performance in others.
Learning to Learn and Sample BRDFs
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Liu, Chen; Fischer, Michael; Ritschel, Tobias; Myszkowski, Karol; Niessner, Matthias
We propose a method to accelerate the joint process of physically acquiring and learning neural Bi-directional Reflectance Distribution Function (BRDF) models. While BRDF learning alone can be accelerated by meta-learning, acquisition remains slow as it relies on a mechanical process. We show that meta-learning can be extended to optimize the physical sampling pattern, too. After our method has been meta-trained for a set of fully-sampled BRDFs, it is able to quickly train on new BRDFs with up to five orders of magnitude fewer physical acquisition samples at similar quality. Our approach also extends to other linear and non-linear BRDF models, which we show in an extensive evaluation.
iFUNDit: Visual Profiling of Fund Investment Styles
(© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) Zhang, R.; Ku, B. K.; Wang, Y.; Yue, X.; Liu, S.; Li, K.; Qu, H.; Hauser, Helwig and Alliez, Pierre
Mutual funds are becoming increasingly popular with the emergence of Internet finance. Clear profiling of a fund's investment style is crucial for fund managers to evaluate their investment strategies, and for investors to understand their investment. However, it is challenging to profile a fund's investment style as it requires a comprehensive analysis of complex multi‐dimensional temporal data. In addition, different fund managers and investors have different focuses when analysing a fund's investment style. To address the issue, we propose , an interactive visual analytic system for fund investment style analysis. The system decomposes a fund's critical features into performance attributes and investment style factors, and visualizes them in a set of coupled views: a fund and manager view, to delineate the distribution of funds' and managers' critical attributes on the market; a cluster view, to show the similarity of investment styles between different funds; and a detail view, to analyse the evolution of fund investment style. The system provides a holistic overview of fund data and facilitates a streamlined analysis of investment style at both the fund and the manager level. The effectiveness and usability of the system are demonstrated through domain expert interviews and case studies by using a real mutual fund dataset.
LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Arunkumar, Anjana; Sharma, Shubham; Agrawal, Rakhi; Chandrasekaran, Sriram; Bryan, Chris; Bujack, Roxana; Archambault, Daniel; Schreck, Tobias
Cross-task generalization is a significant outcome that defines mastery in natural language understanding. Humans show a remarkable aptitude for this, and can solve many different types of tasks, given definitions in the form of textual instructions and a small set of examples. Recent work with pre-trained language models mimics this learning style: users can define and exemplify a task for the model to attempt as a series of natural language prompts or instructions. While prompting approaches have led to higher cross-task generalization compared to traditional supervised learning, analyzing 'bias' in the task instructions given to the model is a difficult problem, and has thus been relatively unexplored. For instance, are we truly modeling a task, or are we modeling a user's instructions? To help investigate this, we develop LINGO, a novel visual analytics interface that supports an effective, task-driven workflow to (1) help identify bias in natural language task instructions, (2) alter (or create) task instructions to reduce bias, and (3) evaluate pre-trained model performance on debiased task instructions. To robustly evaluate LINGO, we conduct a user study with both novice and expert instruction creators, over a dataset of 1,616 linguistic tasks and their natural language instructions, spanning 55 different languages. For both user groups, LINGO promotes the creation of more difficult tasks for pre-trained models, that contain higher linguistic diversity and lower instruction bias. We additionally discuss how the insights learned in developing and evaluating LINGO can aid in the design of future dashboards that aim to minimize the effort involved in prompt creation across multiple domains.
VisCoMET: Visually Analyzing Team Collaboration in Medical Emergency Trainings
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Liebers, Carina; Agarwal, Shivam; Krug, Maximilian; Pitsch, Karola; Beck, Fabian; Bujack, Roxana; Archambault, Daniel; Schreck, Tobias
Handling emergencies requires efficient and effective collaboration of medical professionals. To analyze their performance, in an application study, we have developed VisCoMET, a visual analytics approach displaying interactions of healthcare personnel in a triage training of a mass casualty incident. The application scenario stems from social interaction research, where the collaboration of teams is studied from different perspectives. We integrate recorded annotations from multiple sources, such as recorded videos of the sessions, transcribed communication, and eye-tracking information. For each session, an informationrich timeline visualizes events across these different channels, specifically highlighting interactions between the team members. We provide algorithmic support to identify frequent event patterns and to search for user-defined event sequences. Comparing different teams, an overview visualization aggregates each training session in a visual glyph as a node, connected to similar sessions through edges. An application example shows the usage of the approach in the comparative analysis of triage training sessions, where multiple teams encountered the same scene, and highlights discovered insights. The approach was evaluated through feedback from visualization and social interaction experts. The results show that the approach supports reflecting on teams' performance by exploratory analysis of collaboration behavior while particularly enabling the comparison of triage training sessions.
Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Liu, Ruiyang; Xiang, Jinxu; Zhao, Bowen; Zhang, Ran; Yu, Jingyi; Zheng, Changxi; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
Neural Radiance Fields (NeRF) have significantly advanced the generation of highly realistic and expressive 3D scenes. However, the task of editing NeRF, particularly in terms of geometry modification, poses a significant challenge. This issue has obstructed NeRF's wider adoption across various applications. To tackle the problem of efficiently editing neural implicit fields, we introduce Neural Impostor, a hybrid representation incorporating an explicit tetrahedral mesh alongside a multigrid implicit field designated for each tetrahedron within the explicit mesh. Our framework bridges the explicit shape manipulation and the geometric editing of implicit fields by utilizing multigrid barycentric coordinate encoding, thus offering a pragmatic solution to deform, composite, and generate neural implicit fields while maintaining a complex volumetric appearance. Furthermore, we propose a comprehensive pipeline for editing neural implicit fields based on a set of explicit geometric editing operations. We show the robustness and adaptability of our system through diverse examples and experiments, including the editing of both synthetic objects and real captured data. Finally, we demonstrate the authoring process of a hybrid synthetic-captured object utilizing a variety of editing operations, underlining the transformative potential of Neural Impostor in the field of 3D content creation and manipulation.
IBL-NeRF: Image-Based Lighting Formulation of Neural Radiance Fields
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Choi, Changwoon; Kim, Juhyeon; Kim, Young Min; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
We propose IBL-NeRF, which decomposes the neural radiance fields (NeRF) of large-scale indoor scenes into intrinsic components. Recent approaches further decompose the baked radiance of the implicit volume into intrinsic components such that one can partially approximate the rendering equation. However, they are limited to representing isolated objects with a shared environment lighting, and suffer from computational burden to aggregate rays with Monte Carlo integration. In contrast, our prefiltered radiance field extends the original NeRF formulation to capture the spatial variation of lighting within the scene volume, in addition to surface properties. Specifically, the scenes of diverse materials are decomposed into intrinsic components for rendering, namely, albedo, roughness, surface normal, irradiance, and prefiltered radiance. All of the components are inferred as neural images from MLP, which can model large-scale general scenes. Especially the prefiltered radiance effectively models the volumetric light field, and captures spatial variation beyond a single environment light. The prefiltering aggregates rays in a set of predefined neighborhood sizes such that we can replace the costly Monte Carlo integration of global illumination with a simple query from a neural image. By adopting NeRF, our approach inherits superior visual quality and multi-view consistency for synthesized images as well as the intrinsic components. We demonstrate the performance on scenes with complex object layouts and light configurations, which could not be processed in any of the previous works.
Groupwise Shape Correspondence Refinement with a Region of Interest Focus
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Galmiche, Pierre; Seo, Hyewon; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
While collections of scan shapes are becoming more prevalent in many real-world applications, finding accurate and dense correspondences across multiple shapes remains a challenging task. In this work, we introduce a new approach for refining non-rigid correspondences among a collection of 3D shapes undergoing non-rigid deformation. Our approach incorporates a Region Of Interest (ROI) into the refinement process, which is specified by the user on one shape within the collection. Based on the functional map framework and more specifically on the notion of cycle-consistency, our formulation improves the overall matching consistency while prioritizing that of the region of interest. Specifically, the initial pairwise correspondences are refined by first defining the localized harmonics that are confined within the transferred ROI on each shape, and subsequently applying the CCLB (Canonical Consistent Latent Basis) framework both on the global and the localized harmonics. This leads to an enhanced matching accuracy for both the ROIs and the overall shapes across the collection. We evaluate our method on various synthetic and real scan datasets, in comparison with the state-of-the-art techniques.
State of the Art of Visual Analytics for eXplainable Deep Learning
(Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) La Rosa, B.; Blasilli, G.; Bourqui, R.; Auber, D.; Santucci, G.; Capobianco, R.; Bertini, E.; Giot, R.; Angelini, M.; Hauser, Helwig and Alliez, Pierre
The use and creation of machine‐learning‐based solutions to solve problems or reduce their computational costs are becoming increasingly widespread in many domains. Deep Learning plays a large part in this growth. However, it has drawbacks such as a lack of explainability and behaving as a black‐box model. During the last few years, Visual Analytics has provided several proposals to cope with these drawbacks, supporting the emerging eXplainable Deep Learning field. This survey aims to (i) systematically report the contributions of Visual Analytics for eXplainable Deep Learning; (ii) spot gaps and challenges; (iii) serve as an anthology of visual analytical solutions ready to be exploited and put into operation by the Deep Learning community (architects, trainers and end users) and (iv) prove the degree of maturity, ease of integration and results for specific domains. The survey concludes by identifying future research challenges and bridging activities that are helpful to strengthen the role of Visual Analytics as effective support for eXplainable Deep Learning and to foster the adoption of Visual Analytics solutions in the eXplainable Deep Learning community. An interactive explorable version of this survey is available online at .
Efficient Hardware Acceleration of Robust Volumetric Light Transport Simulation
(© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) Moonen, Nol; Jalba, Andrei C.; Hauser, Helwig and Alliez, Pierre
Efficiently simulating the full range of light effects in arbitrary input scenes that contain participating media is a difficult task. Unified points, beams and paths (UPBP) is an algorithm capable of capturing a wide range of media effects, by combining bidirectional path tracing (BPT) and photon density estimation (PDE) with multiple importance sampling (MIS). A computationally expensive task of UPBP is the MIS weight computation, performed each time a light path is formed. We derive an efficient algorithm to compute the MIS weights for UPBP, which improves over previous work by eliminating the need to iterate over the path vertices. We achieve this by maintaining recursive quantities as subpaths are generated, from which the subpath weights can be computed. In this way, the full path weight can be computed by only using the data cached at the two vertices at the ends of the subpaths. Furthermore, a costly part of PDE is the search for nearby photon points and beams. Previous work has shown that a spatial data structure for photon mapping can be implemented using the hardware‐accelerated bounding volume hierarchy of NVIDIA's RTX GPUs. We show that the same technique can be applied to different types of volumetric PDE and compare the performance of these data structures with the state of the art. Finally, using our new algorithm and data structures we fully implement UPBP on the GPU which we, to the best of our knowledge, are the first to do so.
One Step Further Beyond Trilinear Interpolation and Central Differences: Triquadratic Reconstruction and its Analytic Derivatives at the Cost of One Additional Texture Fetch
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Csébfalvi, Balázs; Myszkowski, Karol; Niessner, Matthias
Recently, it has been shown that the quality of GPU-based trilinear volume resampling can be significantly improved if the six additional trilinear samples evaluated for the gradient estimation also contribute to the reconstruction of the underlying function [Csé19]. Although this improvement increases the approximation order from two to three without any extra cost, the continuity order remains C0. In this paper, we go one step further showing that a C1 continuous triquadratic B-spline reconstruction and its analytic partial derivatives can be evaluated by taking only one more trilinear sample into account. Thus, our method is the first volume-resampling technique that is nearly as fast as trilinear interpolation combined with on-thefly central differencing, but provides a higher-quality reconstruction together with a consistent analytic gradient calculation. Furthermore, we show that our fast evaluation scheme can also be adapted to the Mitchell-Netravali [MN88] notch filter, for which a fast GPU implementation has not been known so far.
SVBRDF Reconstruction by Transferring Lighting Knowledge
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Zhu, Pengfei; Lai, Shuichang; Chen, Mufan; Guo, Jie; Liu, Yifan; Guo, Yanwen; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
The problem of reconstructing spatially-varying BRDFs from RGB images has been studied for decades. Researchers found themselves in a dilemma: opting for either higher quality with the inconvenience of camera and light calibration, or greater convenience at the expense of compromised quality without complex setups. We address this challenge by introducing a twobranch network to learn the lighting effects in images. The two branches, referred to as Light-known and Light-aware, diverge in their need for light information. The Light-aware branch is guided by the Light-known branch to acquire the knowledge of discerning light effects and surface reflectance properties, but without the reliance of light positions. Both branches are trained using the synthetic dataset, but during testing on real-world cases without calibration, only the Light-aware branch is activated. To facilitate a more effective utilization of various light conditions, we employ gated recurrent units (GRUs) to fuse the features extracted from different images. The two modules mutually benefit when multiple inputs are provided. We present our reconstructed results on both synthetic and real-world examples, demonstrating high quality while maintaining a lightweight characteristic in comparison to previous methods.
Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Cogalan, Ugur; Bemana, Mojtaba; Seidel, Hans-Peter; Myszkowski, Karol; Myszkowski, Karol; Niessner, Matthias
Video frame interpolation (VFI) enables many important applications such as slow motion playback and frame rate conversion. However, one major challenge in using VFI is accurately handling high dynamic range (HDR) scenes with complex motion. To this end, we explore the possible advantages of dual-exposure sensors that readily provide sharp short and blurry long exposures that are spatially registered and whose ends are temporally aligned. This way, motion blur registers temporally continuous information on the scene motion that, combined with the sharp reference, enables more precise motion sampling within a single camera shot. We demonstrate that this facilitates a more complex motion reconstruction in the VFI task, as well as HDR frame reconstruction that so far has been considered only for the originally captured frames, not in-between interpolated frames. We design a neural network trained in these tasks that clearly outperforms existing solutions. We also propose a metric for scene motion complexity that provides important insights into the performance of VFI methods at test time.
A Surface Subdivision Scheme Based on Four-Directional S^1_3 Non-Box Splines
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Huang, Zhangjin; Chaine, Raphaëlle; Deng, Zhigang; Kim, Min H.
In this paper, we propose a novel surface subdivision scheme called non-box subdivision, which is generalized from fourdirectional S13 on-box splines. The resulting subdivision surfaces achieve C1 continuity with the convex hull property. This scheme can be regarded as either a four-directional subdivision or a special quadrilateral subdivision. When used as a quadrilateral subdivision, the proposed scheme can control the shape of the limit surface more flexibly than traditional schemes due to the natural introduction of auxiliary face control vertices.
Detail‐Aware Deep Clothing Animations Infused with Multi‐Source Attributes
(Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) Li, T.; Shi, R.; Kanai, T.; Hauser, Helwig and Alliez, Pierre
This paper presents a novel learning‐based clothing deformation method to generate rich and reasonable detailed deformations for garments worn by bodies of various shapes in various animations. In contrast to existing learning‐based methods, which require numerous trained models for different garment topologies or poses and are unable to easily realize rich details, we use a unified framework to produce high fidelity deformations efficiently and easily. Specifically, we first found that the fit between the garment and the body has an important impact on the degree of folds. We then designed an attribute parser to generate detail‐aware encodings and infused them into the graph neural network, therefore enhancing the discrimination of details under diverse attributes. Furthermore, to achieve better convergence and avoid overly smooth deformations, we proposed to reconstruct output to mitigate the complexity of the learning task. Experimental results show that our proposed deformation method achieves better performance over existing methods in terms of generalization ability and quality of details.
SGP 2023 CGF 42-5: Frontmatter
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Memari, Pooran; Solomon, Justin; Memari, Pooran; Solomon, Justin
Practical Acquisition of Shape and Plausible Appearance of Reflective and Translucent Objects
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Lin, Arvin; Lin, Yiming; Ghosh, Abhijeet; Ritschel, Tobias; Weidlich, Andrea
We present a practical method for acquisition of shape and plausible appearance of reflective and translucent objects for realistic rendering and relighting applications. Such objects are extremely challenging to scan with existing capture setups, and have previously required complex lightstage hardware emitting continuous illumination. We instead employ a practical capture setup consisting of a set of desktop LCD screens to illuminate such objects with piece-wise continuous illumination for acquisition. We employ phase-shifted sinusoidal illumination for novel estimation of high quality photometric normals and transmission vector along with diffuse-specular separated reflectance/transmission maps for realistic relighting. We further employ neural in-painting to fill gaps in our measurements caused by gaps in screen illumination, and a novel NeuS-based neural rendering that combines these shape and reflectance maps acquired from multiple viewpoints for high-quality 3D surface geometry reconstruction along with plausible realistic rendering of complex light transport in such objects.

Browse

Browsing Volume 42 (2023) by Issue Date

Results Per Page

Sort Options