VMV2023

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3543878

Browse

Now showing 1 - 20 of 27

Art-directable Stroke-based Rendering on Mobile Devices
(The Eurographics Association, 2023) Wagner, Ronja; Schulz, Sebastian; Reimann, Max; Semmo, Amir; Döllner, Jürgen; Trapp, Matthias; Guthe, Michael; Grosch, Thorsten
This paper introduces an art-directable stroke-based rendering technique for transforming photos into painterly renditions on mobile devices. Unlike previous approaches that rely on time-consuming iterative computations and explicit brush-stroke geometry, our method offers a interactive image-based implementation tailored to the capabilities of modern mobile devices. The technique places curved brush strokes in multiple passes, leveraging a texture bombing algorithm. To maintain and highlight essential details for stylization, we incorporate additional information such as image salience, depth, and facial landmarks as parameters. Our technique enables a user to control and manipulate using a wide range of parameters and masks during editing to adjust and refine the stylized image. The result is an interactive painterly stylization tool that supports high-resolution input images, providing users with an immersive and engaging artistic experience on their mobile devices.
Improving NeRF Quality by Progressive Camera Placement for Free-Viewpoint Navigation
(The Eurographics Association, 2023) Kopanas, Georgios; Drettakis, George; Guthe, Michael; Grosch, Thorsten
Neural Radiance Fields, or NeRFs, have drastically improved novel view synthesis and 3D reconstruction for rendering. NeRFs achieve impressive results on object-centric reconstructions, but the quality of novel view synthesis with free-viewpoint navigation in complex environments (rooms, houses, etc) is often problematic. While algorithmic improvements play an important role in the resulting quality of novel view synthesis, in this work, we show that because optimizing a NeRF is inherently a data-driven process, good quality data play a fundamental role in the final quality of the reconstruction. As a consequence, it is critical to choose the data samples - in this case the cameras - in a way that will eventually allow the optimization to converge to a solution that allows free-viewpoint navigation with good quality. Our main contribution is an algorithm that efficiently proposes new camera placements that improve visual quality with minimal assumptions. Our solution can be used with any NeRF model and outperforms baselines and similar work.
Exploring Physical Latent Spaces for High-Resolution Flow Restoration
(The Eurographics Association, 2023) Paliard, Chloé; Thuerey, Nils; Um, Kiwon; Guthe, Michael; Grosch, Thorsten
We explore training deep neural network models in conjunction with physics simulations via partial differential equations (PDEs), using the simulated degrees of freedom as latent space for a neural network. In contrast to previous work, this paper treats the degrees of freedom of the simulated space purely as tools to be used by the neural network. We demonstrate this concept for learning reduced representations, as it is extremely challenging to faithfully preserve correct solutions over long time-spans with traditional reduced representations, particularly for solutions with large amounts of small scale features. This work focuses on the use of such physical, reduced latent space for the restoration of fine simulations, by training models that can modify the content of the reduced physical states as much as needed to best satisfy the learning objective. This autonomy allows the neural networks to discover alternate dynamics that significantly improve the performance in the given tasks. We demonstrate this concept for various fluid flows ranging from different turbulence scenarios to rising smoke plumes.
Weighted Laplacian Smoothing for Surface Reconstruction of Particle-based Fluids
(The Eurographics Association, 2023) Löschner, Fabian; Böttcher, Timna; Rhys Jeske, Stefan; Bender, Jan; Guthe, Michael; Grosch, Thorsten
In physically-based animation, producing detailed and realistic surface reconstructions for rendering is an important part of a simulation pipeline for particle-based fluids. In this paper we propose a post-processing approach to obtain smooth surfaces from ''blobby'' marching cubes triangulations without visual volume loss or shrinkage of drops and splashes. In contrast to other state-of-the-art methods that often require changes to the entire reconstruction pipeline our approach is easy to implement and less computationally expensive. The main component is Laplacian mesh smoothing with our proposed feature weights that dampen the smoothing in regions of the mesh with splashes and isolated particles without reducing effectiveness in regions that are supposed to be flat. In addition, we suggest a specialized decimation procedure to avoid artifacts due to low-quality triangle configurations generated by marching cubes and a normal smoothing pass to further increase quality when visualizing the mesh with physically-based rendering. For improved computational efficiency of the method, we outline the option of integrating computation of our weights into an existing reconstruction pipeline as most involved quantities are already known during reconstruction. Finally, we evaluate our post-processing implementation on high-resolution smoothed particle hydrodynamics (SPH) simulations.
Uncertain Stream Lines
(The Eurographics Association, 2023) Zimmermann, Janos; Motejat, Michael; Rössl, Christian; Theisel, Holger; Guthe, Michael; Grosch, Thorsten
We present a new approach for the visual representation of uncertain stream lines in vector field ensembles. While existing approaches rely on a particular seed point for the analysis of uncertain streamlines, our approach considers a whole stream line as seed structure. With this we ensure that uncertain stream lines are independent of the particular choice of seed point, and that uncertain stream lines have the same dimensionality as their certain counterparts in a single vector field. Assuming a Gaussian distribution of stream lines, we provide a visual representation of uncertain stream lines based on a mean map and a covariance map. The extension to uncertain path lines in ensembles of time-dependent vector fields is straightforward and is also introduced in the paper. We analyze properties, discuss discretization and performance issues, and apply the new technique to a number of flows ensembles.
MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition
(The Eurographics Association, 2023) Souza, Matheus; Heidrich, Wolfgang; Guthe, Michael; Grosch, Thorsten
Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. Each smartphone manufacturer has developed its ISPs with its own characteristic heuristics for improving the color rendition, for example, skin tones and other visually essential colors. The recent interest in replacing the historically grown ISP systems with deep-learned pipelines to match DSLR's image quality improves structural features in the image. However, these works ignore the superior color processing based on semantic scene analysis that distinguishes mobile phone ISPs from DSLRs. Here we present MetaISP, a single model designed to learn how to translate between the color and local contrast characteristics of different devices. MetaISP takes the RAW image from device A as input and translates it to RGB images that inherit the appearance characteristics of devices A, B, and C. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest. In this approach, we leverage novel attention mechanisms inspired by cross-covariance learn global scene semantics. Additionally, we make use of metadata that typically accompanies raw images, and we estimate scene illuminants when they are not available.
On the Beat: Analysing and Evaluating Synchronicity in Dance Performances
(The Eurographics Association, 2023) Menzel, Malte; Tauscher, Jan-Philipp; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
This paper presents a method to analyse and evaluate synchronicity in dance performances automatically. Synchronisation of a dancer's movement and the accompanying music is a vital characteristic of dance performances. We propose a method that fuses computer vision-based extraction of dancers' body pose information and audio beat tracking to examine the alignment of the dance motions with the background music. Specifically, the motion of the dancer is analysed for rhythmic dance movements that are then subsequently correlated to the musical beats of the soundtrack played during the performance. Using a single mobile phone video recording of a dance performance only, our system is easily usable in dance rehearsal contexts. Our method evaluates accuracy for every motion beat of the performance on a timeline giving users detailed insight into their performance. We evaluated the accuracy of our method using a dataset containing 17 video recordings of real world dance performances. Our results closely match assessments by professional dancers, indicating correct analysis by our method.
Out-of-Core Particle Tracing for Monte Carlo Rendering of Finite-Time Lyapunov Exponents
(The Eurographics Association, 2023) Grätz, Nicholas; Günther, Tobias; Guthe, Michael; Grosch, Thorsten
The motion in time-dependent fluid flows is governed by Lagrangian coherent structures (LCS). One common approach to visualize hyperbolic LCS is to extract and visualize the finite-time Lyapunov exponent. Its visualization on large time-dependent fluid flow is challenging for two reasons. First, the time steps needed for particle tracing do not necessarily fit at once into memory. And second, conventional ray marching exhibits artifacts when the FTLE ridges are sharp, which instead requires Monte Carlo volume rendering techniques to produce unbiased results. So far, these two problems have only been looked at in isolation. In this paper, we implement the first out-of-core Monte Carlo FTLE tracer, which is able to visualize the finitetime Lyapunov exponent field of time-dependent fluid flows that do not fit into main memory at once. To achieve this, we designed a data processing pipeline that alternates between two phases: a photon tracing phase and a particle tracing phase. We demonstrate and evaluate the approach on several large time-dependent vector fields.
Video-Driven Animation of Neural Head Avatars
(The Eurographics Association, 2023) Paier, Wolfgang; Hinzer, Paul; Hilsmann, Anna; Eisert, Peter; Guthe, Michael; Grosch, Thorsten
We present a new approach for video-driven animation of high-quality neural 3D head models, addressing the challenge of person-independent animation from video input. Typically, high-quality generative models are learned for specific individuals from multi-view video footage, resulting in person-specific latent representations that drive the generation process. In order to achieve person-independent animation from video input, we introduce an LSTM-based animation network capable of translating person-independent expression features into personalized animation parameters of person-specific 3D head models. Our approach combines the advantages of personalized head models (high quality and realism) with the convenience of video-driven animation employing multi-person facial performance capture.We demonstrate the effectiveness of our approach on synthesized animations with high quality based on different source videos as well as an ablation study.
Autonomous Particles for In-Situ-Friendly Flow Map Sampling
(The Eurographics Association, 2023) Wolligant, Steve; Rössl, Christian; Chi, Cheng; Thévenin, Dominique; Theisel, Holger; Guthe, Michael; Grosch, Thorsten
Computing and storing flow maps is a common approach to processing and analyzing large flow simulations in a Lagrangian way. Accurate Lagrangian-based visualizations require a good sampling of the flow map. We present an In-Situ-friendly flow map sampling strategy for flows using Autonomous Particles that do not need information of neighboring particles: they can be advected individually without knowing about each other. The main idea is to observe a linear neighborhood of a particle during advection. As soon as the neighborhood cannot be considered linear anymore, an adaptive splitting is performed. For observing the linear neighborhood, each particle is equipped with an ellipsoid that also gets advected by the flow. By splitting these ellipsoids into smaller ones in regions of non-linear behavior, critical and more interesting regions of the flow map are more densely sampled. Our sampling approach uses only forward integration and no adaptive integration from the past. This makes it applicable in and well-suited for in In-Situ environments. We compare our approach to existing sampling techniques and apply it to several artificial and real data sets.
Neural Fields for Interactive Visualization of Statistical Dependencies in 3D Simulation Ensembles
(The Eurographics Association, 2023) Farokhmanesh, Fatemeh; Höhlein, Kevin; Neuhauser, Christoph; Necker, Tobias; Weissmann, Martin; Miyoshi, Takemasa; Westermann, Rüdiger; Guthe, Michael; Grosch, Thorsten
We present neural dependence fields (NDFs) - the first neural network that learns to compactly represent and efficiently reconstruct the statistical dependencies between the values of physical variables at different spatial locations in large 3D simulation ensembles. Going beyond linear dependencies, we consider mutual information as an exemplary measure of non-linear dependence. We demonstrate learning and reconstruction with a large weather forecast ensemble comprising 1000 members, each storing multiple physical variables at a 250×352×20 simulation grid. By circumventing compute-intensive statistical estimators at runtime, we demonstrate significantly reduced memory and computation requirements for reconstructing the major dependence structures. This enables embedding the estimator into a GPU-accelerated direct volume renderer and interactively visualizing all mutual dependencies for a selected domain point.
N-SfC: Robust and Fast Shape Estimation from Caustic Images
(The Eurographics Association, 2023) Kassubeck, Marc; Kappel, Moritz; Castillo, Susana; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
This paper handles the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. While we focus our attention on inline shape reconstruction in glass fabrication processes, our methodology could be adapted to scenarios where the limiting factor is a lack of input measurements to constrain the reconstruction problem completely. The recent Shape from Caustics (SfC) method casts this problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practical application due to reconstruction speed and robustness. Thus, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension incorporating two components into the reconstruction pipeline: a denoising module, which both alleviates the light transport simulation cost, and also helps finding a better minimum; and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations. Extensive experiments demonstrate that we significantly outperform the current state-of-the-art in both computational speed and final surface error.
Semantic Image Abstraction using Panoptic Segmentation for Robotic Painting
(The Eurographics Association, 2023) Stroh, Michael; Gülzow, Jörg-Marvin; Deussen, Oliver; Guthe, Michael; Grosch, Thorsten
We propose a comprehensive pipeline for generating adaptable image abstractions from input pictures, tailored explicitly for robotic painting tasks. Our pipeline addresses several key objectives, including the ability to paint from background to foreground, maintain fine details, capture structured regions accurately, and highlight important objects. To achieve this, we employ a panoptic segmentation network to predict the semantic class membership for each pixel in the image. This step provides us with a detailed understanding of the object categories present in the scene. Building upon the semantic segmentation results, we combine them with a color-based image over-segmentation technique. This process partitions the image into monochromatic regions, each corresponding to a specific semantic object. Next, we construct a hierarchical tree based on the segmentation results, which allows us to merge adjacent regions based on their color difference and semantic class. We take care to ensure that shapes belonging to different semantic objects are not merged together. We iteratively perform adjacency merging until no further combinations are possible, resulting in a refined hierarchical shape tree. To obtain the desired image abstraction, we filter the hierarchical shape tree by examining factors such as color differences, relative sizes, and the layering within the hierarchy of each region in relation to their parent regions. By employing this approach, we can preserve fine details, apply local filtering operations, and effectively combine regions with structured shapes. This results in image abstractions well-suited for robotic painting applications and artistic renderings.
Perceptually Guided Automatic Parameter Optimization for Interactive Visualization
(The Eurographics Association, 2023) Opitz, Daniel; Zirr, Tobias; Dachsbacher, Carsten; Tessari, Lorenzo; Guthe, Michael; Grosch, Thorsten
We propose a new reference-free method for automatically optimizing the parameters of visualization techniques such that the perception of visual structures is improved. Manual tuning may require domain knowledge not only in the field of the analyzed data, but also deep knowledge of the visualization techniques, and thus often becomes challenging as the number of parameters that impact the result grows. To avoid this laborious and difficult task, we first derive an image metric that models the loss of perceived information in the processing of a displayed image by a human observer; good visualization parameters minimize this metric. Our model is loosely based on quantitative studies in the fields of perception and biology covering visual masking, photo receptor sensitivity, and local adaptation. We then pair our metric with a generic parameter tuning algorithm to arrive at an automatic optimization method that is oblivious to the concrete relationship between parameter sets and visualization. We demonstrate our method for several volume visualization techniques, where visual clutter, visibility of features, and illumination are often hard to balance. Since the metric can be efficiently computed using image transformations, it can be applied to many visualization techniques and problem settings in a unified manner, including continuous optimization during interactive visual exploration. We also evaluate the effectiveness of our approach in a user study that validates the improved perception of visual features in results optimized using our model of perception.
Factors Influencing Visual Comparison of Colored Directed Acyclic Graphs
(The Eurographics Association, 2023) Graniczkowska, Cynthia; Pelchmann, Laura; Landesberger, Tatiana von; Pohl, Margit; Guthe, Michael; Grosch, Thorsten
This paper presents a comprehensive investigation of the factors that influence visual comparison in colored node-link diagrams. We conducted a user study in which participants were asked to identify differences in pairs of directed acyclic graphs (DAGs) under time constraints. Previous studies focused on the perception of differences in node-link diagrams without coloring. Our results show that the individual coloring of nodes and edges significantly affects the detection of differences. We were able to confirm previous results, such as the influence of graph density, and also found that uniform coloring in certain areas of the graphs plays an important role in finding differences. Consequently, the results of this study hold potential for developing better comparative visualizations for diverse applications, such as finance or biology.
Digitizing Interlocking Building Blocks
(The Eurographics Association, 2023) Lieb, Sebastian; Thormählen, Thorsten; Rieger, Felix; Guthe, Michael; Grosch, Thorsten
Interlocking building blocks (such as LEGO®) are well-known toys and allow the creation of physical models of real objects or the design of imaginative 3D structures. In this paper, we propose a novel approach for digitizing building blocks with the original LEGO® form factor. We add a microprocessor to each 4 x 2 block, and the blocks communicate with each other via a two-wire connection provided in every nub. This poses the additional challenge that communication and power supply must use the same two-wire connection, which is addressed by alternating between the two modes over time. We introduce a protocol that checks for connections and propagates all connection information through the block network. We can then pass this information to a connected computer, which reconstructs the structure of the block network. We present several successfully digitized example configurations and discuss failure cases. Furthermore, two end-user scenarios are demonstrated, which show the potential of our approach as an intuitive human-computer interface.
Visual-assisted Outlier Preservation for Scatterplot Sampling
(The Eurographics Association, 2023) Yang, Haiyan; Pajarola, Renato; Guthe, Michael; Grosch, Thorsten
Scatterplot sampling has long been an efficient and effective way to resolve the overplotting issues commonly occurring in large-scale scatterplot visualization applications. However, it is challenging to preserve the existence of low-density points or outliers after sampling for a sub-sampling algorithm if, at the same time, faithfully representing the relative data densities is of importance. In this work, we propose to address this issue in a visual-assisted manner. While the whole dataset is sub-sampled, the density of the outliers is modeled and visually integrated into the final scatterplot together with the sub-sampled point data. We showcase the effectiveness of our proposed method in various cases and user studies.
Visually Analyzing Topic Change Points in Temporal Text Collections
(The Eurographics Association, 2023) Krause, Cedric; Rieger, Jonas; Flossdorf, Jonathan; Jentsch, Carsten; Beck, Fabian; Guthe, Michael; Grosch, Thorsten
Texts are collected over time and reflect temporal changes in the themes that they cover. While some changes might slowly evolve, other changes abruptly surface as explicit change points. In an application study for a change point extraction method based on a rolling Latent Dirichlet Allocation (LDA), we have developed a visualization approach that allows exploring such change points and related change patterns. Our visualization not only provides an overview of topics, but supports the detailed exploration of temporal developments. The interplay of general topic contents, development, and similarities with detected change points reveals rich insights into different kinds of change patterns. The approach comprises a combination of views including topic timeline representations with detected change points, comparative word clouds, and temporal similarity matrices. In an interactive exploration, these views adapt to selected topics, words, or points in time. We demonstrate the use cases of our approach in an in-depth application example involving statisticians.
Consistent SPH Rigid-Fluid Coupling
(The Eurographics Association, 2023) Bender, Jan; Westhofen, Lukas; Rhys Jeske, Stefan; Guthe, Michael; Grosch, Thorsten
A common way to handle boundaries in SPH fluid simulations is to sample the surface of the boundary geometry using particles. These boundary particles are assigned the same properties as the fluid particles and are considered in the pressure force computation to avoid a penetration of the boundary. However, the pressure solver requires a pressure value for each particle. These are typically not computed for the boundary particles due to the computational overhead. Therefore, several strategies have been investigated in previous works to obtain boundary pressure values. A popular, simple technique is pressure mirroring, which mirrors the values from the fluid particles. This method is efficient, but may cause visual artifacts. More complex approaches like pressure extrapolation aim to avoid these artifacts at the cost of computation time. We introduce a constraint-based derivation of Divergence-Free SPH (DFSPH) - a common state-of-the-art pressure solver. This derivation gives us new insights on how to integrate boundary particles in the pressure solve without the need of explicitly computing boundary pressure values. This yields a more elegant formulation of the pressure solver that avoids the aforementioned problems.
Leveraging BC6H Texture Compression and Filtering for Efficient Vector Field Visualization
(The Eurographics Association, 2023) Oehrl, Simon; Milke, Jan Frieder; Koenen, Jens; Kuhlen, Torsten W.; Gerrits, Tim; Guthe, Michael; Grosch, Thorsten
The steady advance of compute hardware is accompanied by an ever-steeper amount of data to be processed for visualization. Limited memory bandwidth provides a significant bottleneck to the runtime performance of visualization algorithms while limited video memory requires complex out-of-core loading techniques for rendering large datasets. Data compression methods aim to overcome these limitations, potentially at the cost of information loss. This work presents an approach to the compression of large data for flow visualization using the BC6H texture compression format natively supported, and therefore effortlessly leverageable, on modern GPUs. We assess the performance and accuracy of BC6H for compression of steady and unsteady vector fields and investigate its applicability to particle advection. The results indicate an improvement in memory utilization as well as runtime performance, at a cost of moderate loss in precision.

Browse

Browsing VMV2023 by Issue Date

Results Per Page

Sort Options