VMV2023

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3543878

Browse

Now showing 1 - 20 of 27

Art-directable Stroke-based Rendering on Mobile Devices
(The Eurographics Association, 2023) Wagner, Ronja; Schulz, Sebastian; Reimann, Max; Semmo, Amir; Döllner, Jürgen; Trapp, Matthias; Guthe, Michael; Grosch, Thorsten
This paper introduces an art-directable stroke-based rendering technique for transforming photos into painterly renditions on mobile devices. Unlike previous approaches that rely on time-consuming iterative computations and explicit brush-stroke geometry, our method offers a interactive image-based implementation tailored to the capabilities of modern mobile devices. The technique places curved brush strokes in multiple passes, leveraging a texture bombing algorithm. To maintain and highlight essential details for stylization, we incorporate additional information such as image salience, depth, and facial landmarks as parameters. Our technique enables a user to control and manipulate using a wide range of parameters and masks during editing to adjust and refine the stylized image. The result is an interactive painterly stylization tool that supports high-resolution input images, providing users with an immersive and engaging artistic experience on their mobile devices.
Autonomous Particles for In-Situ-Friendly Flow Map Sampling
(The Eurographics Association, 2023) Wolligant, Steve; Rössl, Christian; Chi, Cheng; Thévenin, Dominique; Theisel, Holger; Guthe, Michael; Grosch, Thorsten
Computing and storing flow maps is a common approach to processing and analyzing large flow simulations in a Lagrangian way. Accurate Lagrangian-based visualizations require a good sampling of the flow map. We present an In-Situ-friendly flow map sampling strategy for flows using Autonomous Particles that do not need information of neighboring particles: they can be advected individually without knowing about each other. The main idea is to observe a linear neighborhood of a particle during advection. As soon as the neighborhood cannot be considered linear anymore, an adaptive splitting is performed. For observing the linear neighborhood, each particle is equipped with an ellipsoid that also gets advected by the flow. By splitting these ellipsoids into smaller ones in regions of non-linear behavior, critical and more interesting regions of the flow map are more densely sampled. Our sampling approach uses only forward integration and no adaptive integration from the past. This makes it applicable in and well-suited for in In-Situ environments. We compare our approach to existing sampling techniques and apply it to several artificial and real data sets.
Consistent SPH Rigid-Fluid Coupling
(The Eurographics Association, 2023) Bender, Jan; Westhofen, Lukas; Rhys Jeske, Stefan; Guthe, Michael; Grosch, Thorsten
A common way to handle boundaries in SPH fluid simulations is to sample the surface of the boundary geometry using particles. These boundary particles are assigned the same properties as the fluid particles and are considered in the pressure force computation to avoid a penetration of the boundary. However, the pressure solver requires a pressure value for each particle. These are typically not computed for the boundary particles due to the computational overhead. Therefore, several strategies have been investigated in previous works to obtain boundary pressure values. A popular, simple technique is pressure mirroring, which mirrors the values from the fluid particles. This method is efficient, but may cause visual artifacts. More complex approaches like pressure extrapolation aim to avoid these artifacts at the cost of computation time. We introduce a constraint-based derivation of Divergence-Free SPH (DFSPH) - a common state-of-the-art pressure solver. This derivation gives us new insights on how to integrate boundary particles in the pressure solve without the need of explicitly computing boundary pressure values. This yields a more elegant formulation of the pressure solver that avoids the aforementioned problems.
Digitizing Interlocking Building Blocks
(The Eurographics Association, 2023) Lieb, Sebastian; Thormählen, Thorsten; Rieger, Felix; Guthe, Michael; Grosch, Thorsten
Interlocking building blocks (such as LEGO®) are well-known toys and allow the creation of physical models of real objects or the design of imaginative 3D structures. In this paper, we propose a novel approach for digitizing building blocks with the original LEGO® form factor. We add a microprocessor to each 4 x 2 block, and the blocks communicate with each other via a two-wire connection provided in every nub. This poses the additional challenge that communication and power supply must use the same two-wire connection, which is addressed by alternating between the two modes over time. We introduce a protocol that checks for connections and propagates all connection information through the block network. We can then pass this information to a connected computer, which reconstructs the structure of the block network. We present several successfully digitized example configurations and discuss failure cases. Furthermore, two end-user scenarios are demonstrated, which show the potential of our approach as an intuitive human-computer interface.
Exploring Physical Latent Spaces for High-Resolution Flow Restoration
(The Eurographics Association, 2023) Paliard, Chloé; Thuerey, Nils; Um, Kiwon; Guthe, Michael; Grosch, Thorsten
We explore training deep neural network models in conjunction with physics simulations via partial differential equations (PDEs), using the simulated degrees of freedom as latent space for a neural network. In contrast to previous work, this paper treats the degrees of freedom of the simulated space purely as tools to be used by the neural network. We demonstrate this concept for learning reduced representations, as it is extremely challenging to faithfully preserve correct solutions over long time-spans with traditional reduced representations, particularly for solutions with large amounts of small scale features. This work focuses on the use of such physical, reduced latent space for the restoration of fine simulations, by training models that can modify the content of the reduced physical states as much as needed to best satisfy the learning objective. This autonomy allows the neural networks to discover alternate dynamics that significantly improve the performance in the given tasks. We demonstrate this concept for various fluid flows ranging from different turbulence scenarios to rising smoke plumes.
Factors Influencing Visual Comparison of Colored Directed Acyclic Graphs
(The Eurographics Association, 2023) Graniczkowska, Cynthia; Pelchmann, Laura; Landesberger, Tatiana von; Pohl, Margit; Guthe, Michael; Grosch, Thorsten
This paper presents a comprehensive investigation of the factors that influence visual comparison in colored node-link diagrams. We conducted a user study in which participants were asked to identify differences in pairs of directed acyclic graphs (DAGs) under time constraints. Previous studies focused on the perception of differences in node-link diagrams without coloring. Our results show that the individual coloring of nodes and edges significantly affects the detection of differences. We were able to confirm previous results, such as the influence of graph density, and also found that uniform coloring in certain areas of the graphs plays an important role in finding differences. Consequently, the results of this study hold potential for developing better comparative visualizations for diverse applications, such as finance or biology.
Greedy Image Approximation for Artwork Generation via Contiguous Bézier Segments
(The Eurographics Association, 2023) Nehring-Wirxel, Julius; Lim, Isaak; Kobbelt, Leif; Guthe, Michael; Grosch, Thorsten
The automatic creation of digital art has a long history in computer graphics. In this work, we focus on approximating input images to mimic artwork by the artist Kumi Yamashita, as well as the popular scribble art style. Both have in common that the artists create the works by using a single, contiguous thread (Yamashita) or stroke (scribble) that is placed seemingly at random when viewed at close range, but perceived as a tone-mapped picture when viewed from a distance. Our approach takes a rasterized image as input and creates a single, connected path by iteratively sampling a set of candidate segments that extend the current path and greedily selecting the best one. The candidates are sampled according to art style specific constraints, i.e. conforming to continuity constraints in the mathematical sense for the scribble art style. To model the perceptual discrepancy between close and far viewing distances, we minimize the difference between the input image and the image created by rasterizing our path after applying the contrast sensitivity function, which models how human vision blurs images when viewed from a distance. Our approach generalizes to colored images by using one path per color. We evaluate our approach on a wide range of input images and show that it is able to achieve good results for both art styles in grayscale and color.
Improving NeRF Quality by Progressive Camera Placement for Free-Viewpoint Navigation
(The Eurographics Association, 2023) Kopanas, Georgios; Drettakis, George; Guthe, Michael; Grosch, Thorsten
Neural Radiance Fields, or NeRFs, have drastically improved novel view synthesis and 3D reconstruction for rendering. NeRFs achieve impressive results on object-centric reconstructions, but the quality of novel view synthesis with free-viewpoint navigation in complex environments (rooms, houses, etc) is often problematic. While algorithmic improvements play an important role in the resulting quality of novel view synthesis, in this work, we show that because optimizing a NeRF is inherently a data-driven process, good quality data play a fundamental role in the final quality of the reconstruction. As a consequence, it is critical to choose the data samples - in this case the cameras - in a way that will eventually allow the optimization to converge to a solution that allows free-viewpoint navigation with good quality. Our main contribution is an algorithm that efficiently proposes new camera placements that improve visual quality with minimal assumptions. Our solution can be used with any NeRF model and outperforms baselines and similar work.
Interactions for Seamlessly Coupled Exploration of High-Dimensional Images and Hierarchical Embeddings
(The Eurographics Association, 2023) Vieth, Alexander; Lelieveldt, Boudewijn; Eisemann, Elmar; Vilanova, Anna; Höllt, Thomas; Guthe, Michael; Grosch, Thorsten
High-dimensional images (i.e., with many attributes per pixel) are commonly acquired in many domains, such as geosciences or systems biology. The spatial and attribute information of such data are typically explored separately, e.g., by using coordinated views of an image representation and a low-dimensional embedding of the high-dimensional attribute data. Facing ever growing image data sets, hierarchical dimensionality reduction techniques lend themselves to overcome scalability issues. However, current embedding methods do not provide suitable interactions to reflect image space exploration. Specifically, it is not possible to adjust the level of detail in the embedding hierarchy to reflect changing level of detail in image space stemming from navigation such as zooming and panning. In this paper, we propose such a mapping from image navigation interactions to embedding space adjustments. We show how our mapping applies the "overview first, details-on-demand" characteristic inherent to image exploration in the high-dimensional attribute space. We compare our strategy with regular hierarchical embedding technique interactions and demonstrate the advantages of linking image and embedding interactions through a representative use case.
Leveraging BC6H Texture Compression and Filtering for Efficient Vector Field Visualization
(The Eurographics Association, 2023) Oehrl, Simon; Milke, Jan Frieder; Koenen, Jens; Kuhlen, Torsten W.; Gerrits, Tim; Guthe, Michael; Grosch, Thorsten
The steady advance of compute hardware is accompanied by an ever-steeper amount of data to be processed for visualization. Limited memory bandwidth provides a significant bottleneck to the runtime performance of visualization algorithms while limited video memory requires complex out-of-core loading techniques for rendering large datasets. Data compression methods aim to overcome these limitations, potentially at the cost of information loss. This work presents an approach to the compression of large data for flow visualization using the BC6H texture compression format natively supported, and therefore effortlessly leverageable, on modern GPUs. We assess the performance and accuracy of BC6H for compression of steady and unsteady vector fields and investigate its applicability to particle advection. The results indicate an improvement in memory utilization as well as runtime performance, at a cost of moderate loss in precision.
MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition
(The Eurographics Association, 2023) Souza, Matheus; Heidrich, Wolfgang; Guthe, Michael; Grosch, Thorsten
Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. Each smartphone manufacturer has developed its ISPs with its own characteristic heuristics for improving the color rendition, for example, skin tones and other visually essential colors. The recent interest in replacing the historically grown ISP systems with deep-learned pipelines to match DSLR's image quality improves structural features in the image. However, these works ignore the superior color processing based on semantic scene analysis that distinguishes mobile phone ISPs from DSLRs. Here we present MetaISP, a single model designed to learn how to translate between the color and local contrast characteristics of different devices. MetaISP takes the RAW image from device A as input and translates it to RGB images that inherit the appearance characteristics of devices A, B, and C. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest. In this approach, we leverage novel attention mechanisms inspired by cross-covariance learn global scene semantics. Additionally, we make use of metadata that typically accompanies raw images, and we estimate scene illuminants when they are not available.
N-SfC: Robust and Fast Shape Estimation from Caustic Images
(The Eurographics Association, 2023) Kassubeck, Marc; Kappel, Moritz; Castillo, Susana; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
This paper handles the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. While we focus our attention on inline shape reconstruction in glass fabrication processes, our methodology could be adapted to scenarios where the limiting factor is a lack of input measurements to constrain the reconstruction problem completely. The recent Shape from Caustics (SfC) method casts this problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practical application due to reconstruction speed and robustness. Thus, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension incorporating two components into the reconstruction pipeline: a denoising module, which both alleviates the light transport simulation cost, and also helps finding a better minimum; and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations. Extensive experiments demonstrate that we significantly outperform the current state-of-the-art in both computational speed and final surface error.
Neural Fields for Interactive Visualization of Statistical Dependencies in 3D Simulation Ensembles
(The Eurographics Association, 2023) Farokhmanesh, Fatemeh; Höhlein, Kevin; Neuhauser, Christoph; Necker, Tobias; Weissmann, Martin; Miyoshi, Takemasa; Westermann, Rüdiger; Guthe, Michael; Grosch, Thorsten
We present neural dependence fields (NDFs) - the first neural network that learns to compactly represent and efficiently reconstruct the statistical dependencies between the values of physical variables at different spatial locations in large 3D simulation ensembles. Going beyond linear dependencies, we consider mutual information as an exemplary measure of non-linear dependence. We demonstrate learning and reconstruction with a large weather forecast ensemble comprising 1000 members, each storing multiple physical variables at a 250×352×20 simulation grid. By circumventing compute-intensive statistical estimators at runtime, we demonstrate significantly reduced memory and computation requirements for reconstructing the major dependence structures. This enables embedding the estimator into a GPU-accelerated direct volume renderer and interactively visualizing all mutual dependencies for a selected domain point.
On the Beat: Analysing and Evaluating Synchronicity in Dance Performances
(The Eurographics Association, 2023) Menzel, Malte; Tauscher, Jan-Philipp; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
This paper presents a method to analyse and evaluate synchronicity in dance performances automatically. Synchronisation of a dancer's movement and the accompanying music is a vital characteristic of dance performances. We propose a method that fuses computer vision-based extraction of dancers' body pose information and audio beat tracking to examine the alignment of the dance motions with the background music. Specifically, the motion of the dancer is analysed for rhythmic dance movements that are then subsequently correlated to the musical beats of the soundtrack played during the performance. Using a single mobile phone video recording of a dance performance only, our system is easily usable in dance rehearsal contexts. Our method evaluates accuracy for every motion beat of the performance on a timeline giving users detailed insight into their performance. We evaluated the accuracy of our method using a dataset containing 17 video recordings of real world dance performances. Our results closely match assessments by professional dancers, indicating correct analysis by our method.
Optimizing Temporal Stability in Underwater Video Tone Mapping
(The Eurographics Association, 2023) Franz, Matthias; Thang, B. Matthias; Sackhoff, Pascal; Scholz, Timon; Möller, Jannis; Grogorick, Steve; Eisemann, Martin; Guthe, Michael; Grosch, Thorsten
In this paper, we present an approach for temporal stabilization of depth-based underwater image tone mapping methods for application to monocular RGB video. Typically, the goal is to improve the colors of focused objects, while leaving more distant regions nearly unchanged, to preserve the underwater look-and-feel of the overall image. To do this, many methods rely on estimated depth to control the recolorization process, i.e., to enhance colors (reduce blue tint) only for objects close to the camera. However, while single-view depth estimation is usually consistent within a frame, it often suffers from inconsistencies across sequential frames, resulting in color fluctuations during tone mapping. We propose a simple yet effective inter-frame stabilization of the computed depth maps to achieve stable tone mapping results. The evaluation of eight test sequences shows the effectiveness in a wide range of underwater scenarios.
Out-of-Core Particle Tracing for Monte Carlo Rendering of Finite-Time Lyapunov Exponents
(The Eurographics Association, 2023) Grätz, Nicholas; Günther, Tobias; Guthe, Michael; Grosch, Thorsten
The motion in time-dependent fluid flows is governed by Lagrangian coherent structures (LCS). One common approach to visualize hyperbolic LCS is to extract and visualize the finite-time Lyapunov exponent. Its visualization on large time-dependent fluid flow is challenging for two reasons. First, the time steps needed for particle tracing do not necessarily fit at once into memory. And second, conventional ray marching exhibits artifacts when the FTLE ridges are sharp, which instead requires Monte Carlo volume rendering techniques to produce unbiased results. So far, these two problems have only been looked at in isolation. In this paper, we implement the first out-of-core Monte Carlo FTLE tracer, which is able to visualize the finitetime Lyapunov exponent field of time-dependent fluid flows that do not fit into main memory at once. To achieve this, we designed a data processing pipeline that alternates between two phases: a photon tracing phase and a particle tracing phase. We demonstrate and evaluate the approach on several large time-dependent vector fields.
Perceptually Guided Automatic Parameter Optimization for Interactive Visualization
(The Eurographics Association, 2023) Opitz, Daniel; Zirr, Tobias; Dachsbacher, Carsten; Tessari, Lorenzo; Guthe, Michael; Grosch, Thorsten
We propose a new reference-free method for automatically optimizing the parameters of visualization techniques such that the perception of visual structures is improved. Manual tuning may require domain knowledge not only in the field of the analyzed data, but also deep knowledge of the visualization techniques, and thus often becomes challenging as the number of parameters that impact the result grows. To avoid this laborious and difficult task, we first derive an image metric that models the loss of perceived information in the processing of a displayed image by a human observer; good visualization parameters minimize this metric. Our model is loosely based on quantitative studies in the fields of perception and biology covering visual masking, photo receptor sensitivity, and local adaptation. We then pair our metric with a generic parameter tuning algorithm to arrive at an automatic optimization method that is oblivious to the concrete relationship between parameter sets and visualization. We demonstrate our method for several volume visualization techniques, where visual clutter, visibility of features, and illumination are often hard to balance. Since the metric can be efficiently computed using image transformations, it can be applied to many visualization techniques and problem settings in a unified manner, including continuous optimization during interactive visual exploration. We also evaluate the effectiveness of our approach in a user study that validates the improved perception of visual features in results optimized using our model of perception.
PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis
(The Eurographics Association, 2023) Hahlbohm, Florian; Kappel, Moritz; Tauscher, Jan-Philipp; Eisemann, Martin; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
This paper presents a point-based, neural rendering approach for complex real-world objects from a set of photographs. Our method is specifically geared towards representing fine detail and reflective surface characteristics at improved quality over current state-of-the-art methods. From the photographs, we create a 3D point model based on optimized neural feature points located on a regular grid. For rendering, we employ view-dependent spherical harmonics shading, differentiable rasterization, and a deep neural rendering network. By combining a point-based approach and novel regularizers, our method is able to accurately represent local detail such as fine geometry and high-frequency texture while at the same time convincingly interpolating unseen viewpoints during inference. Our method achieves about 7 frames per second at 800×800 pixel output resolution on commodity hardware, putting it within reach for real-time rendering applications.
Ray Tracing Spherical Harmonics Glyphs
(The Eurographics Association, 2023) Peters, Christoph; Patel, Tark; Usher, Will; Johnson, Chris R.; Guthe, Michael; Grosch, Thorsten
Spherical harmonics glyphs are an established way to visualize high angular resolution diffusion imaging data. Starting from a unit sphere, each point on the surface is scaled according to the value of a linear combination of spherical harmonics basis functions. The resulting glyph visualizes an orientation distribution function. We present an efficient method to render these glyphs using ray tracing. Our method constructs a polynomial whose roots correspond to ray-glyph intersections. This polynomial has degree 2k+2 for spherical harmonics bands 0;2; : : : ; k. We then find all intersections in an efficient and numerically stable fashion through polynomial root finding. Our formulation also gives rise to a simple formula for normal vectors of the glyph. Additionally, we compute a nearly exact axis-aligned bounding box to make ray tracing of these glyphs even more efficient. Since our method finds all intersections for arbitrary rays, it lets us perform sophisticated shading and uncertainty visualization. Compared to prior work, it is faster, more flexible and more accurate.
Semantic Image Abstraction using Panoptic Segmentation for Robotic Painting
(The Eurographics Association, 2023) Stroh, Michael; Gülzow, Jörg-Marvin; Deussen, Oliver; Guthe, Michael; Grosch, Thorsten
We propose a comprehensive pipeline for generating adaptable image abstractions from input pictures, tailored explicitly for robotic painting tasks. Our pipeline addresses several key objectives, including the ability to paint from background to foreground, maintain fine details, capture structured regions accurately, and highlight important objects. To achieve this, we employ a panoptic segmentation network to predict the semantic class membership for each pixel in the image. This step provides us with a detailed understanding of the object categories present in the scene. Building upon the semantic segmentation results, we combine them with a color-based image over-segmentation technique. This process partitions the image into monochromatic regions, each corresponding to a specific semantic object. Next, we construct a hierarchical tree based on the segmentation results, which allows us to merge adjacent regions based on their color difference and semantic class. We take care to ensure that shapes belonging to different semantic objects are not merged together. We iteratively perform adjacency merging until no further combinations are possible, resulting in a refined hierarchical shape tree. To obtain the desired image abstraction, we filter the hierarchical shape tree by examining factors such as color differences, relative sizes, and the layering within the hierarchy of each region in relation to their parent regions. By employing this approach, we can preserve fine details, apply local filtering operations, and effectively combine regions with structured shapes. This results in image abstractions well-suited for robotic painting applications and artistic renderings.

Browse

Browsing VMV2023 by Title

Results Per Page

Sort Options