Browsing by Author "Magnor, Marcus"

Now showing 1 - 7 of 7
  • Thumbnail Image
    Item
    Automatic Infant Face Verification via Convolutional Neural Networks
    (The Eurographics Association, 2018) Wöhler, Leslie; Zhang, Hangjian; Albuquerque, Georgia; Magnor, Marcus; Beck, Fabian and Dachsbacher, Carsten and Sadlo, Filip
    In this paper, we investigate how convolutional neural networks (CNN) can learn to solve the verification task for faces of young children. One of the main issues of automatic face verification approaches is how to deal with facial changes resulting from aging. Since the facial shape and features change drastically during early childhood, the recognition of children can be challenging even for human observers. Therefore, we design CNNs that take two infant photographs as input and verify whether they belong to the same child. To specifically train our CNNs to recognize young children, we collect a new infant face dataset including 4,528 face images of 42 subjects in the age range of 0 to 6 years. Our results show an accuracy of up to 85 percent for face verification using our dataset with no overlapping subjects between the training and test data, and 72 percent in the FG-NET dataset for the age range from 0 to 4 years.
  • No Thumbnail Available
    Item
    Immersive Free‐Viewpoint Panorama Rendering from Omnidirectional Stereo Video
    (© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) Mühlhausen, Moritz; Kappel, Moritz; Kassubeck, Marc; Wöhler, Leslie; Grogorick, Steve; Castillo, Susana; Eisemann, Martin; Magnor, Marcus; Hauser, Helwig and Alliez, Pierre
    In this paper, we tackle the challenging problem of rendering real‐world 360° panorama videos that support full 6 degrees‐of‐freedom (DoF) head motion from a prerecorded omnidirectional stereo (ODS) video. In contrast to recent approaches that create novel views for individual panorama frames, we introduce a video‐specific temporally‐consistent multi‐sphere image (MSI) scene representation. Given a conventional ODS video, we first extract information by estimating framewise descriptive feature maps. Then, we optimize the global MSI model using theory from recent research on neural radiance fields. Instead of a continuous scene function, this multi‐sphere image (MSI) representation depicts colour and density information only for a discrete set of concentric spheres. To further improve the temporal consistency of our results, we apply an ancillary refinement step which optimizes the temporal coherency between successive video frames. Direct comparisons to recent baseline approaches show that our global MSI optimization yields superior performance in terms of visual quality. Our code and data will be made publicly available.
  • Thumbnail Image
    Item
    Learning a Perceptual Quality Metric for Correlation in Scatterplots
    (The Eurographics Association, 2019) Wöhler, Leslie; Zou, Yuxin; Mühlhausen, Moritz; Albuquerque, Georgia; Magnor, Marcus; Schulz, Hans-Jörg and Teschner, Matthias and Wimmer, Michael
    Visual quality metrics describe the quality and efficiency of multidimensional data visualizations in order to guide data analysts during exploration tasks. Current metrics are usually based on empirical algorithms which do not accurately represent human perception and therefore often differ from the analysts' expectations. We propose a new perception-based quality metric using deep learning that rates the correlation of data dimensions visualized by scatterplots. First, we created a data set containing over 15,000 pairs of scatterplots with human annotations on the perceived correlation between the data dimensions. Afterwards, we trained two different Convolutional Neural Networks (CNN), one extracts features from scatterplot images and the other directly from data vectors. We evaluated both CNNs on our test set and compared them to previous visual quality metrics. The experiments show that our new metric is able to represent human perception more accurately than previous methods.
  • Thumbnail Image
    Item
    N-SfC: Robust and Fast Shape Estimation from Caustic Images
    (The Eurographics Association, 2023) Kassubeck, Marc; Kappel, Moritz; Castillo, Susana; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
    This paper handles the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. While we focus our attention on inline shape reconstruction in glass fabrication processes, our methodology could be adapted to scenarios where the limiting factor is a lack of input measurements to constrain the reconstruction problem completely. The recent Shape from Caustics (SfC) method casts this problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practical application due to reconstruction speed and robustness. Thus, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension incorporating two components into the reconstruction pipeline: a denoising module, which both alleviates the light transport simulation cost, and also helps finding a better minimum; and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations. Extensive experiments demonstrate that we significantly outperform the current state-of-the-art in both computational speed and final surface error.
  • Thumbnail Image
    Item
    On the Beat: Analysing and Evaluating Synchronicity in Dance Performances
    (The Eurographics Association, 2023) Menzel, Malte; Tauscher, Jan-Philipp; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
    This paper presents a method to analyse and evaluate synchronicity in dance performances automatically. Synchronisation of a dancer's movement and the accompanying music is a vital characteristic of dance performances. We propose a method that fuses computer vision-based extraction of dancers' body pose information and audio beat tracking to examine the alignment of the dance motions with the background music. Specifically, the motion of the dancer is analysed for rhythmic dance movements that are then subsequently correlated to the musical beats of the soundtrack played during the performance. Using a single mobile phone video recording of a dance performance only, our system is easily usable in dance rehearsal contexts. Our method evaluates accuracy for every motion beat of the performance on a timeline giving users detailed insight into their performance. We evaluated the accuracy of our method using a dataset containing 17 video recordings of real world dance performances. Our results closely match assessments by professional dancers, indicating correct analysis by our method.
  • Thumbnail Image
    Item
    PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis
    (The Eurographics Association, 2023) Hahlbohm, Florian; Kappel, Moritz; Tauscher, Jan-Philipp; Eisemann, Martin; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
    This paper presents a point-based, neural rendering approach for complex real-world objects from a set of photographs. Our method is specifically geared towards representing fine detail and reflective surface characteristics at improved quality over current state-of-the-art methods. From the photographs, we create a 3D point model based on optimized neural feature points located on a regular grid. For rendering, we employ view-dependent spherical harmonics shading, differentiable rasterization, and a deep neural rendering network. By combining a point-based approach and novel regularizers, our method is able to accurately represent local detail such as fine geometry and high-frequency texture while at the same time convincingly interpolating unseen viewpoints during inference. Our method achieves about 7 frames per second at 800×800 pixel output resolution on commodity hardware, putting it within reach for real-time rendering applications.
  • Thumbnail Image
    Item
    Stereo Inverse Brightness Modulation for Guidance in Dynamic Panorama Videos in Virtual Reality
    (© 2020 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2020) Grogorick, Steve; Tauscher, Jan‐Philipp; Heesen, Nikkel; Castillo, Susana; Magnor, Marcus; Benes, Bedrich and Hauser, Helwig
    The peak of virtual reality offers new exciting possibilities for the creation of media content but also poses new challenges. Some areas of interest might be overlooked because the visual content fills up a large portion of viewers' visual field. Moreover, this content is available in 360° around the viewer, yielding locations completely out of sight, making, for example, recall or storytelling in cinematic Virtual Reality (VR) quite difficult.In this paper, we present an evaluation of Stereo Inverse Brightness Modulation for effective and subtle guidance of participants' attention while navigating dynamic virtual environments. The used technique exploits the binocular rivalry effect from human stereo vision and was previously shown to be effective in static environments. Moreover, we propose an extension of the method for successful guidance towards target locations outside the initial visual field.We conduct three perceptual studies, using 13 distinct panorama videos and two VR systems (a VR head mounted display and a fully immersive dome projection system), to investigate (1) general applicability to dynamic environments, (2) stimulus parameter and VR system influence, and (3) effectiveness of the proposed extension for out‐of‐sight targets. Our results prove the applicability of the method to dynamic environments while maintaining its unobtrusive appearance.