38-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2632813

Browse

Browsing 38-Issue 7 by Subject "Computational photography"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Light Field Video Compression and Real Time Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Hajisharif, Saghi; Miandji, Ehsan; Larsson, Per; Tran, Kiet; Unger, Jonas; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    Light field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post-capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real-time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.
  • Thumbnail Image
    Item
    ShutterApp: Spatio-temporal Exposure Control for Videos
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Salamon, Nestor; Billeter, Markus; Eisemann, Elmar; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    A camera's shutter controls the incoming light that is reaching the camera sensor. Different shutters lead to wildly different results, and are often used as a tool in movies for artistic purpose, e.g., they can indirectly control the effect of motion blur. However, a physical camera is limited to a single shutter setting at any given moment. ShutterApp enables users to define spatio-temporally-varying virtual shutters that go beyond the options available in real-world camera systems. A user provides a sparse set of annotations that define shutter functions at selected locations in key frames. From this input, our solution defines shutter functions for each pixel of the video sequence using a suitable interpolation technique, which are then employed to derive the output video. Our solution performs in real-time on commodity hardware. Hereby, users can explore different options interactively, leading to a new level of expressiveness without having to rely on specialized hardware or laborious editing.
  • Thumbnail Image
    Item
    Wavelet Flow: Optical Flow Guided Wavelet Facial Image Fusion
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Ding, Hong; Yan, Qingan; Fu, Gang; Xiao, Chunxia; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    Estimating the correspondence between the images using optical flow is the key component for image fusion, however, computing optical flow between a pair of facial images including backgrounds is challenging due to large differences in illumination, texture, color and background in the images. To improve optical flow results for image fusion, we propose a novel flow estimation method, wavelet flow, which can handle both the face and background in the input images. The key idea is that instead of computing flow directly between the input image pair, we estimate the image flow by incorporating multi-scale image transfer and optical flow guided wavelet fusion. Multi-scale image transfer helps to preserve the background and lighting detail of input, while optical flow guided wavelet fusion produces a series of intermediate images for further fusion quality optimizing. Our approach can significantly improve the performance of the optical flow algorithm and provide more natural fusion results for both faces and backgrounds in the images. We evaluate our method on a variety of datasets to show its high outperformance.