Browsing by Author "Lee, Seungyong"

Now showing 1 - 7 of 7
  • No Thumbnail Available
    Item
    Defocus and Motion Blur Detection with Deep Contextual Features
    (The Eurographics Association and John Wiley & Sons Ltd., 2018) Kim, Beomseok; Son, Hyeongseok; Park, Seong-Jin; Cho, Sunghyun; Lee, Seungyong; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
    We propose a novel approach for detecting two kinds of partial blur, defocus and motion blur, by training a deep convolutional neural network. Existing blur detection methods concentrate on designing low-level features, but those features have difficulty in detecting blur in homogeneous regions without enough textures or edges. To handle such regions, we propose a deep encoder-decoder network with long residual skip-connections and multi-scale reconstruction loss functions to exploit high-level contextual features as well as low-level structural features. Another difficulty in partial blur detection is that there are no available datasets with images having both defocus and motion blur together, as most existing approaches concentrate only on either defocus or motion blur. To resolve this issue, we construct a synthetic dataset that consists of complex scenes with both types of blur. Experimental results show that our approach effectively detects and classifies blur, outperforming other state-of-the-art methods. Our method can be used for various applications, such as photo editing, blur magnification, and deblurring.
  • No Thumbnail Available
    Item
    Global Texture Mapping for Dynamic Objects
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Kim, Jungeon; Kim, Hyomin; Park, Jaesik; Lee, Seungyong; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    We propose a novel framework to generate a global texture atlas for a deforming geometry. Our approach distinguishes from prior arts in two aspects. First, instead of generating a texture map for each timestamp to color a dynamic scene, our framework reconstructs a global texture atlas that can be consistently mapped to a deforming object. Second, our approach is based on a single RGB-D camera, without the need of a multiple-camera setup surrounding a scene. In our framework, the input is a 3D template model with an RGB-D image sequence, and geometric warping fields are found using a state-of-the-art non-rigid registration method [GXW*15] to align the template mesh to noisy and incomplete input depth images. With these warping fields, our multi-scale approach for texture coordinate optimization generates a sharp and clear texture atlas that is consistent with multiple color observations over time. Our approach is accelerated by graphical hardware and provides a handy configuration to capture a dynamic geometry along with a clean texture atlas. We demonstrate our approach with practical scenarios, particularly human performance capture. We also show that our approach is resilient on misalignment issues caused by imperfect estimation of warping fields and inaccurate camera parameters.
  • Thumbnail Image
    Item
    Naturalness-Preserving Image Tone Enhancement Using Generative Adversarial Networks
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Son, Hyeongseok; Lee, Gunhee; Cho, Sunghyun; Lee, Seungyong; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    This paper proposes a deep learning-based image tone enhancement approach that can maximally enhance the tone of an image while preserving the naturalness. Our approach does not require carefully generated ground-truth images by human experts for training. Instead, we train a deep neural network to mimic the behavior of a previous classical filtering method that produces drastic but possibly unnatural-looking tone enhancement results. To preserve the naturalness, we adopt the generative adversarial network (GAN) framework as a regularizer for the naturalness. To suppress artifacts caused by the generative nature of the GAN framework, we also propose an imbalanced cycle-consistency loss. Experimental results show that our approach can effectively enhance the tone and contrast of an image while preserving the naturalness compared to previous state-of-the-art approaches.
  • Thumbnail Image
    Item
    Real-Time Video Deblurring via Lightweight Motion Compensation
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Son, Hyeongseok; Lee, Junyong; Cho, Sunghyun; Lee, Seungyong; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
    While motion compensation greatly improves video deblurring quality, separately performing motion compensation and video deblurring demands huge computational overhead. This paper proposes a real-time video deblurring framework consisting of a lightweight multi-task unit that supports both video deblurring and motion compensation in an efficient way. The multi-task unit is specifically designed to handle large portions of the two tasks using a single shared network and consists of a multi-task detail network and simple networks for deblurring and motion compensation. The multi-task unit minimizes the cost of incorporating motion compensation into video deblurring and enables real-time deblurring. Moreover, by stacking multiple multi-task units, our framework provides flexible control between the cost and deblurring quality. We experimentally validate the state-of-theart deblurring quality of our approach, which runs at a much faster speed compared to previous methods and show practical real-time performance (30.99dB@30fps measured on the DVD dataset).
  • No Thumbnail Available
    Item
    Semantic Reconstruction: Reconstruction of Semantically Segmented 3D Meshes via Volumetric Semantic Fusion
    (The Eurographics Association and John Wiley & Sons Ltd., 2018) Jeon, Junho; Jung, Jinwoong; Kim, Jungeon; Lee, Seungyong; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
    Semantic segmentation partitions a given image or 3D model of a scene into semantically meaning parts and assigns predetermined labels to the parts. With well-established datasets, deep networks have been successfully used for semantic segmentation of RGB and RGB-D images. On the other hand, due to the lack of annotated large-scale 3D datasets, semantic segmentation for 3D scenes has not yet been much addressed with deep learning. In this paper, we present a novel framework for generating semantically segmented triangular meshes of reconstructed 3D indoor scenes using volumetric semantic fusion in the reconstruction process. Our method integrates the results of CNN-based 2D semantic segmentation that is applied to the RGB-D stream used for dense surface reconstruction. To reduce the artifacts from noise and uncertainty of single-view semantic segmentation, we introduce adaptive integration for the volumetric semantic fusion and CRF-based semantic label regularization. With these methods, our framework can easily generate a high-quality triangular mesh of the reconstructed 3D scene with dense (i.e., per-vertex) semantic labels. Extensive experiments demonstrate that our semantic segmentation results of 3D scenes achieves the state-of-the-art performance compared to the previous voxel-based and point cloud-based methods.
  • Thumbnail Image
    Item
    Spatiotemporal Texture Reconstruction for Dynamic Objects Using a Single RGB-D Camera
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Kim, Hyomin; Kim, Jungeon; Nam, Hyeonseo; Park, Jaesik; Lee, Seungyong; Mitra, Niloy and Viola, Ivan
    This paper presents an effective method for generating a spatiotemporal (time-varying) texture map for a dynamic object using a single RGB-D camera. The input of our framework is a 3D template model and an RGB-D image sequence. Since there are invisible areas of the object at a frame in a single-camera setup, textures of such areas need to be borrowed from other frames. We formulate the problem as an MRF optimization and define cost functions to reconstruct a plausible spatiotemporal texture for a dynamic object. Experimental results demonstrate that our spatiotemporal textures can reproduce the active appearances of captured objects better than approaches using a single texture map.
  • No Thumbnail Available
    Item
    Structure‐Texture Decomposition of Images with Interval Gradient
    (© 2017 The Eurographics Association and John Wiley & Sons Ltd., 2017) Lee, Hyunjoon; Jeon, Junho; Kim, Junho; Lee, Seungyong; Chen, Min and Zhang, Hao (Richard)
    This paper presents a novel filtering‐based method for decomposing an image into structures and textures. Unlike previous filtering algorithms, our method adaptively smooths image gradients to filter out textures from images. A new gradient operator, the interval gradient, is proposed for adaptive gradient smoothing. Using interval gradients, textures can be distinguished from structure edges and smoothly varying shadings. We also propose an effective gradient‐guided algorithm to produce high‐quality image filtering results from filtered gradients. Our method avoids gradient reversal in the filtering results and preserves sharp features better than existing filtering approaches, while retaining simplicity and highly parallel implementation. The proposed method can be utilized for various applications that require accurate structure‐texture decomposition of images.This paper presents a novel filtering‐based method for decomposing an image into structures and textures. Unlike previous filtering algorithms, our method adaptively smooths image gradients to filter out textures from images. A new gradient operator, the interval gradient, is proposed for adaptive gradient smoothing. Using interval gradients, textures can be distinguished from structure edges and smoothly varying shadings. We also propose an effective gradient‐guided algorithm to produce high‐quality image filtering results from filtered gradients. Our method avoids gradient reversal in the filtering results and preserves sharp features better than existing filtering approaches, while retaining simplicity and highly parallel implementation.