Browsing by Author "Hu, Shi-Min"

Now showing 1 - 4 of 4
  • No Thumbnail Available
    Item
    Deep Video Stabilization Using Adversarial Networks
    (The Eurographics Association and John Wiley & Sons Ltd., 2018) Xu, Sen-Zhe; Hu, Jun; Wang, Miao; Mu, Tai-Jiang; Hu, Shi-Min; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
    Video stabilization is necessary for many hand-held shot videos. In the past decades, although various video stabilization methods were proposed based on the smoothing of 2D, 2.5D or 3D camera paths, hardly have there been any deep learning methods to solve this problem. Instead of explicitly estimating and smoothing the camera path, we present a novel online deep learning framework to learn the stabilization transformation for each unsteady frame, given historical steady frames. Our network is composed of a generative network with spatial transformer networks embedded in different layers, and generates a stable frame for the incoming unstable frame by computing an appropriate affine transformation. We also introduce an adversarial network to determine the stability of a piece of video. The network is trained directly using the pair of steady and unsteady videos. Experiments show that our method can produce similar results as traditional methods, moreover, it is capable of handling challenging unsteady video of low quality, where traditional methods fail, such as video with heavy noise or multiple exposures. Our method runs in real time, which is much faster than traditional methods.
  • Thumbnail Image
    Item
    A Divergence-free Mixture Model for Multiphase Fluids
    (The Eurographics Association and John Wiley & Sons Ltd., 2020) Jiang, Yuntao; Li, Chenfeng; Deng, Shujie; Hu, Shi-Min; Bender, Jan and Popa, Tiberiu
    We present a novel divergence free mixture model for multiphase flows and the related fluid-solid coupling. The new mixture model is built upon a volume-weighted mixture velocity so that the divergence free condition is satisfied for miscible and immiscible multiphase fluids. The proposed mixture velocity can be solved efficiently by adapted single phase incompressible solvers, allowing for larger time steps and smaller volume deviations. Besides, the drift velocity formulation is corrected to ensure mass conservation during the simulation. The new approach increases the accuracy of multiphase fluid simulation by several orders. The capability of the new divergence-free mixture model is demonstrated by simulating different multiphase flow phenomena including mixing and unmixing of multiple fluids, fluid-solid coupling involving deformable solids and granular materials.
  • Thumbnail Image
    Item
    Learning Explicit Smoothing Kernels for Joint Image Filtering
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Fang, Xiaonan; Wang, Miao; Shamir, Ariel; Hu, Shi-Min; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    Smoothing noises while preserving strong edges in images is an important problem in image processing. Image smoothing filters can be either explicit (based on local weighted average) or implicit (based on global optimization). Implicit methods are usually time-consuming and cannot be applied to joint image filtering tasks, i.e., leveraging the structural information of a guidance image to filter a target image.Previous deep learning based image smoothing filters are all implicit and unavailable for joint filtering. In this paper, we propose to learn explicit guidance feature maps as well as offset maps from the guidance image and smoothing parameter that can be utilized to smooth the input itself or to filter images in other target domains. We design a deep convolutional neural network consisting of a fully-convolution block for guidance and offset maps extraction together with a stacked spatially varying deformable convolution block for joint image filtering. Our models can approximate several representative image smoothing filters with high accuracy comparable to state-of-the-art methods, and serve as general tools for other joint image filtering tasks, such as color interpolation, depth map upsampling, saliency map upsampling, flash/non-flash image denoising and RGB/NIR image denoising.
  • Thumbnail Image
    Item
    A Rigging-Skinning Scheme to Control Fluid Simulation
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Lu, Jia-Ming; Chen, Xiao-Song; Yan, Xiao; Li, Chen-Feng; Lin, Ming; Hu, Shi-Min; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    Inspired by skeletal animation, a novel rigging-skinning flow control scheme is proposed to animate fluids intuitively and efficiently. The new animation pipeline creates fluid animation via two steps: fluid rigging and fluid skinning. The fluid rig is defined by a point cloud with rigid-body movement and incompressible deformation, whose time series can be intuitively specified by a rigid body motion and a constrained free-form deformation, respectively. The fluid skin generates plausible fluid flows by virtually fluidizing the point-cloud fluid rig with adjustable zero- and first-order flow features and at fixed computational cost. Fluid rigging allows the animator to conveniently specify the desired low-frequency flow motion through intuitive manipulations of a point cloud, while fluid skinning truthfully and efficiently converts the motion specified on the fluid rig into plausible flows of the animation fluid, with adjustable fine-scale effects. Besides being intuitive, the rigging-skinning scheme for fluid animation is robust and highly efficient, avoiding completely iterative trials or time-consuming nonlinear optimization. It is also versatile, supporting both particle- and grid- based fluid solvers. A series of examples including liquid, gas and mixed scenes are presented to demonstrate the performance of the new animation pipeline.