Browsing by Author "Yang, Jingyu"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Generating 3D Faces using Multi-column Graph Convolutional Networks
    (The Eurographics Association and John Wiley & Sons Ltd., 2019) Li, Kun; Liu, Jingying; Lai, Yu-Kun; Yang, Jingyu; Lee, Jehee and Theobalt, Christian and Wetzstein, Gordon
    In this work, we introduce multi-column graph convolutional networks (MGCNs), a deep generative model for 3D mesh surfaces that effectively learns a non-linear facial representation. We perform spectral decomposition of meshes and apply convolutions directly in the frequency domain. Our network architecture involves multiple columns of graph convolutional networks (GCNs), namely large GCN (L-GCN), medium GCN (M-GCN) and small GCN (S-GCN), with different filter sizes to extract features at different scales. L-GCN is more useful to extract large-scale features, whereas S-GCN is effective for extracting subtle and fine-grained features, and M-GCN captures information in between. Therefore, to obtain a high-quality representation, we propose a selective fusion method that adaptively integrates these three kinds of information. Spatially non-local relationships are also exploited through a self-attention mechanism to further improve the representation ability in the latent vector space. Through extensive experiments, we demonstrate the superiority of our end-to-end framework in improving the accuracy of 3D face reconstruction. Moreover, with the help of variational inference, our model has excellent generating ability.
  • No Thumbnail Available
    Item
    Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images
    (The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Kun; Jiao, Nianhong; Liu, Yebin; Wang, Yangang; Yang, Jingyu; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
    Multi-person pose and shape estimation is very challenging, especially when the persons have close interactions. Existing methods only work well when people are well spaced out in the captured images. However, close interaction among people is very common in real life, which is more challenge due to complex articulation, frequent occlusion and inherent ambiguities. We present a fully-automatic markerless motion capture method to simultaneously estimate 3D poses and shapes of closely interacting people from multi-view sequences. We first predict the 2D joints for each person in an image, and then design a spatio-temporal tracker for multi-person pose tracking based on multi-view videos. Finally, we estimate 3D poses and shapes of all the persons with multi-view constraints using a skinned multi-person linear model (SMPL). Experimental results demonstrate that our method achieves fast but accurate pose and shape estimation results for multi-person close interaction cases. Compared with existing methods, our method does not need pre-segmentation for each person and manual intervention, which greatly reduces the complexity of the system including time complexity and system processing complexity.
  • Thumbnail Image
    Item
    SHREC 2020 Track: Non-rigid Shape Correspondence of Physically-Based Deformations
    (The Eurographics Association, 2020) Dyke, Roberto M.; Zhou, Feng; Lai, Yu-Kun; Rosin, Paul L.; Guo, Daoliang; Li, Kun; Marin, Riccardo; Yang, Jingyu; Schreck, Tobias and Theoharis, Theoharis and Pratikakis, Ioannis and Spagnuolo, Michela and Veltkamp, Remco C.
    Commonly, novel non-rigid shape correspondence techniques focus on particular matching challenges. This can lead to the potential trade-off of poorer performance in other scenarios. An ideal dataset would provide a granular means for degrees of evaluation. In this paper, we propose a novel dataset of real scans that contain challenging non-isometric deformations to evaluate non-rigid point-to-point correspondence and registration algorithms. The deformations included in our dataset cover extreme types of physically-based contortions of a toy rabbit. Furthermore, shape pairs contain incrementally different types and amounts of deformation, this enables performance to be systematically evaluated with respect to the nature of the deformation. A brief investigation into different methods for initialising correspondence was undertaken, and a series of experiments were subsequently conducted to investigate the performance of state-of-the-art methods on the proposed dataset. We find that methods that rely on initial correspondences and local descriptors that are sensitive to local surface changes perform poorly in comparison to other strategies, and that a template-based approach performs the best.