Joint Hand and Object Pose Estimation from a Single RGB Image using High-level 2D Constraints

Song, Hao-Xuan; Mu, Tai-Jiang; Martin, Ralph R.

Joint Hand and Object Pose Estimation from a Single RGB Image using High-level 2D Constraints

Files

v41i7pp383-394.pdf (2 MB)

Date

2022

Authors

Song, Hao-Xuan
Mu, Tai-Jiang
Martin, Ralph R.

Publisher

The Eurographics Association and John Wiley & Sons Ltd.

Abstract

Joint pose estimation of human hands and objects from a single RGB image is an important topic for AR/VR, robot manipulation, etc. It is common practice to determine both poses directly from the image; some recent methods attempt to improve the initial poses using a variety of contact-based approaches. However, few methods take the real physical constraints conveyed by the image into consideration, leading to less realistic results than the initial estimates. To overcome this problem, we make use of a set of high-level 2D features which can be directly extracted from the image in a new pipeline which combines contact approaches and these constraints during optimization. Our pipeline achieves better results than direct regression or contactbased optimization: they are closer to the ground truth and provide high quality contact.

CCS Concepts: Computing methodologies → Reconstruction

        @article{10.1111:cgf.14685
,
journal = {Computer Graphics Forum},
title = {{Joint Hand and Object Pose Estimation from a Single RGB Image using High-level 2D Constraints
}},
author = {Song, Hao-Xuan and 
Mu, Tai-Jiang and 
Martin, Ralph R.
},
year = {2022
},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.
},
ISSN = {1467-8659
},
DOI = {10.1111/cgf.14685
}
}