HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow

Simple item page
dc.contributor.authorWang, Jiayien_US
dc.contributor.authorLuvizon, Diogoen_US
dc.contributor.authorMueller, Franziskaen_US
dc.contributor.authorBernard, Florianen_US
dc.contributor.authorKortylewski, Adamen_US
dc.contributor.authorCasas, Danen_US
dc.contributor.authorTheobalt, Christianen_US
dc.contributor.editorBender, Janen_US
dc.contributor.editorBotsch, Marioen_US
dc.contributor.editorKeim, Daniel A.en_US
dc.date.accessioned2022-09-26T09:28:58Z
dc.date.available2022-09-26T09:28:58Z
dc.date.issued2022
dc.description.abstractReconstructing two-hand interactions from a single image is a challenging problem due to ambiguities that stem from projective geometry and heavy occlusions. Existing methods are designed to estimate only a single pose, despite the fact that there exist other valid reconstructions that fit the image evidence equally well. In this paper we propose to address this issue by explicitly modeling the distribution of plausible reconstructions in a conditional normalizing flow framework. This allows us to directly supervise the posterior distribution through a novel determinant magnitude regularization, which is key to varied 3D hand pose samples that project well into the input image. We also demonstrate that metrics commonly used to assess reconstruction quality are insufficient to evaluate pose predictions under such severe ambiguity. To address this, we release the first dataset with multiple plausible annotations per image called MultiHands. The additional annotations enable us to evaluate the estimated distribution using the maximum mean discrepancy metric. Through this, we demonstrate the quality of our probabilistic reconstruction and show that explicit ambiguity modeling is better-suited for this challenging problem.en_US
dc.description.sectionheadersSession IV
dc.description.seriesinformationVision, Modeling, and Visualization
dc.identifier.doi10.2312/vmv.20221209
dc.identifier.isbn978-3-03868-189-2
dc.identifier.pages99-106
dc.identifier.pages8 pages
dc.identifier.urihttps://doi.org/10.2312/vmv.20221209
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/vmv20221209
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies --> Tracking; Computer vision; Neural networks
dc.subjectComputing methodologies
dc.subjectTracking
dc.subjectComputer vision
dc.subjectNeural networks
dc.titleHandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flowen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
099-106.pdf
Size:
6.16 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1002_crc_mm.mp4
Size:
82.63 MB
Format:
Unknown data format
Download
Thumbnail Image
Name:
paper1002_supplemental_materials.pdf
Size:
9.66 MB
Format:
Adobe Portable Document Format
Download
Collections
VMV2022