Personalized Visual Dubbing through Virtual Dubber and Full Head Reenactment

Simple item page
dc.contributor.authorJeon, Bobaeen_US
dc.contributor.authorPaquette, Ericen_US
dc.contributor.authorMudur, Sudhiren_US
dc.contributor.authorPopa, Tiberiuen_US
dc.contributor.editorCeylan, Duyguen_US
dc.contributor.editorLi, Tzu-Maoen_US
dc.date.accessioned2025-05-09T09:35:21Z
dc.date.available2025-05-09T09:35:21Z
dc.date.issued2025
dc.description.abstractVisual dubbing aims to modify facial expressions to ''lip-sync'' a new audio track. While person-generic talking head generation methods achieve expressive lip synchronization across arbitrary identities, they usually lack person-specific details and fail to generate high-quality results. Conversely, person-specific methods require extensive training. Our method combines the strengths of both methods by incorporating a virtual dubber, a person-generic talking head, as an intermediate representation. We then employ an autoencoder-based person-specific identity swapping network to transfer the actor identity, enabling fullhead reenactment that includes hair, face, ears, and neck. This eliminates artifacts while ensuring temporal consistency. Our quantitative and qualitative evaluation demonstrate that our method achieves a superior balance between lip-sync accuracy and realistic facial reenactment.en_US
dc.description.sectionheadersShort Paper 2
dc.description.seriesinformationEurographics 2025 - Short Papers
dc.identifier.doi10.2312/egs.20251034
dc.identifier.isbn978-3-03868-268-4
dc.identifier.issn1017-4656
dc.identifier.pages4 pages
dc.identifier.urihttps://doi.org/10.2312/egs.20251034
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/egs20251034
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Image manipulation; Animation
dc.subjectComputing methodologies → Image manipulation
dc.subjectAnimation
dc.titlePersonalized Visual Dubbing through Virtual Dubber and Full Head Reenactmenten_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
egs20251034.pdf
Size:
3.29 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
short1011_mm.mp4
Size:
183.76 MB
Format:
Video MP4
Download
Thumbnail Image
Name:
short1011_supp.pdf
Size:
15.25 MB
Format:
Adobe Portable Document Format
Download
Collections
EG 2025 - Short Papers