ZeroEGGS: Zero‐shot Example‐based Gesture Generation from Speech

Simple item page
dc.contributor.authorGhorbani, Saeeden_US
dc.contributor.authorFerstl, Ylvaen_US
dc.contributor.authorHolden, Danielen_US
dc.contributor.authorTroje, Nikolaus F.en_US
dc.contributor.authorCarbonneau, Marc‐Andréen_US
dc.contributor.editorHauser, Helwig and Alliez, Pierreen_US
dc.date.accessioned2023-03-22T15:07:14Z
dc.date.available2023-03-22T15:07:14Z
dc.date.issued2023
dc.description.abstractWe present ZeroEGGS, a neural network framework for speech‐driven gesture generation with zero‐shot style control by example. This means style can be controlled via only a short example motion clip, even for motion styles unseen during training. Our model uses a Variational framework to learn a style embedding, making it easy to modify style through latent space manipulation or blending and scaling of style embeddings. The probabilistic nature of our framework further enables the generation of a variety of outputs given the input, addressing the stochastic nature of gesture motion. In a series of experiments, we first demonstrate the flexibility and generalizability of our model to new speakers and styles. In a user study, we then show that our model outperforms previous state‐of‐the‐art techniques in naturalness of motion, appropriateness for speech, and style portrayal. Finally, we release a high‐quality dataset of full‐body gesture motion including fingers, with speech, spanning across 19 different styles. Our code and data are publicly available at .en_US
dc.description.number1
dc.description.sectionheadersArticles
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume42
dc.identifier.doi10.1111/cgf.14734
dc.identifier.issn1467-8659
dc.identifier.pages206-216
dc.identifier.urihttps://doi.org/10.1111/cgf.14734
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14734
dc.publisherEurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.en_US
dc.rightsCC BY-NC-ND Attribution-NonCommercial-NoDerivatives 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subjectanimation
dc.subjectgestures
dc.subjectcharacter control
dc.subjectmotion capture
dc.titleZeroEGGS: Zero‐shot Example‐based Gesture Generation from Speechen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v42i1pp206-216-cgf14734.pdf
Size:
1.42 MB
Format:
Adobe Portable Document Format
Download
Collections
42-Issue 1