Mapping of Facial Action Units to Virtual Avatar Blend Shape Movement
Loading...
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Action Units and blend shapes are two frameworks to describe facial movement. However, mappings between the two frameworks are underinvestigated. We present an automated mapping technique using machine learning. Our model infers ARKitcompatible blend shape weights from action unit intensities extracted with OpenFace. We use a GRU architecture to retain time-dependent information leveraging the particularities of Recurrent Neural Networks while still permitting fast, real-time inference. Our generalized model yields an activation precision of 90% and an activation recall of 85%.
Description
CCS Concepts: Human-centered computing → Virtual reality; Computing methodologies → Machine learning algorithms
@inproceedings{10.2312:egve.20241392,
booktitle = {ICAT-EGVE 2024 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments - Posters and Demos},
editor = {Tanabe, Takeshi and Yem, Vibol},
title = {{Mapping of Facial Action Units to Virtual Avatar Blend Shape Movement}},
author = {Wolff, Tony and Dollack, Felix and Perusquia-Hernandez, Monica and Uchiyama, Hideaki and Kiyokawa, Kiyoshi},
year = {2024},
publisher = {The Eurographics Association},
ISSN = {1727-530X},
ISBN = {978-3-03868-246-2},
DOI = {10.2312/egve.20241392}
}