Directional Texture Editing for 3D Models
Loading...
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.
Abstract
Texture editing is a crucial task in 3D modelling that allows users to automatically manipulate the surface materials of 3D models. However, the inherent complexity of 3D models and the ambiguous text description lead to the challenge of this task. To tackle this challenge, we propose ITEM3D, a exture diting odel designed for automatic object editing according to the text nstructions. Leveraging the diffusion models and the differentiable rendering, ITEM3D takes the rendered images as the bridge between text and 3D representation and further optimizes the disentangled texture and environment map. Previous methods adopted the absolute editing direction, namely score distillation sampling (SDS) as the optimization objective, which unfortunately results in noisy appearances and text inconsistencies. To solve the problem caused by the ambiguous text, we introduce a relative editing direction, an optimization objective defined by the noise difference between the source and target texts, to release the semantic ambiguity between the texts and images. Additionally, we gradually adjust the direction during optimization to further address the unexpected deviation in the texture domain. Qualitative and quantitative experiments show that our ITEM3D outperforms the state‐of‐the‐art methods on various 3D objects. We also perform text‐guided relighting to show explicit control over lighting. Our project page: .
Description
@article{10.1111:cgf.15196,
journal = {Computer Graphics Forum},
title = {{Directional Texture Editing for 3D Models}},
author = {Liu, Shengqi and Chen, Zhuo and Gao, Jingnan and Yan, Yichao and Zhu, Wenhan and Lyu, Jiangjing and Yang, Xiaokang},
year = {2024},
publisher = {© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.},
DOI = {10.1111/cgf.15196}
}