MaterIA: Single Image High-Resolution Material Capture in the Wild

Martin, Rosalie; Roullier, Arthur; Rouffet, Romain; Kaiser, Adrien; Boubekeur, Tamy

MaterIA: Single Image High-Resolution Material Capture in the Wild

Files

v41i2pp163-177.pdf (240.7 MB)

paper1153_3.pdf (350.59 MB)

Date

2022

Authors

Martin, Rosalie
Roullier, Arthur
Rouffet, Romain
Kaiser, Adrien
Boubekeur, Tamy

Publisher

The Eurographics Association and John Wiley & Sons Ltd.

Abstract

We propose a hybrid method to reconstruct a physically-based spatially varying BRDF from a single high resolution picture of an outdoor surface captured under natural lighting conditions with any kind of camera device. Relying on both deep learning and explicit processing, our PBR material acquisition handles the removal of shades, projected shadows and specular highlights present when capturing a highly irregular surface and enables to properly retrieve the underlying geometry. To achieve this, we train two cascaded U-Nets on physically-based materials, rendered under various lighting conditions, to infer the spatiallyvarying albedo and normal maps. Our network processes relatively small image tiles (512x512 pixels) and we propose a solution to handle larger image resolutions by solving a Poisson system across these tiles. We complete this pipeline with analytical solutions to reconstruct height, roughness and ambient occlusion.

        @article{10.1111:cgf.14466
,
journal = {Computer Graphics Forum},
title = {{MaterIA: Single Image High-Resolution Material Capture in the Wild
}},
author = {Martin, Rosalie and 
Roullier, Arthur and 
Rouffet, Romain and 
Kaiser, Adrien and 
Boubekeur, Tamy
},
year = {2022
},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.
},
ISSN = {1467-8659
},
DOI = {10.1111/cgf.14466
}
}