Artistic Style Transfer Based on Attention with Knowledge Distillation
Loading...
Date
2024
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.
Abstract
Artistic style transfer involves the adaption of an input image to reflect the style of a reference image while maintaining its original content. This technique, now a prominent focus due to its prospective use in creative fields like digital art and graphic design, typically applies normalization techniques and attention mechanisms. While these methods yield decent results, they often fall short due to distortion of content image details and non‐artefact styles. In this paper, we introduce a novel approach that synergizes adaptive instance normalization (AdaIN), attention mechanisms, knowledge distillation (KD) and strategically placed internal layers, and new enhancements designed to preserve content details and provide a nuanced control over the style transfer process. We introduce a Detail Enhancement Module to amplify high‐frequency details in the content image, enhancing edge and texture preservation. A Multi‐scale Strategy is implemented to ensure uniform style application across various detail levels, leading to more coherent stylization. The Content Feature Refinement process refines content features, sharpening and emphasizing details to preserve structural and textural integrity. AdaIN's distinctive feature of efficiently collecting style data is exploited in our approach, coupled with attention mechanisms' inherent ability to conserve content information. We supplement this blend with KD for the enhancement of model accuracy and efficiency. Additionally, the introduction of internal layers acts as a conduit to further improve the style transfer process, increasing the transfer level of features and fostering better stylized results. The cornerstone of our technique lies in preserving the content structure amidst complex style transfers. Experimental results affirm the superior performance of our method over existing techniques in both quantitative and qualitative evaluations.
Description
@article{10.1111:cgf.15127,
journal = {Computer Graphics Forum},
title = {{Artistic Style Transfer Based on Attention with Knowledge Distillation}},
author = {Al‐Mekhlafi, Hanadi and Liu, Shiguang},
year = {2024},
publisher = {© 2024 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.},
DOI = {10.1111/cgf.15127}
}