Track 08 – Digital Technologies for Colour

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3607224

Browse

Browsing Track 08 – Digital Technologies for Colour by Subject "Computing methodologies → Neural networks"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Text2Autochrome: Text guided autochrome synthesis using generative models
    (The Eurographics Association, 2025) Kühn, Paul Julius; Sinha, Saptarshi Neil; Nguyen, Duc Anh; Horst, Robin; Kuijper, Arjan; Fellner, Dieter W.; Campana, Stefano; Ferdani, Daniele; Graf, Holger; Guidi, Gabriele; Hegarty, Zackary; Pescarin, Sofia; Remondino, Fabio
    Autochrome is an early color photography technique that is highly sensitive and prone to deterioration, limiting their public display. A limited collection of digitized autochromes exists, often with defects due to their fragile nature. We applied generative AI methods, specifically Low-Rank Adaptation (LoRA), to fine-tune diffusion models, enabling efficient use of computational resources. Our curated dataset of vintage digitized autochromes showcased various styles and served as the basis for training the LoRA model, resulting in the generation of digitized autochromes that preserved the original color filter effects and characteristic granularity. By leveraging generative AI, we can utilize the multi-modal capabilities of the model, allowing each user to generate images through concept-based prompts. This approach empowers users to creatively interact with the model, producing personalized images while maintaining the historical color fidelity and structure of autochromes. This capability also enables us to generate defect-free autochromes, which can be utilized for synthetic training in autochrome restoration efforts. We evaluated our approach using the CLIPScore metric for quantitative similarity and conducted a user study for qualitative feedback on the generated images. Our results show that the fine-tuned LoRA model effectively captures the essence of autochromes, producing visually appealing images that respect the historical aesthetic. Considering the potential for misinterpretation and ethical concerns surrounding text-to-image methods using deep learning with historical photographs, we are committed to enhancing transparency by releasing our model weights and training datasets, thereby empowering the community to better understand, evaluate, and address these important issues. Further we release an interactive demo together with the fine-tuned weights available via huggingface.