Text2Autochrome: Text guided autochrome synthesis using generative models

Simple item page
dc.contributor.authorKühn, Paul Juliusen_US
dc.contributor.authorSinha, Saptarshi Neilen_US
dc.contributor.authorNguyen, Duc Anhen_US
dc.contributor.authorHorst, Robinen_US
dc.contributor.authorKuijper, Arjanen_US
dc.contributor.authorFellner, Dieter W.en_US
dc.contributor.editorCampana, Stefanoen_US
dc.contributor.editorFerdani, Danieleen_US
dc.contributor.editorGraf, Holgeren_US
dc.contributor.editorGuidi, Gabrieleen_US
dc.contributor.editorHegarty, Zackaryen_US
dc.contributor.editorPescarin, Sofiaen_US
dc.contributor.editorRemondino, Fabioen_US
dc.date.accessioned2025-09-05T08:39:07Z
dc.date.available2025-09-05T08:39:07Z
dc.date.issued2025
dc.description.abstractAutochrome is an early color photography technique that is highly sensitive and prone to deterioration, limiting their public display. A limited collection of digitized autochromes exists, often with defects due to their fragile nature. We applied generative AI methods, specifically Low-Rank Adaptation (LoRA), to fine-tune diffusion models, enabling efficient use of computational resources. Our curated dataset of vintage digitized autochromes showcased various styles and served as the basis for training the LoRA model, resulting in the generation of digitized autochromes that preserved the original color filter effects and characteristic granularity. By leveraging generative AI, we can utilize the multi-modal capabilities of the model, allowing each user to generate images through concept-based prompts. This approach empowers users to creatively interact with the model, producing personalized images while maintaining the historical color fidelity and structure of autochromes. This capability also enables us to generate defect-free autochromes, which can be utilized for synthetic training in autochrome restoration efforts. We evaluated our approach using the CLIPScore metric for quantitative similarity and conducted a user study for qualitative feedback on the generated images. Our results show that the fine-tuned LoRA model effectively captures the essence of autochromes, producing visually appealing images that respect the historical aesthetic. Considering the potential for misinterpretation and ethical concerns surrounding text-to-image methods using deep learning with historical photographs, we are committed to enhancing transparency by releasing our model weights and training datasets, thereby empowering the community to better understand, evaluate, and address these important issues. Further we release an interactive demo together with the fine-tuned weights available via huggingface.en_US
dc.description.sectionheadersPERCEIVE: Exhibiting the ''Unexhibitable''
dc.description.seriesinformationDigital Heritage
dc.identifier.doi10.2312/dh.20253061
dc.identifier.isbn978-3-03868-277-6
dc.identifier.pages10 pages
dc.identifier.urihttps://doi.org/10.2312/dh.20253061
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/dh20253061
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Neural networks
dc.subjectComputing methodologies → Neural networks
dc.titleText2Autochrome: Text guided autochrome synthesis using generative modelsen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
dh20253061.pdf
Size:
20.24 MB
Format:
Adobe Portable Document Format
Download
Collections
Track 08 – Digital Technologies for Colour