Interactive Photo Editing on Smartphones via Intrinsic Decomposition

Simple item page
dc.contributor.authorShekhar, Sumiten_US
dc.contributor.authorReimann, Maxen_US
dc.contributor.authorMayer, Maximilianen_US
dc.contributor.authorSemmo, Amiren_US
dc.contributor.authorPasewaldt, Sebastianen_US
dc.contributor.authorDöllner, Jürgenen_US
dc.contributor.authorTrapp, Matthiasen_US
dc.contributor.editorMitra, Niloy and Viola, Ivanen_US
dc.date.accessioned2021-04-09T08:01:53Z
dc.date.available2021-04-09T08:01:53Z
dc.date.issued2021
dc.description.abstractIntrinsic decomposition refers to the problem of estimating scene characteristics, such as albedo and shading, when one view or multiple views of a scene are provided. The inverse problem setting, where multiple unknowns are solved given a single known pixel-value, is highly under-constrained. When provided with correlating image and depth data, intrinsic scene decomposition can be facilitated using depth-based priors, which nowadays is easy to acquire with high-end smartphones by utilizing their depth sensors. In this work, we present a system for intrinsic decomposition of RGB-D images on smartphones and the algorithmic as well as design choices therein. Unlike state-of-the-art methods that assume only diffuse reflectance, we consider both diffuse and specular pixels. For this purpose, we present a novel specularity extraction algorithm based on a multi-scale intensity decomposition and chroma inpainting. At this, the diffuse component is further decomposed into albedo and shading components. We use an inertial proximal algorithm for non-convex optimization (iPiano) to ensure albedo sparsity. Our GPUbased visual processing is implemented on iOS via the Metal API and enables interactive performance on an iPhone 11 Pro. Further, a qualitative evaluation shows that we are able to obtain high-quality outputs. Furthermore, our proposed approach for specularity removal outperforms state-of-the-art approaches for real-world images, while our albedo and shading layer decomposition is faster than the prior work at a comparable output quality. Manifold applications such as recoloring, retexturing, relighting, appearance editing, and stylization are shown, each using the intrinsic layers obtained with our method and/or the corresponding depth data.en_US
dc.description.number2
dc.description.sectionheadersAnalyzing and Integrating RGB-D Images
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume40
dc.identifier.doi10.1111/cgf.142650
dc.identifier.issn1467-8659
dc.identifier.pages497-510
dc.identifier.urihttps://doi.org/10.1111/cgf.142650
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf142650
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectComputing methodologies
dc.subjectImage
dc.subjectbased rendering
dc.subjectImage processing
dc.subjectComputational photography
dc.titleInteractive Photo Editing on Smartphones via Intrinsic Decompositionen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
v40i2pp497-510.pdf
Size:
72.54 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
paper_1041_supplementary.pdf
Size:
216.9 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
cgf40-2_142650_ProjektDeal.pdf
Size:
72.62 MB
Format:
Adobe Portable Document Format
Description:
Projekt Deal version
Download
Collections
40-Issue 2