Spectral Methods for Multimodal Data Analysis
No Thumbnail Available
Date
2016-11-01
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Spectral methods have proven themselves as an important and versatile tool in
a wide range of problems in the fields of computer graphics, machine learning,
pattern recognition, and computer vision, where many important problems boil
down to constructing a Laplacian operator and finding a few of its eigenvalues
and eigenfunctions. Classical examples include the computation of diffusion distances
on manifolds in computer graphics, Laplacian eigenmaps, and spectral
clustering in machine learning.
In many cases, one has to deal with multiple data spaces simultaneously. For
example, clustering multimedia data in machine learning applications involves
various modalities or “views” (e.g., text and images), and finding correspondence
between shapes in computer graphics problems is an operation performed between
two or more modalities.
In this thesis, we develop a generalization of spectral methods to deal with
multiple data spaces and apply them to problems from the domains of computer
graphics, machine learning, and image processing. Our main construction is
based on simultaneous diagonalization of Laplacian operators. We present an
efficient numerical technique for computing joint approximate eigenvectors of
two or more Laplacians in challenging noisy scenarios, which also appears to
be the first general non-smooth manifold optimization method. Finally, we use
the relation between joint approximate diagonalizability and approximate commutativity
of operators to define a structural similarity measure for images. We
use this measure to perform structure-preserving color manipulations of a given
image.
To the best of our knowledge, the original contributions of this work are the
following:
1 Introduction of joint diagonalization methods to the fields of machine learning,
computer vision, pattern recognition, image processing, and graphics;
2 Formulation of the coupled approximate diagonalization problem that extends
the joint diagonalization to cases with no bijective correspondence
between the domains, and its application in a wide range of problems in
the above fields;
3 Introduction of a new structural similarity measure of images based on the
approximate commutativity of their respective Laplacians, and its application
in image processing problems such as color-to-gray conversion, colors
adaptation for color-blind viewers, gamut mapping, and multispectral image
fusion;
4 Development of Manifold Alternating Direction Method of Multipliers (MADMM),
the first general method for non-smooth optimization with manifold constraints,
and its applications to several problems.
Description