SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Reranking
Loading...
Date
2017
Journal Title
Journal ISSN
Volume Title
Publisher
Association for Computing Machinery, Inc (ACM)
Abstract
We present a sketch-based image retrieval system, designed to answer arbitrary queries that may go beyond searching for predefined object or scene categories. While sketching is fast and intuitive to formulate visual queries, pure sketch-based image retrieval often returns many outliers because it lacks a semantic understanding of the query. Our key idea is to combine sketch-based queries with interactive, semantic re-ranking of query results. We leverage progress in deep learning and use a feature representation learned for image classification for re-ranking. This allows us to cluster semantically similar images, re-rank based on the clusters, and present more meaningful query results to the user. We report on two large-scale benchmarks and demonstrate that our re-ranking approach leads to significant improvements over the state of the art. Finally, a user study designed to evaluate a practical use case confirms the benefits of our approach.
Description
@inproceedings{10.1145:3092907.3092910,
booktitle = {Sketch-Based Interfaces and Modeling},
editor = {Holger Winnemoeller and Lyn Bartram},
title = {{SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Reranking}},
author = {Portenier, Tiziano and Hu, Qiyang and Favaro, Paolo and Zwicker, Matthias},
year = {2017},
publisher = {Association for Computing Machinery, Inc (ACM)},
ISSN = {1812-3503},
ISBN = {978-1-4503-5080-8},
DOI = {10.1145/3092907.3092910}
}