Sketch-based image retrieval using sketch tokens

Abstract

One fundamental challenge of Sketch-based Image Retrieval (SBIR) is the appearance gap between sketches and natural images. To bridge the gap, we propose a framework that describes both types of images based on sketch tokens. Sketch tokens are mid-level representations of local edge structures. Compared with describing images with pixel-level features, describing images with sketch tokens is more accurate and robust. We compute the responses of image patches to sketch tokens, and propose a local descriptor to describe object shape by capturing the sketch token responses. Bag-of-visual-word mode is utilized to represent images, and inverse indexing is built to accelerate the retrieval process. We compared the proposed work with state-of-the-art methods (SHoG, GF-HOG) on two public datasets. The experimental results show that our method outperforms them and significantly improves SBIR performance.