Using Semantic Commonsense Resources in Image Retrieval

Abstract

Many people use the Internet is to find pictures of things. When extraneous images appear in response to simple queries on a search engine, the user has a hard time understanding why his seemingly clear request was not properly satisfied. If the computer could only understand what he wanted better, then maybe the results would be more precise. We believe that the introduction of an ontology, though hidden from the user, into current image retrieval engines would provide more accurate image responses to his query. Coordinating the use of an ontology (OWL representation of WordNet) with image processing techniques, we have developed a system that, given an initial query, automatically returns images associated with the query by specializing the query concept using only its deepest hyponyms from the ontology. We show that picking randomly from this new set of images provides a better representation for the initial, more general query. In addition, we exploit the visual aspects of the images for these deepest hyponyms (the leaves of WordNet) to cluster the images into coherent sets. In this way we can present the results in a structured, and even ontologically labeled, manner to the user