Smart Search: Investigations into human visual search in structured environments

Abstract

[PhD thesis] To us, visual search for objects in the environment feels effortless as compared to other tasks such as multiplying large numbers. However, our efforts at building artificial systems have revealed that the former is computationally more challenging than the latter. That makes us wonder how our brain efficiently carries out visual searches. Decades of research indicate that the efficiency of human visual search relies on a plethora of processes, primary of which are: one, processing the hierarchical construction of the visual world (simple features such as orientations of lines constituting complex features such as shapes), two, selectively processing information relevant to the search task (e.g., suppress processing from parts of the image that contain non-target features), and three, learning the relationships between the constituent elements of the world that can guide the information selection process (e.g., knowing where an object occurs in a scene helps us constrain the search to those locations). Furthering our understanding of the processes underlying efficient search, I present new evidence using artificial neural networks, neuroimaging experiments (fMRI and EEG), and large-scale behavioral experiments. The main contributions are as follows: one, the search for body shapes can occur parallelly across our field of view; two, where selective attention needs to be deployed in a hierarchical visual system depends on the representational capacity of that visual system; three, the knowledge about the co-occurrences amongst the distractors can be learned and utilized to increase our search efficiency. I conclude the thesis by discussing the questions raised through our investigations and the research directions aimed at furthering our understanding of our seemingly effortless, but smart, visual search capabilities.