Smartphone-Based Seamless Scene and Object Recognition for Visually Impaired Persons

Abstract

This study introduces a mobile application designed to assist visually impaired persons (VIPs) in navigating complex environments, such as supermarkets. Recent assistive tools often identify objects in isolation without providing contextual awareness. In contrast, our proposed system uses seamless scene and object recognition to help users efficiently locate target items and understand their surroundings. Employing a “human-in-the-loop approach”, users control their smartphone camera direction to explore the space. Experiments conducted in a simulated shopping environment show that the system enhances object-finding efficiency and improves user orientation. This approach not only increases independence, but also promotes inclusivity by enabling VIPs to perform everyday tasks with greater confidence and autonomy.