Neuro-Symbolic Scene Graph Reasoning for Explainable Computer Vision

Abstract

Scene graph reasoning is a crucial aspect of computer vision, enabling machines to understand complex relationships between objects in a scene. However, traditional deep learning approaches often lack explainability, making it difficult to understand their decision-making process. This paper proposes a novel neuro-symbolic framework for scene graph reasoning that combines the strengths of neural networks and symbolic reasoning to achieve both high accuracy and explainability. Our framework utilizes a graph neural network to extract visual features and construct an initial scene graph. This graph is then refined and reasoned upon using symbolic rules and knowledge, allowing for explicit inference and justification of the relationships between objects. We demonstrate the effectiveness of our approach on several benchmark datasets, showing that it outperforms existing methods in terms of both accuracy and explainability. Furthermore, we provide detailed analysis and visualizations to illustrate how our framework arrives at its conclusions, providing insights into the reasoning process. The results show that our model captures complex relationships more effectively than purely neural approaches, and the symbolic component allows us to generate explanations for the model's predictions. This work paves the way for more transparent and trustworthy computer vision systems.