Large language model-driven knowledge graph reasoning for enhanced semantic segmentation

Abstract

Urban scene segmentation is essential for 3D city modeling and plays a crucial role in various remote sensing applications, including urban planning and environmental monitoring. While integrating knowledge graphs with scene segmentation has improved accuracy, existing methods often depend on dataset-specific knowledge graphs, limiting their generalizability across diverse remote sensing data. To address this, we propose a novel framework that leverages large language models (LLMs) to construct a universal knowledge graph from multi-source geospatial data and incorporate it into remote sensing semantic segmentation tasks, enhancing adaptability and robustness in urban scene understanding. Specifically, the framework comprises two key components: (1) a Graph Construction module that employs LLMs to extract cross-domain semantic relationships and build a universal knowledge graph, and (2) a Knowledge Graph Fusion module (KGFusion) that incorporates the graph into a semantic segmentation network to enhance semantic understanding. To evaluate the adaptability of our method across diverse domains, we curated a mixed dataset encompassing urban, rural, and port scenes. Experimental findings validate the efficiency and adaptability of our method, achieving 70.94% mIoU on the UAVid dataset and 63.23% on the Mixed dataset, outperforming the baseline by 0.43% and 1.04%, respectively. These results validate the robustness of our method in cross-domain scenarios and highlight its potential for broader applications in complex urban environments.