Large-Scale LoD2 Building Modeling using Deep Multimodal Feature Fusion

Abstract

In today’s rapidly urbanizing world, accurate 3D city models are crucial for sustainable urban management. The existing technology for 3D city modeling still relies on an extensive amount of manual work and the provided solutions may vary depending on the urban structure of different places. According to the CityGML3 standard of 3D city modeling, in LoD2, the roof structures need to be modeled which is a challenging task due to the complexity and diversity of roof types. While high-resolution images can be utilized to classify roof types, they have difficulties in areas with poor contrast or shadows. This study proposes a deep learning approach that combines RGB optical and height information of buildings to improve the accuracy of roof type classification and automatically generate a 3D city model. The proposed methodology is divided into two phases: (1) classifying roof types into the nine most popular roof types in New Brunswick, Canada, using a multimodal feature fusion network, and (2) generating a large-scale LoD2 3D city model using a model-driven approach. The evaluation results show an overall accuracy of 97.58% and a Kappa coefficient of 0.9705 for the classification phase and an RMSE of 1.03 (m) for the 3D modeling.