Semantic segmentation of urban areas using relabeled heterogeneous unmanned aerial datasets and combined deep learning network

Abstract

Unmanned aerial vehicles (UAVs) can overcome several limitations of satellite and aerial platforms using their multiple visit ability. However, UAVs usually collect images of small and simple regions from a large image scene and obtain high-resolution images from various viewing angles and altitudes. Multiple datasets created in various regions and conditions can be helpful considering data expansion to improve the usability of the UAV datasets with deep learning. The combined segmentation network (CSN), which can train two datasets simultaneously by sharing encoding blocks, was used to segment heterogeneous UAV datasets, such as UAVid and semantic drone dataset. CSN shared encoding blocks to learn general features from two datasets and decoding blocks trained separately on each dataset. For the preprocessing step, classes of each dataset were adjusted to minimize the difference between the two datasets. Experiment results show that CSN can segment more accurately for specific classes, such as background and vegetation, which have low ratios in the single dataset. This study presented the potential application of integrated heterogeneous UAV imagery datasets by learning shared layers. Thus, surface inspection would be effectively conducted using UAV datasets.