Multi-Modal Data Fusion, Image Segmentation, and Object Identification using Unsupervised Machine Learning: Conception, Validation, Applications, and a Basis for Multi-Modal Object Detection and Tracking

Abstract

Remote sensing and instrumentation is constantly improving and increasing in capability. Included within this, is the increase in amount of different instrument types, with various combinations of spatial and spectral resolutions, pointing angles, and various other instrument-specific qualities. While the increase in instruments, and therefore datasets, is a boon for those aiming to study the complexities of the various Earth systems, it can also present a large number of new challenges. With this information in mind, our group has set our aims on combining datasets with different spatial and spectral resolutions in an effective and as-general-as-possible way, with as little pre-existing per-instrument or per-dataset bias as possible, in order to create a system that can use pre-existing instrumentation/datasets as a sensor web of sorts. This dissertation explores the efficacy of methodologies for for generic data fusion, image segmentation, and object identification on multi-modal, remotely sensed data.\nIn Chapter 1, methodologies, background, and the overall framework is discussed. In Chapter 2, we evaluate the efficacy of the methodologies introduced in Chapter 1. Where available, we compare against ground truth data, or pre-existing classification datasets, with a high degree of agreement, and in other situations qualitative evaluation is used. The results of our evaluations show that the methodologies proposed are highly capable in terms of understanding the structure of the data input, in a very specific manner, and in some cases provide a significant improvement to previously existing datasets. After the methodology was validated, we moved to looking for concrete applications for this methodology. The techniques are first tested on fire and smoke plume identification, and is evaluated for assistance in dust plume identification and harmful algal bloom identification. This is explored in Chapter 3. Lastly, conclusions are drawn and discussions about current and future work are presented.