On Online Unsupervised Domain Adaptation

Abstract

Recent advances in Artificial Intelligence (AI) have been markedly accelerated by the convergence of advances in Machine Learning (ML) and the exponential growth in computational power. Within this dynamic landscape, the concept of Domain Adaptation (DA) is dedicated to the seamless transference of knowledge across domains characterized by disparate data distributions. This thesis ventures into the challenging and nuanced terrain of Online Unsupervised Domain Adaptation (OUDA), where the unlabeled data stream arrives from the target domain incrementally and gradually diverges from the source domain. This thesis presents two innovative and complementary approaches -- a manifold-based approach and a time-domain-based approach -- to effectively tackle the intricate OUDA challenges.The manifold-based approach seeks to address this gap by incorporating the domain alignment process in an incremental computation manner, and this novel technique leverages the computation of transformation matrices, based on the projection of both source and target data onto the Grassmann manifold. This projection aligns both domains by incrementally minimizing their dissimilarities, effectively ameliorating the divergence between the source and target data. This manifold-based approach capitalizes on the cumulative temporal information within the data stream, utilizing the Incremental Computation of Mean-Subspace (ICMS) technique. This technique efficiently computes the average subspace of target subspaces on the Grassmann manifold, adeptly capturing the evolving dynamics of the data distribution. The alignment process is further fortified by integrating the flow of target subspaces on the manifold. As the target data stream unfolds over time, this approach incorporates this information, yielding robust and adaptive transformation matrices. In addition, the efficient computation of the mean-subspace, closely aligned with the Karcher mean, attests to the computational feasibility of the manifold-based approach, thus, enabling real-time feedback computations for the OUDA problem.The time-domain-based approach utilizes the cluster-wise information and its flow information from each time-step to accurately predict target labels in the incoming target data, propagate consistently the class labels to future incoming target data, and efficiently utilize the predicted labels in the target data together with the source data to incrementally update the learning model in a supervised-learning scenario. This process effectively transforms the OUDA problem into a supervised-learning scenario. We leverage a neural-network-based model to align target features, cluster them class-wise and extend them linearly from the origin of the latent space as the time-step progresses. This alignment process enables accurate predictions and target label propagation based on the trajectories of the target features. We achieve target label propagation through the novel Flow-based Hierarchical Optimal Transport (FHOT) method, which considers element-wise, cluster-wise, and distribution-wise correspondences of adjacent target features. The learning model is continuously updated with incoming target data and their predicted labels.To comprehensively assess the impact and contributions of these two approaches to the OUDA problem, we conducted extensive experiments across diverse datasets. Our analysis covered each stage of the manifold-based approach, comparing its performance with prior methods in terms of classification accuracy and computational efficiency. The time-domain-based approach was validated through linear feature alignment in the latent space, resulting in accurate label predictions. Notably, the flow-based hierarchical optimal transport technique substantially enhanced classification accuracy, particularly with increasing time-steps. Furthermore, learning model updates using target data and predicted labels significantly improved classification accuracy.