Kernel methods for unsupervised domain adaptation

Abstract

In many applications (computer vision, natural language processing, speech recognition, etc.), the curse of domain mismatch arises when the test data (of a target domain) and the training data (of some source domain(s)) come from different distributions. Thus, developing techniques for domain adaptation, i.e., generalizing models from the sources to the target, has been a pressing need. When the learner has access to only unlabeled data from the target domain (and labeled data from the source domain), the problem is called unsupervised domain adaptation. Advances in domain adaptation can significantly increase our capability to deploy autonomous and intelligent systems in challenging environments where uncertainty prevails. ? This thesis work provides a comprehensive set of techniques with multiple levels of focus on unsupervised domain adaptation, including learning domain?invariant feature representations in order to eliminate the discrepancy between the source domain and the target domain, manipulating data instances to match the distributions of two domains, quantizing the ""adaptabilities"" of different source domains given a particular target domain, and discovering latent domains from heterogeneous data so the individual domains can be better and more efficiently modeled. We demonstrate the effectiveness of the developed methods on well?benchmarked datasets and tasks (visual object recognition, sentiment analysis, and cross?view human activity recognition). ? In terms of domain adaptation algorithms, this thesis develops two complementary approaches using kernel methods, one to infer domain?invariant geodesic flow kernels (GFKs) and the other to directly match the underlying distributions of two domains. GFK models data by subspaces and interpolates an infinite number of phantom domains between the source domain and the target domain. We then use the ""kernel trick"" to average out domain?specific idiosyncrasies and arrive at a domain?invariant kernel. Built upon GFK, we propose an approach to identifying the most adaptable data instances of the source domain, named as landmarks, to the target domain. Due to that the landmarks are more similar to the target domain in the sense of their underlying distributions, adapting from the landmarks gives rise to better performance on the target than adapting from the original source domain. ? This thesis also contributes to other aspects of domain adaptation. We make some preliminary efforts on answering the open question of how to evaluate the ""adaptability"" of a source domain to the target domain. This results in a rank?of?domains (ROD) metric, which exploits both geometrical and statistical discrepancies between two domains. Besides, this thesis raises the concern about how to define or represent a domain with real data. While by a domain we refer to the underlying distribution of the observed data, the distribution is often unknown. A standard practice has been equating datasets with domains. However, our studies show that this is not necessarily the best for the adaptation tasks. An automatic algorithm is instead proposed to ""reshape"" the data into domains which are better in terms of the adaptation performance to the target domain. ? To further explore kernels, which play the central role in our approaches to domain adaptation, this thesis concludes by researching kernels in a probabilistic model, determinantal point process (DPP). We propose a novel model, sequential DPP, for supervised video summarization and derive a large?margin training algorithm for learning the kernels in DPPs.