Fine-Grained Representation Alignment for Zero-Shot Domain Adaptation

Abstract

Most existing domain adaptation methods learn with both (labeled) samples in the source domain and (unlabeled) samples in the target domain. Relying on the availability of target domain samples, however, is not always feasible in real-world applications. In this paper, we propose a new method to address this issue, in which the target domain samples do not need to be available for the task of interest. To improve the performance of such a zero-shot domain adaptation (ZSDA), we learn with not only source samples in the task of interest, but also seek additional assistance from those dual-domain samples in an irrelevant task. To overcome the problems induced by the unavailability of target samples in the task of interest, we exploit the hypothesis that the domain correlation is consistent across tasks and learn to transfer it from the irrelevant task to the task of interest. Specifically, our method aims to learn a domain-invariant representation space in which the source-domain classifier is directly transferable to the target domain. We achieve this by restricting the two domains to share both inter-category structure and intra-category structure in the representation space. Experiment results on five benchmarking datasets indicate that our proposed method significantly outperforms the existing representative baselines.