Data-Aware Zero-Shot Neural Architecture Search for Image Recognition

Abstract

Zero-shot neural architecture search (NAS) has shown great potential in designing image recognition networks for its high efficiency and low resource consumption. However, most of the existing zero-shot NAS methods fail to utilize prior information in datasets when calculating the score of candidate networks, leading to inferior performance. Our theoretical analysis and experimental results reveal that utilizing the samples in datasets as input for calculating scores can obtain better search results. Besides, we notice some samples in the dataset have larger score variance than the others. Based on these findings, we design data-aware zero-shot (DAZS) NAS. We introduce a generator to generate data for a score calculation with affordable overhead, and adopt contrastive learning to optimize the generator for a more stable score. Experiments show that our DAZS achieves superior results against the state-of-the-art method on both CIFAR and ImageNet-1k, and has good transferability.