Search-Efficient NAS: Neural Architecture Search for Classification

Abstract

Recently, there has been an increasing trend in automating the process of NAS. Most SOTA approaches required higher computational costs and hardware. Among all methods, the Differentiable Architecture Search (DARTS) made the automating process available within a few GPU days. However, the performance of DARTS is observed to struggle with memory issues and often collapsed when used a larger number of epochs while searching architectures. To overcome the issue of memory overhead and longer search time, the paper proposes a method by sampling the selected channels only, which not only reduces the redundant operations but will also minimize the search time. This method has reduced the GPU search hours drastically to 6.3 hours as compared to DARTS.