Swin Transformer with Local Aggregation

Abstract

Despite the many advantages of Convolutional Neural Networks (CNN), their perceptual fields are usually small and not conducive to capturing global features. In contrast, Transformer is able to capture long-range dependencies and obtain global information of an image with self-attention. For combining the advantages of CNN and Transformer, we propose to integrate the Local Aggregation module to the structure of Swin Transformer. The Local Aggregation module includes lightweight Depthwise Convolution and Pointwise Convolution, and it can locally capture the information of feature map at stages of Swin Transformer. Our experiments demonstrate that accuracy can be improved with such an integrated model. On the Cifar-10 dataset, the Top-1 accuracy reaches 87.74%, which is 3.32% higher than Swin, and the Top-5 accuracy reaches 99.54%; on the Mini-ImageNet dataset, the Top-1 accuracy reaches 79.1%, which is 7.68% higher than Swin, and the Top-5 accuracy reaches 94.02%, which is 3.25% higher than Swin 3.25%.