Clothes detection and classification using convolutional neural networks

Abstract

In this paper we describe development of a computer vision system for accurate detection and classification of clothes for e-commerce images. We present a set of experiments on well established architectures of convolutional neural networks, including Residual networks, SqueezeNet and Single Shot MultiBox Detector (SSD). The clothes detection network was trained and tested on DeepFashion dataset, which contains box annotations for locations of clothes. Classification task was evaluated on a set of images of dresses that were collected from online shops. Ground truth labels were inferred from shop items metadata for five different attributes, including color, pattern, sleeve, neckline and hemline, each consisting of several possible classes. Automatic gathering of labels resulted in an average of 83% rate of correct labels. In the experiments we evaluate the impact on classification accuracy of a set of potential improvements, including data augmentation by generating diverse backgrounds, increasing the size of the network and using ensembles. We analyse the accuracy improvements with respect to the processing efficiency. Finally, we present the achieved accuracy rates in the clothes detection task and outline the most successful network configurations for dresses classification.