Deep Convolutional Networks for Image Classification

Abstract

Image classification is an important problem in machine learning. Deep neural networks, particularly deep convolutional networks, have recently contributed great improvements to end-to-end learning quality for this problem. Such networks significantly reduce the need for human designed features in the image recognition process. In this thesis I address two questions: first, how best to design the architecture of a convolutional neural network for image classification; and second, how to improve the activation functions used in convolutional neural networks. I review the history of convolutional network architectures, then propose an efficient network structure named ”TinyNet” that reduces network size while preserving state of the art image classification performance. For the second question I propose a new kind of activation function, called the ”Randomized Leaky Rectified Linear Unit”, which improves the empirical generalization performance of the now widely used Rectified Linear Unit. Also, I make an explanation of the difficulty of training deep sigmoid network. The thesis culminates in a demonstration of the TinyNet architecture with Randomized Leaky Rectified Linear Units, which obtains state-of-art results on the CIFAR-10 image classification data set without any preprocessing. To further demonstrate the generality of the results, I apply the general convolutional neural network structure to a different image classification problem, with completely different textures and shapes, and again achieve state-of-art results on a data set from the National Data Science Bowl competition.