Max-Variance Convolutional Neural Network Model Compression

Abstract

In this paper, we present a method for convolutional neural network model compression which is based on the removal of filter banks that correspond to unimportant weights. To do this, we depart from the relationship between consecutive layers so as to obtain a factor that can be used to assess the degree upon which each pair of filters are coupled to each other. This allows us to use the unit-response of the coupling between two layers so as to remove pathways int he network that are negligible. Moreover, since the back-propagation gradients tend to diminish as the chain rule is applied from the output to the input layer, here we maximise the variance on the coupling factors while enforcing a monotonicity constraint that assures the most relevant pathways are preserved. We show results on widely used networks employing classification and facial expression recognition datasets. In our experiments, our approach delivers a very competitive trade-off between compression rates and performance as compared to both, the uncompressed models and alternatives elsewhere in the literature. pages = 271-279.