Self-Supervised Autoencoders for Visual Anomaly Detection

Abstract

We focus on detecting anomalies in images where the data distribution is supported by a lower-dimensional embedded manifold. Approaches based on autoencoders have aimed to control their capacity either by reducing the size of the bottleneck layer or by imposing sparsity constraints on their activations. However, none of these techniques explicitly penalize the reconstruction of anomalous regions, often resulting in poor detection. We tackle this problem by adapting a self-supervised learning regime that essentially implements a denoising autoencoder with structured non-i.i.d. noise. Informally, our objective is to regularize the model to produce locally consistent reconstructions while replacing irregularities by acting as a filter that removes anomalous patterns. Formally, we show that the resulting model resembles a nonlinear orthogonal projection of partially corrupted images onto the submanifold of uncorrupted examples. Furthermore, we identify the orthogonal projection as an optimal solution for a specific regularized autoencoder related to contractive and denoising variants. In addition, orthogonal projection provides a conservation effect by largely preserving the original content of its arguments. Together, these properties facilitate an accurate detection and localization of anomalous regions by means of the reconstruction error. We support our theoretical analysis by achieving state-of-the-art results (image/pixel-level AUROC of 99.8/99.2%) on the MVTec AD dataset—a challenging benchmark for anomaly detection in the manufacturing domain.