Zero/few-shot anomaly localisation using language prompts

Abstract

State of the art methods for anomaly localisation in product images take a patch based approach that models an anomaly patch in an image as an outlier to a distribution of normal image patches. These approaches require the availability of sufficient normal and sometimes even abnormal product images. In this work we present a zero/few-shot anomaly localisation method, where, given an image and a set of textual prompts describing the possible anomalies, the solution can localise the defects in the image in the form of an anomaly map. We present a few approaches for identifying prompts optimised for improving localisation performance. We benchmark our proposed method with a recently proposed WinCLIP method and shows promising results using metrics like F1-max, AUROC, AUPR etc.