Concept-based explanations for neural networks

Abstract

Convolutional neural networks (CNNs) play a crucial role in data-intensive tasks across various industries. The integration of CNNs within Industry 4.0 has accelerated automation, particularly in visual quality control and predictive maintenance. Despite their adoption, CNNs are often used in underspecified tasks, which may lead them to learn undesirable spurious correlations or biases from data. This can result in unexpected behaviors that undermine the reliability of automated systems. This risk highlights the need for robust explainability methods to ensure models perform tasks for the right reasons and align with human expert knowledge. This dissertation addresses CNN interpretability through Explainable Artificial Intelligence (XAI), specifically by developing Concept Extraction (CE) methods. These methods, aimed at automatically extracting distinguishable, high-level patterns learned by models, have shown promising results. But they fail when dealing with industrial data and models. The core hypothesis of our work is that by considering the properties of models, such as how they encode scale or translation of features, we can improve CE methods for industrial applications where these properties are meaningful. This work rethinks CE in CNNs by considering model properties and usage constraints. We extend patch-based methods with SPACE (Scale-Preserving Automatic Concept Extraction) to handle scale variance of features such as pinholes or scratches. Additionally, we introduce ECLAD (Extracting Concepts with Local Aggregated Descriptors), which is capable of not only extracting but also localizing concepts. We validate these methods in real-world industrial applications, providing a clear explanation of the visual features used by the models, as well as allowing the detection of undesired biases. Lastly, we develop CoRe (Concept Regularization) to retrain models and mitigate undesired biases using human feedback and concept-based regularization. Through our work, we introduce SPACE, ECLAD, and CoRe, methods tailored to industrial models and data, providing detailed and robust explanations for understanding CNNs. We demonstrate that a global analysis of a trained model allows experts to identify and mitigate undesired biases. Beyond the domain of industrial AI, this thesis improves CE methods in general. Our contributions to XAI include novel methods for extracting scale-sensitive concepts from models, providing pixel-wise localization of concept, and developing recourse mechanisms to reduce the importance of undesired biases.