An Empirical Analysis of Vision Transformers Robustness to Spurious Correlations in Health Data

Abstract

In recent years, vision transformers have become the state-of-the-art architecture for computer vision tasks in the medical domain. However, their robustness to health out- of-distribution (OOD) data remains an open question. In this paper, we focus on the scenario of diabetic retinopathy diagnosis and conduct an empirical analysis of the robustness of vision transformers to health OOD data. Our experiments are conducted on a large medical image dataset of diabetic retinopathy, and we evaluate the models' performance on both in-distribution and OOD data. Our results show that vision transformers are highly susceptible to health OOD data, leading to high-confidence predictions even when the data is far from the distribution seen during training. This work provides a comprehensive analysis of the robustness of vision transformers to health OOD data in diabetic retinopathy diagnosis, highlighting the importance of considering this aspect when using these models for medical image analysis.