Order-ViT: Order Learning Vision Transformer for Cancer Classification in Pathology Images

Abstract

In computational pathology, cancer classification is one of the most widely studied tasks. There exist numerous tools for cancer classification, which are mainly built based upon convolutional neural networks or Transformers. These tools, by and large, formulate cancer classification as a categorical classification problem, which ignores the intrinsic relationship among cancer grades. Herein, we propose an order learning vision transformer for cancer classification that can not only learn the histopathological patterns of individual cancer grades but also utilize the ordering relationship among cancer grades. Built based upon vision transformer, the proposed method simultaneously conducts categorical classification per input sample and order classification for a pair of input and reference samples. Moreover, it introduces a voting scheme to identify less confident samples and to improve the accuracy of the decision on such samples. The proposed method is evaluated on two types of cancer datasets including colorectal and gastric cancers. Experimental results show that the proposed method outperforms other classification models and can facilitate improved cancer diagnosis in clinics.