T-MGCL: Molecule Graph Contrastive Learning Based on Transformer for Molecular Property Prediction

Abstract

In recent years, machine learning has gained increasing traction in the study of molecules, enabling researchers to tackle challenging tasks including molecular property prediction and drug design.Consequently, there remains an open challenge to develop a neural network architecture that can make use of extensive amounts of unlabeled data for training while still providing competitive results in various molecular property prediction tasks. To address this challenge, we propose a Molecule Graph Contrastive Learning approach based on the Transformer framework (T-MGCL). Our approach involves expanding numerous unsupervised molecular graphs and using a contrast estimator to ensure consistency among various graph expansions of the same molecule. Transformer framework is employed to consider the distance between atoms and molecular graph attributes, thereby accounting for structural information that may be overlooked by traditional graph neural networks. Our experimental results demonstrate that the T-MGCL model outperforms other models in several molecular property prediction tasks. Additionally, we observe that the attention weight learned by T-MGCL can be easily explained from a chemical perspective.