Convolutional Transformer via Graph Embeddings for Few-shot Toxicity and Side Effect Prediction

Abstract

The prediction of chemical toxicity and adverse side effects is a crucial task in drug discovery.Graph neural networks (GNNs) have accelerated the discovery of compounds with improved molecular profiles for effective drug development.Recently, Transformer networks have also managed to capture the long-range dependence in molecules to preserve the global aspects of molecular embeddings for molecular property prediction.In this paper, we propose a few-shot GNN-Transformer, FS-GNNCvTR to face the challenge of low-data toxicity and side effect prediction.Specifically, we introduce a convolutional Transformer to model the local spatial context of molecular graph embeddings while preserving the global information of deep representations.Furthermore, a two-module meta-learning framework is proposed to iteratively update model parameters across fewshot tasks with limited available data.Experiments on small-sized biological datasets for toxicity and side effect prediction, Tox21 and SIDER, demonstrate a superior performance of FS-GNNCvTR compared to standard graph-based methods.The code and data underlying this article are available in the repository, https://github.com/larngroup/FS-GNNCvTR.