A medical question answering system using large language models and knowledge graphs

Abstract

Question answering systems have become prominent in all areas, while in the medical domain it has been challenging because of the abundant domain knowledge. Retrieval based approach has become promising as large pretrained language models come forth. This study focuses on building a retrieval-based medical question answering system, tackling the challenge with large language models and knowledge extensions via graphs. We first retrieve an extensive but coarse set of answers via Elasticsearch efficiently. Then, we utilize semantic matching with pretrained language models to achieve a fine-grained ranking enhanced with named entity recognition and knowledge graphs to exploit the relation of the entities in question and answer. A new architecture based on siamese structures for answer selection is proposed. To evaluate the approach, we train and test the model on two Chinese data sets, NLPCC2017 and cMedQA. We also conduct experiments on two English data sets, TREC-QA and WikiQA. Our model achieves consistent improvement as compared to strong baselines on all data sets. Qualification studies with cMedQA and our in-house data set show that our system gains highly competitive performance. The proposed medical question answering system outperforms baseline models and systems in quantification and qualification evaluations.