Automated Multi-Choice Question Answering System using Natural Language Processing

Abstract

Machine reading comprehension not only involves reading unstructured text but also understanding it. Compared to questions of a simpler kind, this job calls for more sophisticated reading comprehension, such as logical reasoning, summarizing, and computation. They demand the intelligent system's ability to understand human language, making it an essential tool for testing its capabilities. We used unstructured corpora such as ARC corpora and Aristo mini corpora as contexts to answer multiple choice questions, which are indexed using ElasticSearch API. To answer multiple choice questions, we assigned questions from subjects such as Chemistry, and Biology; The goal is to target science-related questions instead of general questions, as they are clearly more difficult to answer. We have developed an automated multiple choice question answering template that is deployed on localhost using the Flask framework; the model accepts a question followed by a series of answer choices, and there is a PDF file upload function where the user can also upload a PDF file containing a question or a set of questions followed by a pair of candidate answers options. Searching the corpus for the appropriate context for each question is an important part of answering the question, using discriminators such as the answer discriminator and the document discriminator, because it was found that the TF-IDF score alone does not always provide the result. The best possible documents to answer the question in hand. UnifiedQA was used for modeling and a model without discriminators was also made to compare the results, and it was found that with discriminators, better performance was achieved at the expense of computational speed. A current increase of up to 7% was observed, which is huge, so better context has been shown to improve model performance and accuracy.