Comparative Evaluation of Large Language Models for Abstractive Summarization

Abstract

With applications across multiple fields, the rapid advancement of Large Language Models has ushered in a new era of natural language processing capabilities. Text summarization is one such crucial application—a skill that is extremely important in the information-rich world of today. This study compares and contrasts four cutting-edge Large Language Models, with an emphasis on how well they provide concise and understandable summaries: Google's PEGASUS, Microsoft's ProphetNet, Facebook's BART, and T5 (fine-tuned). We present an extended evaluation employing five different metrics such as METEOR, BLEU, cosine similarity, BERTScore, and ROUGE metric. Our analysis explores the intricate details of each paradigm, highlighting its advantages as well as any shortcomings. The findings presented in this research offer a comprehensive picture of the state of Large Language Model-based summarization today, illuminating the fundamental elements influencing model efficacy and laying the groundwork for further developments in the area.