Bidirectional and Auto-Regressive Transformer (BART) for Indonesian Abstractive Text Summarization

Abstract

Automatic summarization technology is developing rapidly to reduce reading time and obtain relevant information in Natural Language Processing technology research. There are two main approaches to text summarization: abstractive and extractive. The challenge of abstractive summarization results is higher than abstractive because abstractive summarization produces new and more natural words. Therefore, this research aims to produce abstractive summaries from Indonesian language texts with good readability. This research uses the Bidirectional and Auto-Regressive Transformer (BART) model, an innovative Transformers model combining two leading Transformer architectures, namely the BERT encoder and GPT decoder. The dataset used in this research is Liputan6, with model performance evaluation using ROUGE evaluation. The research results show that BART can produce good abstractive summaries with ROUGE-1, ROUGE-2, and ROUGE-L values of 37.19, 14.03, and 33.85, respectively.