Text Summarization Generation Based on Improved Transformer Model

Abstract

In the era of big data, the number of internet users is increasing yearly, and each user receives a massive amount of information every day. The low-value density of massive text data makes it difficult to utilize its information. In the face of this problem, text summarization generation technology that can extract the paper's leading content and condense the paper's main idea is a powerful solution. In text summarization generation, abstractive summarization techniques with high readability, clear expression, and closer proximity to human language habits are often favored. However, mainstream technologies still have issues like summarizations' detachment from focus. In this paper, keyword information is employed to join the text summarization generation task, and a keyword information module is added to the Transformer to better perform the summarization generation task. Finally, the ROUGE evaluation standard is used to compare the performance of the above models on the large-scale Chinese short text summarization (LCSTS) dataset. The experimental results show that the model generates higher-quality summarizations using keyword information, and the related methods have considerable potential. Future work can explore better model structures, more suitable methods for utilizing keyword information, and more accurate keyword extraction techniques to improve the quality of abstractive summarizations.