Structure-aware Table-to-Text Generation with Prefix-tuning

Abstract

Table-to-text generation is designed to generate descriptive natural language for structured tables that conforms to objective facts and follows the source data. The current challenge in this field is to capture the structural information of the table and improve the quality of the generated text. The existing sequence-to-sequence approach is to linearize the table, which leads to miss captured structure information and is not conducive to the model learning contextual semantics. In this paper, we introduce structural-aware self-attention, which focuses on table structure to capture cell relationships between the same row or column. In this way, the generated descriptive text can more accurately reflect the correlation between the cells in the table, discarding irrelevant information. In order to adapt the pre-trained language model to the table-to-text generation task, we introduce prefix-tuning. Traditional fine-tuning methods update all model parameters, which leads to increased training costs. In contrast, using prefix-tuning for a more lightweight approach can improve model performance considerably. Attaching continuous prompts to tokens helps the model better understand the structure and semantics of the input sequence. All of our models are extended based on T5 and have strong competitiveness in the ToTTo dataset and Hitab dataset compared with several classical baselines.