Pre-trained language models fine-tuned with SVM for legal textual entailment recognition

Abstract

The breakthroughs in natural language processing (NLP) are not only a crucial step in technological evolution but also deliver significant benefits across various fields demanding high intelligence and precision. One of the notable NLP applications is in the analysis and processing of legal texts. Capitalizing on this trend, the 10th Workshop on Vietnamese Language and Speech Processing (VLSP) 2023 hosted a new challenge: Legal textual entailment recognition (RTE). The task involves determining whether a given statement is logically entailed by the relevant legal passage. Our proposed method leverages a novel layer based on Support Vector Machine (SVM) kernel formulations, effectively capturing nuanced relationships in the input data. Additionally, it capitalizes on the advantages of the natural language inference (NLI) datasets which are very close to textual entailment recognition (RTE) for enhancing performance and generalization. Our approach not only yielded accurate results but also demonstrated efficiency in the use of data resources, helping our A3N1 team achieve notable accuracy, with a score of 0.7194 on the test set, and ranking third on the leaderboard.