Transformer-Based Framework for Enhancing Software Defect Prediction: Integration with LSTM and Hybrid Learning

Abstract

Software defect prediction (SDP) represents an essential facet of software quality assurance, facilitating the early identification of potential defects minimizing development costs and optimizing efficiency. This paper advances current work by applying transformer-based deep learning architectures for defect prediction and overcoming the limitations of structures such as Long Short-Term Memory (LSTM) neural networks. By recognizing transformers' powerful attention mechanism, we introduce a novel SDP model capable of capturing complex dependencies that exist in software code. The proposed model will use datasets from the PROMISE repository and further evaluated in contrast to LSTM and hybrid machine learning (ML) models. This paper will also investigate cross-project defect prediction employing heterogeneous datasets and the use of transfer learning methods to generalize learning across software projects. Results from the experimental tasks demonstrate that both transformer-based models outperformed LSTM and traditional ML algorithms regarding precision, recall, and F1 scores, particularly for tasks based on large-scale and imbalanced datasets. The current study illustrates the possibility of using transformers for not only static defect prediction but also demonstrates the feasibility for dynamic and real time tracking for defect prediction in evolving software systems. This study identifies new directions for future research development regarding the application of transformers for automated software quality assurance.