Test Case Prioritization Based on Neural Networks Classification: A Replication Study and Hyper-Parameter Optimization Using Taguchi Methods

Abstract

Regression testing plays an essential role in developing a highly qualitative software system. When code is changed, it is needed to provide confidence that modifications are correct. Running all test cases would be time-consuming, however, Test Case Prioritization (TCP) could help in this direction by ordering the execution of test cases considering various criteria, for example, fault detection and time constraint. This research aims to achieve two main objectives: the first is to replicate the experiments outlined in the original article, and the second is to determine the optimal hyperparameters for existing AI models, with a particular focus on neural networks. The Taguchi approach was utilized, implementing an L4 design to optimize the layers and $hidden \ neurons$ parameters, while an L9 design also incorporated the epochs and $batch \_size$ parameters. The results of the replicated experiments confirm the original findings. Two datasets were used to discover the best parameters in both L4 and L9 designs. In the context of L4, we notice that a better performance is achieved for 2 layers instead of 3. Regarding hidden neurons number, the value 90 is better for the real dataset, however, the value 60 is preferred for the synthetic dataset. In the L9 design experiments conducted on both datasets, we observed comparable trends. The configurations that resulted in enhanced accuracy include 4 layers with 120 hidden neurons, 90 epochs, and a batch size of 64. Our findings corroborated the initial research on employing artificial intelligence for TCP, and by utilizing the Taguchi method, we identified the best parameters for the neural network models applied in TCP.