DeepRank: Test Case Prioritization for Deep Neural Networks

Abstract

Deep neural networks (DNNs) have been widely used in safety-critical fields such as autonomous driving and medical diagnosis.However, DNNs are easily disturbed to make wrong decisions, which may lead to loss of life or property.Therefore, it is vital to test DNN adequately.In practice, to reveal the incorrect behavior of DNN and improve its robustness, testers usually need massive labeled data to test and optimize DNN.However, labeling test inputs to detect the correctness of DNN predictions is an expensive and time-consuming task that even affects the efficiency of DNN testing.To relieve the labeling-cost problem, we propose DeepRank, a test case prioritization technique based on cross-entropy loss.The key idea of DeepRank is that the higher the loss value of a test case relative to the DNN, the more likely it is to be mispredicted and the more conducive it is to improve the robustness of the DNN through retraining.Therefore, the cross-entropy loss value can be used for test case prioritization.We experimentally validate our approach on two datasets and three DNNs models.The experimental results demonstrate that DeepRank is significantly better than existing test case prioritization methods regarding fault-revealing capability and retraining effectiveness.