Towards Model Robustness and Generalization Against Adversarial Examples for Deep Neural Networks

Abstract

Recent years have witnessed the remarkable success of deep neural network (DNN) models spanning a wide range of applications including image classification, image generation, object detection and natural language processing. Despite the impressive performance boosting over various learning tasks, DNNs are demonstrated to be strikingly vulnerable to certain well-crafted adversarial perturbations. While such perturbations are imperceptible to human, they can easily mislead the prediction of DNNs with high confidence. Along with the increasing deployment of DNN models in safety-critical scenarios, it becomes extremely crucial to ensure model robustness against potential adversarial attacks. One of the most popular methods to defend adversarial attacks is adversarial training method. In this thesis, we aim to provide new understanding on adversarial example and analyze the current adversarial training methods from perspectives of latent representation/distribution, smoothness, optimization and robustness generalization. Moreover, we also analyze the relationship between robustness and generalization. For latent representation adversarial examples, we focus on considering how to learn the robust representations and the latent distribution which retains the more structure information of clean data distribution. For smoothness, we describe two methods to promote the latent and output smoothness of deep neural networks and analyze the relationship between smoothness and robust generalization. For optimization of adversarial training, we analyze the drawback of adversarial training and introduce a better optimization method for adversarial training. For robustness generalization, we analyze why robustness generalization is hard to achieve and introduce a simple but effective method to improve the robustness generalization of adversarial training. Finally, we analyze the relationship between robustness and generalization theoretically and empirically.