Over-parameterization and adversarial robustness in neural networks: an overview and empirical analysis

Thanks to their extensive capacity, over-parameterized neural networks exhibit superior predictive capabilities and generalization. Nevertheless, having a large parameter space is considered one of the main suspects of the neural network vulnerability to adversarial examples– input samples crafted ad hoc to induce a desired misclassification. The relevant literature has presented contradictory claims regarding the robustness of over-parameterized networks. These contradictory findings might be due to the failure of the attack employed to evaluate the robustness of the networks. Previous research has demonstrated that depending on the considered model, the algorithm employed to generate adversarial examples may not function properly, leading to overestimating its robustness. In this work, we empirically study the robustness of over-parameterized networks against adversarial examples. However, unlike the previous works, we also evaluate the considered attack’s reliability, employing the tools at the state of the art, to support the results’ veracity. Our results show that over-parameterized networks are more robust to adversarial attacks than their under-parameterized counterparts.