Evaluating the Impact of One-Shot Structured Pruning: A Noise Robustness Perspective

Abstract

The increasing complexity of neural networks in combination with the growing interest in edge computing has led to a surge in the development of neural network compression techniques. These methods aim to reduce the computational complexity of the networks while maintaining a high level of performance. However, the impact of compression on the robustness of the network is often overlooked. Therefore, in this work, we investigate the robustness of compressed neural networks by assessing the way a neural network behaves under different types and levels of noise. More specifically, we propose a robustness evaluation method that employs two different evaluation methodologies: a Multi-Level Monte Carlo simulation [1] and the CIFAR-100-C [2] dataset. We compare popular neural networks with their compressed versions, generated using an XAI-inspired one-shot structured channel pruning methodology. We systematically evaluate the robustness of the models pruned for different scenarios. Our results show that the pruned models can maintain their robustness properties in the presence of noise, and that specialization through pruning can improve a model's ability to handle corruptions.