Hierarchical feature block ranking for data-efficient intrusion detection modeling

Abstract

The intrusion detection field has been increasing the adoption of newer datasets after relying mainly on KDD99 and NSL-KDD. Both the height and the width of the newer datasets have increased substantially since they are geared towards evaluation by machine learning methods. The feature sets however are most often statistics, derived either from the packets, or more commonly from the (reconstructed) flows. The ease with which connected clusters of features can be extracted as well as the tendency to be overinclusive to provide researchers with as much data as possible has introduced significant bloat in the datasets. In order to improve the effective and efficient use of the datasets, this article proposes a hybrid feature selection mechanism based on a first-pass filter method and a second-pass embedded method with a central role for statistical testing to identify hierarchies of dominant feature sets. The non-destructive approach allows for the hierarchies to be inspected, interpreted and related to each other. The proposed approach is validated by constructing the feature hierarchies at three different resolutions for all recent datasets published by the Canadian Institute for Cybersecurity (IDS2017, DoS2017, IDS2018 and DDoS2019, millions of samples, 76 features). Three standard supervised learners were given increasing access to the feature (blocks) in terms of their hierarchical position. The results show that attack classes with a clear network component can be detected with cross-validated balanced accuracy, precision and recall above 99%, even when the classification model has been built from just 1 to 4 features, while additionally under a very restrictive sampling regimen: training (0.8%), validation (0.2%) and testing (99%). When selecting models only for classification performance more attack classes are detected more reliably, and while this increases feature use to an average of 12, this is still preferable over using the datasets' standard set of 76 features.