Machine Learning based Intrusion Detection as a Service: Task Assignment and Capacity Allocation in a Multi-tier Architecture

Abstract

Intrusion Detection Systems (IDS) play an important role for detecting network intrusions. Because the intrusions have many variants and zero days, traditional signature- and anomaly-based IDS often fail to detect it. Machine learning (ML), on the other hand, has better capabilities for detecting variants. In this paper, we adopt ML-based IDS which consists of three in-sequence tasks: pre-processing, binary detection, and multi-class detection. We proposed ten different task assignments, which map these three tasks into a three-tier network for distributed IDS. We evaluated these with queueing theory to determine which tasks assignments are more appropriate for particular service providers. With simulated annealing, we allocated the total capacity appropriately to each tier. Our results suggest that the service provider can decide on the task assignments that best suit their needs. Only edge or a combination of edge and cloud could be utilized due to their shorter delay and greater operational simplicity. Utilizing only the fog or a combination of fog and edge remains the most private, which allows tenants to not have to share their raw private data with other parties and save more bandwidth. A combination of fog and cloud is easier to manage while still addressing privacy concerns, but the delay was 40% slower than the fog and edge combination. Our results also indicate that more than 85% of the total capacity is allocated and spread across nodes in the lowest tier for pre-processing to reduce delays.