Learning-based network anomaly detection

Ghani, Humera

London Met Repository

Tools

Lists

Ghani, Humera (2026) Learning-based network anomaly detection. Doctoral thesis, London Metropolitan University.

Abstract
Documents
Details
Record

[+][-]

Abstract

The rapid proliferation of Internet of Things (IoT) networks has exposed a critical research gap: existing intrusion detection systems (IDS) are fundamentally limited in their ability to detect a diverse spectrum of evolving attacks within high-dimensional, imbalanced, and heterogeneous real-world data. Most prior IDS research relies on artificially balanced datasets and frequently excludes minority attack classes, resulting in models that lack generalisability and fail to provide adequate protection in practical IoT deployments. Furthermore, conventional approaches typically employ single method feature reduction and single-stage classifiers, which are insufficient for capturing the complex temporal and non-linear patterns inherent in IoT traffic.
This thesis is distinctly different from existing work by introducing a novel IDS framework that directly addresses these limitations. It combines advanced statistical data analysis, feature reduction, and a hierarchical classification strategy. The proposed methodology introduces two distinct feature reduction techniques: the first integrates Recursive Feature Elimination (RFE) and Principal Component Analysis (PCA), while the second technique uses an attention-based feature selection mechanism. These approaches effectively reduce the feature space by over 50% while preserving critical information, enabling robust detection of both frequent and rare attack types. The detection architecture leverages a two-stage classifier, integrating Bidirectional Long Short-Term Memory (BiLSTM) networks with Extreme Gradient Boosting (XGB), to capture complex temporal and non-linear patterns in IoT network traffic. Extensive experiments on the highly imbalanced IoT-23 and UNSW-NB15 benchmark datasets demonstrate the novelty and effectiveness of the proposed system.
The framework achieves up to 97.15% accuracy and 98.47% F1-score, with perfect detection rates for major attack classes and strong performance on minority classes without any data resampling or class exclusion. These results set new benchmarks for IoT anomaly detection and validate the system’s ability to generalise across diverse, real-world data distributions.
In conclusion, this research presents a novel, empirically validated IDS framework that addresses the key limitations of prior work: it operates effectively on real, imbalanced IoT data, detects a broad spectrum of attacks, and achieves state-of the-art quantitative results. Future work will pursue several directions to strengthen the proposed framework. The framework should be evaluated on a wider range of real-world IoT datasets, such as industrial control systems, healthcare networks, and smart city infrastructure, to provide stronger evidence of its generalisability. Adaptive feature selection that dynamically updates feature importance as attack patterns evolve, together with open-world detection methods, would help identify novel or zero-day attacks, while real-time stream processing with continual learning would enable live classification without full retraining. Developing a lightweight version through compression, quantisation, or knowledge distillation would support deployment on resource-constrained edge devices. Finally, the most pressing priority is reducing the false positive rate through novel loss functions tailored to imbalanced classification, complemented by post-hoc explainability methods to improve transparency in high-stakes settings.

Documents