Lists
Lists
Maosa, Herbert, Ouazzane, Karim and Sowinski-Mydlarz, Viktor (2022) Real-time cyber analytics data collection framework. International Journal of Information Security and Privacy (IJISP), 16 (1). pp. 1-10. ISSN 1930-1650
For effective security, it is critical that event data is collected in near real time as possible to enable early detection and response to threats. Performing analytics from event logs stored in databases slows down the response time due to the time cost of database insertion and retrieval operations. We present a data collection framework that minimizes the need for long term storage. Events are buffered in memory, up to a configurable threshold, before being streamed in real time using live streaming technologies. The framework deploys virtualized data collecting agents that ingest data from multiple sources including external Threat Intelligence. The framework enables the correlation of events from various sources, improving detection precision. We have tested the framework in a real time, machine-learning based threat detection system. Our results show a time gain of 300 milliseconds in transmission time from event capture to analytics system, compared with storage based collection frameworks. Threat detection was measured at 95%, which is comparable to the benchmark snort IDS.
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.
Download (418kB) | Preview
 |
View Item |