A novel reinforcement learning model for post-incident malware investigations

Dunsin, Dipo, Ghanem, Mohamed Chahine, Ouazzane, Karim and Vassilev, Vassil (2024) A novel reinforcement learning model for post-incident malware investigations. In: The 11th IEEE International Conference on Social Networks Analysis, Management and Security, December 9-11, 2024, Gran Canaria, Spain. (In Press)

Abstract

This Research proposes a Novel Reinforcement Learning (RL) model to optimise malware forensics investigation during cyber incident response. It aims to improve forensic investigation efficiency by reducing false negatives and adapting current practices to evolving malware signatures. The proposed RL System leverages techniques such as Q-learning and the Markov Decision Process (MDP) to train the system to identify malware patterns in live memory dumps, thereby automating forensic tasks. The RL model is based on a detailed malware workflow diagram that guides the analysis of malware artefacts using static and behavioural techniques as well as machine learning algorithms. Furthermore, it seeks to address challenges in the UK justice system by ensuring the accuracy of forensic evidence. We conduct testing and evaluation in controlled environments, using datasets created with Windows operating systems to simulate malware infections. The experimental results demonstrate that RL improves malware detection rates compared to conventional methods, with the RL model's performance varying depending on the complexity and learning rate of the environment. The study concludes that while RL offers promising potential for automating malware forensics, its efficacy across diverse malware types requires ongoing refinement of reward systems and feature extraction methods.

Documents
9761:49652
[thumbnail of SNAMS 2024 RL for Malware Investigation FINAL.pdf]
Preview
SNAMS 2024 RL for Malware Investigation FINAL.pdf - Accepted Version
Available under License Creative Commons Attribution 4.0.

Download (1MB) | Preview
Details
Record
View Item View Item