SMU Data Science Review

Enhancing Network Security through Dual-Layer Log Analysis: Integrating Machine Learning Classifiers with Large Language Models for Intelligent Anomaly Detection

Anthony Burton-Cordova, Southern Methodist UniversityFollow
O'Neil Gray, Southern Methodist UniversityFollow
Mohammad Al Rousan, Southern Methodist UniversityFollow

Abstract

This paper presents an innovative approach to enhancing network security by integrating machine learning algorithms with fine-tuned large language models (LLMs) to provide an expert assistant querying. The proposed method utilizes machine learning for efficient preprocessing and feature extraction from log data, followed by the application of a fine-tuned LLM to analyze and interpret anomalies with greater accuracy. This dual-layer detection system is designed to improve the identification of subtle and sophisticated security threats. The research team’s extensive evaluation using real-world log datasets indicates that the combined approach increases detection rates and communicates results in an understandable manner, demonstrating its potential for improving overall network security management.

Recommended Citation

Burton-Cordova, Anthony; Gray, O'Neil; and Al Rousan, Mohammad (2024) "Enhancing Network Security through Dual-Layer Log Analysis: Integrating Machine Learning Classifiers with Large Language Models for Intelligent Anomaly Detection," SMU Data Science Review: Vol. 8: No. 3, Article 1.
Available at: https://scholar.smu.edu/datasciencereview/vol8/iss3/1