Oak Ridge Cyber Analytics: Methods for Advanced Intrusion Detection and Cyber Security Data Analysis

Abstract

Two technology gaps that inhibit organizational cyber security defense systems are the absence of a systematic approach to correlating events across intrusion detection tools, and the lack of reliable analysis tools to filter irrelevant alerts and highlight the events or behaviors that merit attention. This report outlines the research directions pursued to address these technology gaps, and the implementation of a proof-of-concept system to advance the state-of-the-art in intrusion detection and cyber security data analysis. The diversity, volume, and complexity of cyber security data warrants a multi-faceted approach to analysis. Semi-supervised machine learning, leveraging live network statistics, is used to filter alerts produced by off-the-shelf intrusion detection tools. The clustering of the raw alert text produced by niche intrusion detection tools is used to provide a summary view of reported cyber security events and the ability to navigate quickly to collections of alerts that are the most concerning. Swarm intelligence algorithms are used to characterize temporal patterns in the cyber security data, and to identify those behaviors in the data stream that are consistent with known attack behaviors. In addition, the developed toolset provides the software infrastructure to collect and merge multi-modal, multi-source cyber security data, and provides an interactive environment for analysis. Each of the research areas is detailed as well as a report of the prototype implementation of the software system that combines their capabilities.