Practical Cyborgism - Machine Learning for Bro Logs

Overview

Explore machine learning techniques for analyzing Bro logs in this conference talk from Security Onion 2016. Dive into practical applications of cyborgism, focusing on HTTP proxy logs analysis. Learn about supervised and unsupervised machine learning approaches, including binary classification with random forests and outlier detection using isolation forests. Discover how to generate synthetic abnormal data, understand decision trees, and leverage scikit-learn and Python for efficient model training, testing, and evaluation. Gain insights into identifying influential features and interpreting classifier explanations. Acquire valuable ideas for improving log file analysis and enhancing cybersecurity practices through the integration of machine learning methodologies.

Syllabus

Intro
A story we all know: Regular expressions
When's the last time you heard...?
Problem Statement: HTTP Proxy Logs
Machine Assisted Analysis
Two different types of machine learning
Supervised: Binary Classification
Classification With Random Forests
Generating synthetic abnormal data
Decision Trees
Unsupervised: Outlier Detection
Isolation Forests Liu, Ting, Zhao
A quick note about parameters
Classification With Isolation Forests
The beauty of scikit leam & python
Identifying Training & Test Data
Training, Testing & Evaluating a Model
Bonus: Most influential Features with
Analyzing Log Files
Bonus: Classifier Explanations with
Ideas for improvement