Anomaly Detection for Security
Download as PPTX, PDF•7 likes•1,834 views
The document discusses real-time analytics and anomaly detection techniques for enhanced security, particularly focusing on identifying phishing attacks and rogue agents. It highlights various methods for anomaly detection, including static thresholds, clustering, and advanced techniques, while also addressing challenges like the base rate fallacy. The goal is to develop autonomous systems that can detect and automatically remediate security incidents in near real-time.
Anomaly Detection for Security
- 1. Anomaly Detection for Security Cody Rioux - @codyrioux Real-Time Analytics - Insight Engineering
- 2. Overview. ● Real-Time Analytics ● Anomaly: Fast Incident Detection ○ Techniques ○ Case Study: Detecting Phishing ○ Challenges: Base Rate Fallacy ● Outlier: Identifying Rogue Agents ○ Clustering ○ Case Study: Cleaning Up Rogue Agents ● Recap
- 3. We are drowning in information but starved for knowledge. - John Naisbitt Real-Time Analytics
- 4. Real-Time Analytics ● Part of Insight Engineering. ● Build systems that make intelligent decisions about our operational environment. ○ Make decisions in near real-time. ○ Automate actions in the production environment. ● Support operational availability and reliability.
- 6. Case Study: Phishing ● Just hired as the only security staff at a startup. ● Fell victim to a phishing attack last week. ○ They did not know it happened when it was happening. ○ They did not know what to do about it ● You’re tasked with solving this problem.
- 7. Incident Detection for Stats Geeks Anomaly Detection
- 12. Techniques Basic ● Static thresholds ● Exponential Smoothing ● Three-sigma rule Advanced ● Robust Anomaly Detection (RAD) - Netflix ● Kolmogorov-Smirnov ● Highest density interval (HDI) ● t-digest ● Linear models
- 18. Techniques Basic ● Static thresholds - Doesn’t play well with nonstationary signals. ● Exponential Smoothing - Black Swan days like Christmas, Superbowl cause issues. ● Three-sigma rule - Works (very) well only for signals drawn from a Gaussian.
- 19. Show me the Money! ● No threshold configuration ● We require examples of normal, not examples of anomaly ● Automatically adapt to moving signals ● Higher accuracy enables automatic reaction ● Ensemble (combination) of techniques eliminates some downsides
- 20. Base Rate Fallacy Intrusion is comparatively rare which affords you many opportunities to generate a false positive.
- 21. Base Rate Fallacy ● 10,000 log entries ● 99% Accuracy ● 0.01% Intrusions 1 Real incident 100 false + and 10% chance of false -
- 22. Case Study So far we can automatically alert interested parties to the possibility of an intrusion.
- 23. Identifying Rogue Agents in a Production Environment Outlier Detection
- 25. Rogue Agents? ● Identify brute force attempts on login systems ● Flag cheaters in online video games ● Identify participating ip addresses in a phishing scam
- 27. Case Study Revisited You’ve devised an automated technique for identifying attacks, now we require an autonomous system for remediation of attacks.
- 28. Goal: identify accounts and IP Addresses that are not behaving like their peers.
- 29. Clustering ● DBSCAN ● K-Means ● Gaussian Mixture Models Conceptually ● If a point belongs to a group it should be near lots of other points as measured by some distance function.
- 31. Case Study Revisited Lets cluster accounts based on their login habits and initiate an automatic password reset and notification.
- 32. Case Study Revisited Lets cluster IP addresses based on their login habits and automatically ban them.
- 33. Full stack autonomous incident detection and remediation. Recap
- 34. Case Study Recap ● Anomaly Detection enables us to... ○ Automatically identify potential attacks in real-time. ○ Notify interested parties of the attack. ○ React to those attacks without user intervention. ● Outlier Detection with Clustering enables us to… ○ Identify rogue agents within the environment. ○ Reset customer passwords for potentially compromised accounts. ○ Ban IP Addresses identified to be participating in the phishing scheme.
- 35. Literature Machine Learning: The High Interest Credit Card of Technical Debt (Sculley et al., 2014)
- 36. Literature ● The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection (Alexsson, 1999) ● Practical Machine Learning: A New Look at Anomaly Detection (Dunning, 2014) ● ALADIN: Active Learning of Anomalies to Detect Intrusion (Stokes and Platt, 2008) ● Distinguishing cause from effect using observational data: methods and benchmarks (Mooij et al., 2014) ● Enhancing Performance Prediction Robustness by Combining Analytical Modeling and Machine Learning (Didona et al., 2015)
- 37. Implementations ● Robust Anomaly Detection (RAD) - Netflix ● Seasonal Hybrid ESD - Twitter ● Extendible Generic Anomaly Detection System (EGADS) - Yahoo ● Kale - Etsy