All your logs are belong to you!
1 like1,434 views
The presentation by Xavier Mertens discusses the importance of effective log management and security event monitoring in organizations. It emphasizes the need for visibility, proper analysis, and compliance with legal requirements to effectively handle security incidents and manage risks. Additionally, it introduces various tools and methods for log collection, normalization, and correlation to enhance security operational capabilities.
![USB Stick Detection
• Each time an USB stick is inserted,
Windows creates a new registry entry:
HKLMSYSTEMCurrentControlSetEnumUSBSTOR
Disk&Ven_USB&Prod_Flash_Disk&Rev_0.00
• Create a new OSSEC rule:
[USB Storage Detected] [any] [] r:HKLMSYSTEMCurrentControlSet
ServicesUSBSTOR;](https://image.slidesharecdn.com/bsideslondon-talk-110427194614-phpapp02/85/All-your-logs-are-belong-to-you-38-320.jpg)
All your logs are belong to you!
- 1. All Your Security Events are Belong to ...You! BSidesLondon 2011 - Xavier Mertens
- 2. $ whoami • Xavier Mertens (@xme) • Security Consultant • CISSP, CISA, CeH • Security Blogger • Volunteer for security projects:
- 3. $ cat disclaimer.txt “The opinions expressed in this presentation are those of the speaker and do not reflect those of past, present or future employers, partners or customers”
- 5. How is Your Log-Fu? • Logs? Which logs? • It’s BORING! • Most organizations are NOT prepared to deal with security incidents • If anything can go wrong, it will! (Murphy’s law) • Enough internal resources?
- 6. Need for Visi bility! • Computer: “programmable electronic machine that performs high-speed mathematical or logical operations or that assembles, stores, correlates, or otherwise processes information” Too cool! • Integration with multiple sources increases the change to detect suspicious events. • Detect activity below the radar.
- 7. Technical Issues • Networks are complex • Some components/knowledge are outsourced • Millions of daily events • Lot of console/tools • Lot of protocols/applications
- 8. Find the Differences Aug 27 14:33:01 macosx ipfw: 12190 Deny TCP 192.168.13.1:2060 192.168.13.104:5000 in via en1 %PIX-3-313001: Denied ICMP type=11, code=0 from 192.168.30.2 on interface 2
- 9. Economic Issues • “Time is money” • Real-time operations • Downtime has a huge financial impact • Reduced staff & budget • Happy shareholders • Log management == Insurance (Risk management)
- 10. Legal Issues • Compliance requirements • Big names • Initiated by the group or business • Local laws • Due diligence & due care
- 11. Legal Requirements • Internal • You are not Big-Brother! • Team-members must be aware of the procedures • External • Notify your users & visitors which information is logged, how and for which purposes
- 12. Belgian Example: CBFA From a document published in April 2009: “Any institution that connects to the Internet must have a security policy which takes into account: ... the creation, the archiving of event logs which permit the analyze, follow-up and reporting.”
- 13. Challenges • Creation & archiving of log files • Analyze (Normalization) • Follow-up • Reporting • (Correlation)
- 14. Layer Approach Correlation Reporting Search Storage Normalization Log Collection
- 15. Raw Material • Your logs are belong to you! • If not stored internally (cloud, outsourcing), claim access to them • All applications/devices generate events • Developers, you MUST generate GOOD events
- 16. 3rd Party Sources • Vulnerabilities Databases • Blacklists (IP addresses, ASNs) • “Physical” Data • Geolocalization • Badge readers
- 17. Security Convergence • Mix of logical control: • Passwords, access-lists • Blacklists (IP addresses, AS’s, domains) • and physical control: • Badge readers • Geo-localization
- 18. The Recipe
- 19. Collection • Push or pull methods • Use a supported protocols • Open vs. Proprietary • Ensure integrity • As close as the source
- 20. Normalization • Parse events • Fill in common fields • Date, Src, Dst, User, Device, Type, Port, ...
- 21. Storage • Index • Store • Archive • Ensure integrity (again)
- 22. Search • CLI tools remain used (grep|awk|sort| tail|...) • You know Google? • Investigations / Forensic • Looking for “smoke signals”
- 23. Reporting • Automated / On-demand • Reliable only if first steps are successful • Reports must address the audience (technical vs business)
- 24. Correlation • Generation of new events based on the way other events occurred (based on their logic, their time or recurrence) • Correlation will be successful only of the other layers are properly working • Is a step to incident management
- 26. <warning> Please keep v€ndor$ away from the next slide </warning>
- 27. Let’s Kill Some Myths • Big players do not always provide the best solutions. A Formula-1 is touchy to drive! • Why pay $$$ and use <10% of the features? (the “Microsoft Office” effect) • But even free softwares have costs! • False sense of security
- 28. LM vs. SIEM • A LM (“Log Management”) addresses the lowest layers from the collection to reporting. • A SIEM (“Security Information & Event Management”) adds the correlation layer (and often incidents management tools)
- 29. Grocery Shopping • Compliance • Suspicious activity • Web applications monitoring • Correlation • Supported devices • Buying a SIEM is a very specific project
- 30. Free Tools to the Rescue
- 31. Syslog Daemons • Syslog is well implemented • Lot of forked implementations • syslogd, rsyslogd, syslog-ng • Multiple sources • Supports TLS, TCP • Several tools exists to export to Syslog (ex: SNARE) • But a hell to parse
- 32. SEC • “Simple Event Correlation” • Performs correlation of logs based on Perl regex • Produces new events, triggers scripts, writes to files • Example: track IOS devices reload type=single continue=takeNext ptype=regexp pattern=d+:d+:d+.*?(S+)s+d+:.*?%SYS-5-RELOAD: (.*) desc=(WARNING) reload requested for $1 action=pipe '%s details:$2' mail -s 'cisco event' [email protected]
- 33. OSSEC • HIDS • Log collection & parsing • Active-Response • Rootkit detection • File integrity checking • Agents (UNIX, Windows) • Log archiving
- 34. Protocols • CEF - “Common Event Format” | ArcSight • CEE - “Common Event Expression” | Mitre • RELP - “Reliable Event Logging Protocol” • SDEE - “Security Device Event Exchange” | Cisco
- 35. Miscellaneous • MySQL • iptables / ulogd • GoogleMaps API • Some Perl code • liblognorm • Cloud Services (don’t be afraid)
- 36. Some Recipes Using OSSEC
- 37. USB Stick Detection • Purpose: • Protection against data leak • Security policies enforcment • Ingredients: • OSSEC Windows Agents • Windows Registry
- 38. USB Stick Detection • Each time an USB stick is inserted, Windows creates a new registry entry: HKLMSYSTEMCurrentControlSetEnumUSBSTOR Disk&Ven_USB&Prod_Flash_Disk&Rev_0.00 • Create a new OSSEC rule: [USB Storage Detected] [any] [] r:HKLMSYSTEMCurrentControlSet ServicesUSBSTOR;
- 39. MySQL Integrity Audit • Purpose: • Track changes on some MySQL tables. • Ingredients: • MySQL Triggers • MySQL UDF (“User Defined Functions”) • OSSEC parser + rules
- 41. Temporary Tables • Purpose: • To detect suspicious users & IP’s • Ingredients: • MySQL • Patch ossec-analysisd • External public sources
- 42. Temporary Tables
- 43. Using Google Maps • Purpose: What’s the difference between: 195.75.200.200 (Netherlands) 195.76.200.200 (Spain) • Ingredients: • Google Maps API • Perl scripting • Geo-IP API (Geocity Lite)
- 45. OSSEC Dashboard • Because one picture is worth a thousand words! • Ingredients • MySQL OSSEC support • LAMP server
- 46. OSSEC Dashboard
- 47. More Visibility • LaaS (Loggly) • Splunk • Secviz.org
- 48. Conclusions • The raw material is already yours! • The amount of data cannot be reviewed manually. • Suspicious activity occurs below the radar. • Stick to your requirements! • It costs $$$ and HH:MM • Make your logs more valuable via external sources
- 49. Thank You! Q&A? http://blog.rootshell.be http://twitter.com/xme