Présentation et démo ELK/SIEM/Wazuh
2 likes2,705 views
This document provides an overview and demonstration of using open source tools for security information and event management (SIEM). It begins with an introduction to SIEM and the ELK stack (Elasticsearch, Logstash, Kibana) for data aggregation, correlation, alerting and dashboards. The document demonstrates using Logstash to parse Apache logs and load them into Elasticsearch. It also discusses clustering and sizing requirements. Finally, it introduces Wazuh as an open source SIEM solution built on OSSEC and the ELK stack.
![11 The ELK stackParse Apache access logs with Logstash
Original logs
178.194.37.205 - - [10/Feb/2017:16:00:12 +0100] "POST /wp-admin/admin-ajax.php HTTP/1.1" 200 102
"https://www.clevernetsystems.com/wp-admin/post.php?post=5674&action=edit" "Mozilla/5.0 (X11; Fedora; Linux x86_64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
54.205.244.176 - - [10/Feb/2017:16:00:23 +0100] "GET /monitoring-mysql-replication-with-munin/feed/ HTTP/1.1" 200 887
"http://www.google.com" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43
Safari/537.31"
108.61.68.156 - - [10/Feb/2017:16:00:25 +0100] "GET /installing-rhel-packages-without-network-connection/ HTTP/1.1" 200
14379 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71
Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /recruitment/ HTTP/1.1" 200 9093 "-" "Mozilla/5.0 (Windows NT 6.1)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /wp-content/themes/enfold/css/grid.css?ver=2 HTTP/1.1" 200 2050
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/css/base.css?ver=2 HTTP/1.1" 200 3990
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/js/aviapopup/magnific-popup.css?ver=1
HTTP/1.1" 200 1914 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"](https://image.slidesharecdn.com/presentationelksiemslidesmeetup-170718124808/85/Presentation-et-demo-ELK-SIEM-Wazuh-11-320.jpg)
Présentation et démo ELK/SIEM/Wazuh
- 1. 1 Open Source SIEM in 2017Geneva Open Source Meetup 20170629 – Café Voisins Jérôme Steunenberg Clément Hampaï Société romande spécialisée dans les solutions d'infrastructure, de développement web et logiciels sur mesure et de data intelligence https://www.meetup.com/fr- FR/Geneve-Open-Source-Meetup/ https://www.meetup.com/fr- FR/Lausanne-Open-Source-Meetup/ Merci Café Voisins!
- 2. 2 ProgrammeGeneva Open Source Meetup 20170629 – Café Voisins 18h30 : Accueil des participants 19h : Présentation ELK/SIEM/Wazuh 20h15 : Q&A 20h30 : Buvons un verre !
- 3. 3 Open Source SIEM in 2017By Clever Net Systems
- 4. 4 Open Source SIEMWhat is SIEM ? SIEM = Security Information and Event Management = SIM (security information management / long-term log management) + SEM (security event management / real-time monitoring)
- 5. 5 Open Source SIEMCapabilities of SIEM Data aggregation: exhaustive, comprehensive and consolidated centralization of logs Correlation: event linking through common attributes in order to extract meaning from raw data Alerting: automatic analysis of correlated data or raw events turned into alerts Dashboards: centralized high-level overview of data Compliance: automatic gathering of compliance data, reporting on level of compliance Retention: retention of data due to compliance requirements and/or for long term analysis Forensic analysis: study of what happened
- 6. 6 Open Source SIEMWhich events do we correlate ? Logs • Syslogs / Windows WMI event logs / Network and firewall logs • Application & DB logs Scan results • File integrity checking • Registry keys integrity checking (Windows) • Signature based malware / rootkits detection • Antivirus software logs Behavioral monitoring • Netflow, Ntop, Nagios, Centreon, etc. • Application behaviour (multiple logins, etc...) Threat detection • HIDS & NIDS • Needs threat DB (Snort, Suricata, OSSEC, etc.) • Signature & Anomaly based Vulnerability assessment • OpenVAS, Metasploit, Aircrack, Nessus, etc. • Compliance scanners (PCI-DSS, CIS, etc.)
- 7. 7 Open Source SIEMVery incomplete OSS & proprietary vendor landscape
- 8. 8 The ELK stackData centralization and correlation Logstash Elasticsearch Kibana Beats Ingest, transform and stash Visualize and navigate data Distributed, RESTful search and analytics engine Lightweight data shipper https://www.elastic.co/guide/en/logstash/current/input-plugins.html
- 9. 9 The ELK stackElastic components Open Source (free to use) • Logstash (collector / transformer) • Elasticsearch (full-text indexing) • Kibana (analysis interface) • Beats (data shipper) (previously known as logstash-forwarder) Proprietary plugins (X-Pack) • Security (prev. Shield) - access protection • Alerting (prev. Watcher) • Monitoring (prev. Marvel) • Reporting • Graph • Machine learning Costs • By JVM, not by daily data quantity (Splunk) • Yearly • Two different levels • Need three licences for a cluster • Licences comes with engineering & support
- 10. 10 The ELK stackParse Apache access logs with Logstash
- 11. 11 The ELK stackParse Apache access logs with Logstash Original logs 178.194.37.205 - - [10/Feb/2017:16:00:12 +0100] "POST /wp-admin/admin-ajax.php HTTP/1.1" 200 102 "https://www.clevernetsystems.com/wp-admin/post.php?post=5674&action=edit" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" 54.205.244.176 - - [10/Feb/2017:16:00:23 +0100] "GET /monitoring-mysql-replication-with-munin/feed/ HTTP/1.1" 200 887 "http://www.google.com" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31" 108.61.68.156 - - [10/Feb/2017:16:00:25 +0100] "GET /installing-rhel-packages-without-network-connection/ HTTP/1.1" 200 14379 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36" 198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /recruitment/ HTTP/1.1" 200 9093 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" 198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /wp-content/themes/enfold/css/grid.css?ver=2 HTTP/1.1" 200 2050 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" 198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/css/base.css?ver=2 HTTP/1.1" 200 3990 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" 198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/js/aviapopup/magnific-popup.css?ver=1 HTTP/1.1" 200 1914 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
- 12. 12 The ELK stackParse Apache access logs with Logstash Parsed logs
- 13. 13 The ELK stackDemo i ELK demo 20 minutes Technologies :
- 14. 14 The ELK stackDemo architecture
- 15. 15 The ELK stackDemo architecture
- 16. 16 The ELK stackClustering & scalability Initial empty state First index creation Additional replication node
- 17. 17 The ELK stackClustering & scalability Horizontal scaling – shard reallocation number_of_replicas = 2
- 18. 18 The ELK stackSizing Sizing requirements for 100GB / day of raw data It’s impossible to estimate the hardware and disk requirements. A large number of factors come into play. These numbers will turn out to be completely false. • 4 nodes (3 ES nodes + 1 Logstash / Kibana node) • 8 cores per node + 64GB per node (32GB for the JVM, 32GB for the system) • Virtual or physical nodes • SSD disks preferably • Only local storage (local to the node, or local to the hypervisor, no SAN!) • Disk space requirements vary depending on amount of daily data and retention policy • Multiply disk space requirements by 1.5 with regards to raw data • Multiply by number_of_replicas Ex: 100GB / day and 3 months retention with 2 replicas = 27TB
- 19. 19 WazuhWazuh (OSSEC + ELK) as an OSS SIEM solution
- 21. 21 WazuhDemo i Wazuh demo 15 minutes Technologies :