Splunk App for Stream - Einblicke in Ihren Netzwerkverkehr
0 likes539 views
The document discusses the Splunk App for Stream, which enables real-time insights into private, public and hybrid cloud infrastructures by capturing and analyzing critical events from wire data not found in logs or with other collection methods. It provides an overview of the app, what's new, important features, architecture and deployment, customer success examples, and FAQs.
Splunk App for Stream - Einblicke in Ihren Netzwerkverkehr
- 1. Copyright © 2015 Splunk Inc. The Splunk App for Stream Kai-‐Ping Seidenschnur Sr. Sales Engineer
- 2. Agenda • Splunk Enterprise • IntroducDon to Wire Data • The Splunk App for Stream Overview • What’s New • Important Features • Architecture and Deployment • Demo • Customer Success Examples • FAQ and Summary 2
- 3. Industry Leading PlaPorm For Machine Data Machine Data: Any Loca0on, Type, Volume Online Services Web Services Servers Security GPS LocaDon Storage Desktops Networks Packaged ApplicaDons Custom ApplicaDons Messaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On-‐ Premises Private Cloud Public Cloud Pla9orm Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Ques0on Developer Pla9orm Report and analyze Custom dashboards Monitor and alert Ad hoc search 3
- 4. Industry Leading PlaPorm For Machine Data Machine Data: Any Loca0on, Type, Volume Online Services Web Services Servers Security GPS LocaDon Storage Desktops Networks Packaged ApplicaDons Custom ApplicaDons Messaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On-‐ Premises Private Cloud Public Cloud Pla9orm Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Ques0on Developer Pla9orm Report and analyze Custom dashboards Monitor and alert Ad hoc search Any amount, any locaDon, any source Schema-‐ on-‐the-‐fly Universal indexing No back-‐end RDBMS No need to filter data 4
- 5. Copyright © 2015 Splunk Inc. IntroducDon to Wire Data
- 6. What’s Wire Data? " Machine data " Poly-‐structured data " AuthoritaDve record of real-‐Dme and historical communicaDon between machines and applicaDons 6 tcpdump -‐qns 0 -‐A -‐r blah.pcap 20:57:47.368107 IP 205.188.159.57.25 > 67.23.28.65.42385: tcp 480 0x0000: 4500 0214 834c 4000 3306 f649 cdbc 9f39 [email protected] 0x0010: 4317 1c41 0019 a591 50fe 18ca 9da0 4681 C..A....P.....F. 0x0020: 8018 05a8 848f 0000 0101 080a ffd4 9bb0 ................ 0x0030: 2e43 6bb9 3232 302d 726c 792d 6461 3033 .Ck.220-‐rly-‐da03 0x0040: 2e6d 782e 616f 6c2e 636f 6d20 4553 4d54 .mx.aol.com.ESMT 0x0050: 5020 6d61 696c 5f72 656c 6179 5f69 6e2d P.mail_relay_in-‐ 0x0060: 6461 3033 2e34 3b20 5468 752c 2030 3920 da03.4;.Thu,.09. 0x0070: 4a75 6c20 3230 3039 2031 363a 3537 3a34 Jul.2009.16:57:4 0x0080: 3720 2d30 3430 300d 0a32 3230 2d41 6d65 7.-‐0400..220-‐Ame 0x0090: 7269 6361 204f 6e6c 696e 6520 2841 4f4c rica.Online.(AOL 0x00a0: 2920 616e 6420 6974 7320 6166 6669 6c69 ).and.its.affili 0x00b0: 6174 6564 2063 6f6d 7061 6e69 6573 2064 ated.companies.d
- 7. Ad hoc Analysis on Wire Data Is Challenging Volume, velocity and variety make it difficult to collect, explore, analyze and visualize wire data. Distributed infrastructures introduce challenges in accessing wire data from public and hybrid clouds. Complex network environments make installaDon and management of probes and appliances laborious. 7
- 8. 8 Why Wire Data? Deep Insights Across Use Cases IT, security and business data transmit over the wire Non-‐Intrusive and Passive No impact to workloads No need for instrumentaDon and tagging of applicaDons Holis0c and Comprehensive Real-‐Dme communicaDon across various protocols Correlate with logs, events and metrics for comprehensive analyDcs
- 9. The Splunk App for Stream Overview
- 10. See Everything With the Splunk App for Stream Enables real-‐0me insights into private, public and hybrid cloud infrastructures Delivers rapid deployment, easy scale out and efficient wire data capture Capture and analyze cri0cal events not found in logs or with other collec0on methods 1 2 3 Enhance Opera0onal Intelligence With Wire Data Capture
- 11. Examples of What’s Available From the Wire 11 Performance Metrics Round Trip Time Client Request Time Server Reply Time Server Send Time Total Time Taken Base HTML Load Time Page Content Load Time Total Page Load Time Applica0on Data POST Content AJAX Data SecDon Sub-‐SecDon Page Title Session Cookie Proxied IP Address Error Message Business Data Product ID Customer ID Shopping Cart ID Cart Items Cart Values Discounts Order ID Abandoned?
- 12. 12 Enable New OperaDonal Insights • Add informaDon about applicaDon, infrastructure, security and business acDvity, without needing instrumentaDon • Support new and extends exisDng Splunk use cases across IT, security and the business with wire data capture Enhanced Opera0onal Intelligence Efficient, Cloud-‐Ready Wire Data Collec0on Fast Time to Value • Gain visibility into any public, private or hybrid cloud infrastructures with a sopware soluDon • Control data collecDon volumes with fine-‐grained protocol and aqribute filtering • Deploy quickly from interface-‐driven install • Enable rapid incident response • Easily scale out with centralized management
- 13. Beqer Insights for IT OperaDons • Get real-‐Dme granular insights to reduce MTTR without costly appliances • Analyze all applicaDons and user behavior, measure applicaDon response Dmes and trace transacDon paths • IdenDfy infrastructure performance issues, capacity constraints, changes and establish baselines Value ApplicaDon logs, infrastructure (storage, network, server) logs, performance metrics, events 13 SQL queries, DNS records, IP conversaDons, transacDon traces, ICA latency, response Dmes + Contextual Data Wire Data
- 14. Beqer Insights for App Management Protocol conversaDons on database performance, DNS lookups, client data, business transacDon paths… Measure applicaDon response Dmes, deeper insights for root-‐ cause diagnosDcs, trace transacDons paths, establish baselines, etc. Enriched View ApplicaDon logs, monitoring data, metrics, events 14 + Contextual Data Wire Data
- 15. Beqer Insights for Security • Real-‐Dme DPI with analyDcs enables easier forensics analyses and quicker incident response • Analyze user and applicaDons behavior • Respond Dmely to threats with cost-‐efficient real-‐Dme header and payload field extracDon • Baseline network traffic and understand anomalies associated with APTs and insider threats • Quick install at endpoints, on-‐premises and cloud infrastructures without expensive appliances Value + Contextual Data Firewall logs, applicaDon logs, IDS logs, network logs, perf. metrics, events 15 User and applicaDon traffic, protocol idenDficaDon (TCP, DNS, HTTP, etc.), protocol headers & payload extracDon, SSL decrypDon Wire Data
- 16. Beqer Insights for Digital MarkeDng Browser-‐level customer interacDons Customer Experience – analyze website and applicaDon boqlenecks to improve customer experience and online revenues Customer Support (online, call center) – faster root-‐cause analysis and resoluDon of customer issues with website or apps Enriched View Website log acDvity, clickstream data, metrics 16 + Contextual Data Wire Data
- 17. What’s New
- 18. Distributed Forwarder Management " More deployment flexibility " Per-‐forwarder protocol control to increases management efficiency " Tailored data collecDon by assigning different sets of protocols to groups of forwarders 18 TNS MySQL HTTP DNS TCP SIP Diameter UDP Protocol SelecDon, ConfiguraDon & DistribuDon
- 20. 20 Custom Content ExtracDon Enables Efficient Real-‐Time Insights • Easily and selecDvely analyze web traffic for security risks • IdenDfy data exfiltraDon, including PII or exposed assets • Prevent data loss, perform forensics and reduce troubleshooDng Dme Improved Security Posture Efficient Real-‐Time Business Analyses Efficient IT Ops and Applica0ons Visibility • Real-‐Dme granular insights into key business indicators from web traffic • SelecDve on-‐the-‐fly visibility into shopping carts, user interacDons, etc. • Monitor web services performance on-‐the-‐fly for quick troubleshooDng and performance analysis • Enable real-‐Dme custom protocol monitoring
- 21. 21 Stream Stats Dashboard Enables Granular Analysis of Traffic and Indexing Volume • ProacDvely plan Stream deployment with per-‐protocol visibility into applicaDons traffic bandwidth and Splunk indexing stats • EsDmate per-‐protocol Splunk indexing volume, incoming, outgoing or total traffic bandwidth
- 22. Supported Protocols and PlaPorms • UDP • TCP • HTTP • IMAP • MySQL (login/cmd/ query) • Oracle (TNS) • PostgreSQL • Sybase/SQL Server (TDS) • FTP • SMB • NFS • POP3 • SMTP • LDAP/AD • SIP • XMPP • AMQP • MAPI • IRC Supports Windows 7 (64-‐bit), Windows 2008 R2 (64 bit), Linux (32-‐bit/64-‐bit) and Mac OSX (64-‐bit) • DNS • DHCP • RADIUS • Diameter • BitTorrent • SMPP 22 Improved performance requiring less compute/memory power!
- 24. Stream Forwarder Architecture Protocol Decoder (Deep Packet Inspec0on) Events Decryp0on Request/ Response Network Interface (eth1) Standard Out (To Splunk Forwarder) Packets Streams Request/ Response Request/ Response Protocol Decoder (Deep Packet Inspec0on) Events Decryp0on Standard Out (To Splunk Forwarder) Protocol Decoder (Deep Packet Inspec0on) Events Decryp0on Standard Out (To Splunk Forwarder) Network Interface (ethN) Packets … Threads 24
- 25. Architecture: Dedicated Server 25 End Users TAP or SPAN Firewall Search Head Linux Forwarder Splunk_TA_Stream Servers Internet Splunk Indexers
- 26. Architecture: Run on Servers 26 Splunk Indexers Search Head Physical or Virtual Servers Universal Forwarder Splunk_TA_stream Physical Datacenter, Public or Private Cloud End Users Firewall Internet
- 27. Demo
- 29. Cross-‐Der Visibility Helps Break the Silos Kris Laxdal, IT Manager & Security Analyst “You cannot show up with tradi0onal packet captures tool in the boardroom. Stream and Splunk help us understand issues at the high level and if exec team wants to see the details we can drill down easily. That is what's great about Stream!” IT Opera0ons • High level view with contextual drill-‐down ability • Easy access and visibility into producDon MySQL environment helps app developers troubleshoot issues and roll out releases quicker • Improved collaboraDon between teams: IT operaDons, QA (pre-‐ producDon tesDng),security and development • Improved customer response Dmes due to real-‐Dme visibility into app issues Security • CorrelaDon against indicators of compromise helps invesDgate and miDgate APTs, potenDal data exfiltraDon & other risks Key Customer Benefits
- 30. • Granular applicaDon and network visibility drives easy remediaDon • ProacDve applicaDons and network traffic monitoring enables beqer capacity reporDng and planning • Powerful analyDcal engine enables data analyses by novice users • Quick host-‐based deployment at criDcal network segments – Ability to observe both client and server traffic Key Customer Benefits ApplicaDons Visibility for Capacity Planning Helps with Datacenter MigraDon AVP of Networks and Communica0ons, Large Na0onal Bank “I enjoyed using the Splunk App for Stream as it's giving us a bunch of different perspecDves on our traffic and beqer granularity compared to some of the other tools we used.”
- 31. ApplicaDons Visibility Drives Beqer Digital Asset Management Systems Engineer, Major Media Company “With Splunk and Stream, we have this rich data pla9orm that is bridging all the different data silos. Our MTTR went from days to minutes while the granularity and insight improved. We went from having very liqle visibility into operaDonal and security issues to full insight.” Key Customer Benefits • IT Opera0ons: improved operaDonal insight into digital asset management and streamlined lengthy processes • DevOps/app delivery: faster app releases due to visibility into app performance – Real-‐Dme insight into database queries and latencies – Cross-‐correlaDon with system-‐level performance and user access • Security: visibility into user behavior throughout enDre asset management system helps protect digital assets
- 32. Real-‐Time Insights into Database AcDvity 32 IT Infrastructure Manager, Leading Taiwanese Telco “With Stream, we are are able to roll out applicaDons faster and perform quicker invesDgaDons into operaDonal issues. The Splunk plaPorm is a single interface to all the data for our IT ops and security teams.” Key Customer Benefits • Gain deep operaDonal Oracle database access monitoring • Audit assistance: who, when, how performed database access • Client-‐side visibility • IdenDficaDon of abnormal connecDons • Resolve issues faster with cross-‐correlaDon of applicaDon logs with database-‐access monitoring • Get lightweight monitoring without impact on server performance
- 33. Wire Data Intelligence Improves Security Security Analyst, Payment Processing Company “The thing that makes Stream beqer than any other packet analysis soluDon out there is the staDsDcal analysis from Splunk Enterprise. You can apply it freely to all of the wire data, which enables me to analyze this data in ways not possible before. This visibility help us prevents external infiltraDon and avoid malicious aqacks.” Key Customer Benefits • Real-‐Dme security intelligence to prevent aqacks and infiltraDons • Baselining, trending and applying analyDcs to detect anomalies in traffic (mySQL, postgres, etc.) • Centralized management of all wire data results in operaDonal cost savings • Efficient monitoring of user authenDcaDons for audit and security • Non-‐intrusive and easy monitoring of server communicaDon • Flexible and easy integraDon with Splunk security dashboards
- 34. Streaming AnalyDcs Helps Speed Up Forensics InvesDgaDon 34 Key Customer Benefits • 90% reducDon in incident triage and invesDgaDon Dme • Deeper, quicker and easier understanding of traffic and user acDvity • Immediate insights and improved data collecDon – EliminaDon of moving pcap files around between several tools • Flexible and easy deployment on key network locaDons Security Engineer, Financial Services Ins0tu0on “The biggest value of Stream is how fast we can resolve and close security cases. Before Stream, I had to collect data from mulDple systems and it would take me an hour. With Stream, informaDon is already there and I can get answers within 5 minutes. “
- 35. FAQ and Summary
- 36. FAQ • Yes. The app enables capture of only the relevant wire data for analyDcs, through filters and aggregaDon rules • Select or deselect protocols and associated aqributes with fine-‐grained precision within the app interface Can I limit the amount of data collected with Stream? • Data volume can vary based upon the number of selected protocols, aqributes and the amount of network traffic. UDlize Stream Stats to understand the licensing impact How can I es0mate my indexing volume? • The Stream Examples App contains searches, examples and instrucDons, enabling use cases such as network security scenarios, funnel analysis, shopping cart revenue, SIP conversaDons, and applicaDon and database latencies How can I explore the data collected with Stream? 36
- 37. Enables real-‐0me insights into private, public and hybrid cloud infrastructures Delivers rapid deployment, easy scale out and efficient wire data capture Capture and analyze cri0cal events not found in logs or with other collec0on methods 1 2 3 37 See Everything with Splunk App for Stream Enhance Opera0onal Intelligence With Wire Data Capture
- 38. Thank You