Copyright © 2015 Splunk Inc.
Getting Started with
Splunk Enterprise
Nick Ho
Splunk Sales Engineer
nickho@splunk.com
2
Agenda
1. Splunk Overview
2. Using Splunk (Live Demonstration/Walkthrough)
• Installing & Onboard Data
• Searching
• Field Extraction
• Dashboards
• Alerting
• Analytics
3. Splunk Deployment Architecture (time permitting)
4. Splunk Communities (time permitting)
5. Q&A
4
What is machine data?
Challenges: Volume | Velocity | Variety | Variability
GPS,
RFID,
Hypervisor,
Web Servers,
Email, Messaging,
Clickstreams, Mobile,
Telephony, IVR, Databases,
Sensors, Telematics, Storage,
Servers, Security Devices, Desktops
4
Splunk’s Mission:
Making machine data accessible,
usable and valuable to everyone.
5
What Does Machine Data Look Like?
Sources
Order Processing
Twitter
Care IVR
Middleware
Error
6
Machine Data Contains Critical Insights
Customer ID Order ID
Customer’s Tweet
Time Waiting On
Hold
Twitter ID
Product ID
Company’s Twitter ID
Customer ID
Order ID
Customer ID
Sources
Order Processing
Twitter
Care IVR
Middleware
Error
7
Splunk Unlocks Critical Insights
Order ID
Customer’s Tweet
Time Waiting On
Hold
Product ID
Company’s Twitter ID
Order ID
Customer
ID
Twitter ID
Customer ID
Customer ID
Sources
Order Processing
Twitter
Care IVR
Middleware
Error
8
THE Industry Leading Platform For Machine Data
Machine Data: Any Location, Type, Volume
Online
Services Web
Services
Servers
Security GPS
Location
Storage
Desktops
Networks
Packaged
Applications
Custom
ApplicationsMessaging
Telecoms
Online
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
On-
Premises
Private
Cloud
Public
Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
Developer
Platform
Report
and
analyze
Custom
dashboards
Monitor
and alert
Ad hoc
search
Universal
Machine Data
Platform
No backend database
Schema-on-the-fly
No need to filter data
Quick time to value
Agile reporting and analytics
Real-time architecture
9
IT Service
Intelligence
ITSI
Mainframe
Data
VMware
Platform for Machine Data
Splunk also offers a wealth of apps
to address many use cases
Exchange PCISecurity
DB Connect MobileForwarders
Syslog /
TCP / Other
Sensors &
Control Systems
Rich Ecosystem of Apps
Stream
Passionate
and Vibrant
Community
750 1000 free apps
on
Splunkbase.com
Packet Analysis
(Wire Data)
- App Response Time
- Detect unauthorized access
Import & Correlate
external DB data
- 3rd party tools
- Enrich data already in
Splunk
Place Splunk search & analytics
on top of Hadoop/noSQL
cluster
Mobile Application
Performance Management (APM)
- App Crashes
- User Experience
Installing & Using
Splunk
(Live Demonstration &
Walkthrough)
12
Wireless SID: splunk Password: splunk2016
1. Download Splunk Enterprise -
http://www.splunk.com/en_us/download/splunk-enterprise.html
– Or Google “splunk download” -> Download Splunk Enterprise for Free
2. Download Tutorial Data – http://www.splunkbook.com , 3rd link under
“Related Links” OR
http://docs.splunk.com/images/Tutorial/tutorialdata.zip
Downloading Splunk Enterprise + Tutorial Data
14
IMPORT THE ZIP FILE, not individual files within it:
http://www.splunkbook.com
(sample data is located under ‘related links’ section)
Log into Splunk – http://127.0.0.1:8000 username=admin
password=changeme
To add the file to Splunk:
– Click Add Data
– Click Upload files from my computer.
– Drag and drop you sample data zip file.
– Review and Finish.
Getting Data into Splunk
We will import
sample web
ecommerce store
events
15
Common problems at this point
License expired (already had older version installed)
– Close browser, empty cache, open browser. If that doesn’t work:
– Stop Splunk.
– Uninstall all Splunk versions
 Windows Control Panel->Uninstall programs->Splunk
 OS X. Finder->Applications->Right click Splunk, Move to trash
– Reinstall
– Start Splunk
Can’t start Splunk
– Windows, Search Control panel ->Services->Splunk start
– Linux; cd <SPLUNK dir>/splunk/bin;./splunk start
Let’s get our hands dirty!
17
Searches used
buttercupgames 4*
buttercupgames status=4*
buttercupgames status!=200 | top limit=20 status
buttercupgames status !=200 | timechart count
buttercupgames status!=200 | stats count by status | where count > 700
18
Searches used
buttercupgames status=403 OR status=404 | stats count sparkline by
uri_path
buttercupgames status=404 | timechart count | trendline sma3(count)
buttercupgames status!=200 | timechart count | predict count as
predictedCount
buttercupgames status!=200 | iplocation clientip | geostats count
buttercupgames status!=200 | iplocation clientip | eval
featureId=Country | stats count by featureId | geom geo_countries
19
Dashboard
Deployments &
Architecture
21
Single Instance or Distributed?
Single environment Distributed Environment
Recommended Specs:
6X2 Core CPUs/12GB RAM/800+
IOPs
A Splunk install can be one or all roles…
22
Scales to Hundreds of TBs/Day
Enterprise-class Scale, Resilience and Interoperability
Collect machine data from thousands sources via Splunk forwarders
Compress and store data on Splunk Indexers
Initiate searches and visualize results via Search Heads
Forwarders
Indexer
Search Head
23
Scalability & High Availability
Forwarders load balance across
Indexers
Indexed data can be replicated across
peers and different physical sites
Search Heads can be clustered to
eliminate single point of failure and
handle large search loads
24
Over 1000 Apps @ http://splunkbase.splunk.com
2
25
Time to start SPLUNKING!!!
Documentation
– http://www.splunk.com/base/Documentation
Technical Support
– http://www.splunk.com/support
Videos
– http://www.splunk.com/videos
Education
– http://education.splunk.com
Community
– http://answers.splunk.com
• Splunk Book
– http://splunkbook.com
Where do I go
for help?
Copyright © 2015 Splunk Inc.
SEPT 26-29, 2016
WALT DISNEY WORLD, ORLANDO
SWAN AND DOLPHIN RESORTS
• 5000+ IT & Business Professionals
• 3 days of technical content
• 165+ sessions
• 80+ Customer Speakers
• 35+ Apps in Splunk Apps Showcase
• 75+ Technology Partners
• 1:1 networking: Ask The Experts and Security
Experts, Birds of a Feather and Chalk Talks
• NEW hands-on labs!
• Expanded show floor, Dashboards Control
Room & Clinic, and MORE!
The 7th Annual Splunk Worldwide Users’ Conference
PLUS Splunk University
• Three days: Sept 24-26, 2016
• Get Splunk Certified for FREE!
• Get CPE credits for CISSP, CAP, SSCP
• Save thousands on Splunk education!
#splunkconf2016
2
Thank You

Getting Started with Splunk Enterprise Hands-On Breakout Session

  • 1.
    Copyright © 2015Splunk Inc. Getting Started with Splunk Enterprise Nick Ho Splunk Sales Engineer [email protected]
  • 2.
    2 Agenda 1. Splunk Overview 2.Using Splunk (Live Demonstration/Walkthrough) • Installing & Onboard Data • Searching • Field Extraction • Dashboards • Alerting • Analytics 3. Splunk Deployment Architecture (time permitting) 4. Splunk Communities (time permitting) 5. Q&A
  • 3.
    4 What is machinedata? Challenges: Volume | Velocity | Variety | Variability GPS, RFID, Hypervisor, Web Servers, Email, Messaging, Clickstreams, Mobile, Telephony, IVR, Databases, Sensors, Telematics, Storage, Servers, Security Devices, Desktops 4 Splunk’s Mission: Making machine data accessible, usable and valuable to everyone.
  • 4.
    5 What Does MachineData Look Like? Sources Order Processing Twitter Care IVR Middleware Error
  • 5.
    6 Machine Data ContainsCritical Insights Customer ID Order ID Customer’s Tweet Time Waiting On Hold Twitter ID Product ID Company’s Twitter ID Customer ID Order ID Customer ID Sources Order Processing Twitter Care IVR Middleware Error
  • 6.
    7 Splunk Unlocks CriticalInsights Order ID Customer’s Tweet Time Waiting On Hold Product ID Company’s Twitter ID Order ID Customer ID Twitter ID Customer ID Customer ID Sources Order Processing Twitter Care IVR Middleware Error
  • 7.
    8 THE Industry LeadingPlatform For Machine Data Machine Data: Any Location, Type, Volume Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premises Private Cloud Public Cloud Platform Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Question Developer Platform Report and analyze Custom dashboards Monitor and alert Ad hoc search Universal Machine Data Platform No backend database Schema-on-the-fly No need to filter data Quick time to value Agile reporting and analytics Real-time architecture
  • 8.
    9 IT Service Intelligence ITSI Mainframe Data VMware Platform forMachine Data Splunk also offers a wealth of apps to address many use cases Exchange PCISecurity DB Connect MobileForwarders Syslog / TCP / Other Sensors & Control Systems Rich Ecosystem of Apps Stream Passionate and Vibrant Community 750 1000 free apps on Splunkbase.com Packet Analysis (Wire Data) - App Response Time - Detect unauthorized access Import & Correlate external DB data - 3rd party tools - Enrich data already in Splunk Place Splunk search & analytics on top of Hadoop/noSQL cluster Mobile Application Performance Management (APM) - App Crashes - User Experience
  • 9.
    Installing & Using Splunk (LiveDemonstration & Walkthrough)
  • 10.
    12 Wireless SID: splunkPassword: splunk2016 1. Download Splunk Enterprise - http://www.splunk.com/en_us/download/splunk-enterprise.html – Or Google “splunk download” -> Download Splunk Enterprise for Free 2. Download Tutorial Data – http://www.splunkbook.com , 3rd link under “Related Links” OR http://docs.splunk.com/images/Tutorial/tutorialdata.zip Downloading Splunk Enterprise + Tutorial Data
  • 11.
    14 IMPORT THE ZIPFILE, not individual files within it: http://www.splunkbook.com (sample data is located under ‘related links’ section) Log into Splunk – http://127.0.0.1:8000 username=admin password=changeme To add the file to Splunk: – Click Add Data – Click Upload files from my computer. – Drag and drop you sample data zip file. – Review and Finish. Getting Data into Splunk We will import sample web ecommerce store events
  • 12.
    15 Common problems atthis point License expired (already had older version installed) – Close browser, empty cache, open browser. If that doesn’t work: – Stop Splunk. – Uninstall all Splunk versions  Windows Control Panel->Uninstall programs->Splunk  OS X. Finder->Applications->Right click Splunk, Move to trash – Reinstall – Start Splunk Can’t start Splunk – Windows, Search Control panel ->Services->Splunk start – Linux; cd <SPLUNK dir>/splunk/bin;./splunk start
  • 13.
    Let’s get ourhands dirty!
  • 14.
    17 Searches used buttercupgames 4* buttercupgamesstatus=4* buttercupgames status!=200 | top limit=20 status buttercupgames status !=200 | timechart count buttercupgames status!=200 | stats count by status | where count > 700
  • 15.
    18 Searches used buttercupgames status=403OR status=404 | stats count sparkline by uri_path buttercupgames status=404 | timechart count | trendline sma3(count) buttercupgames status!=200 | timechart count | predict count as predictedCount buttercupgames status!=200 | iplocation clientip | geostats count buttercupgames status!=200 | iplocation clientip | eval featureId=Country | stats count by featureId | geom geo_countries
  • 16.
  • 17.
  • 18.
    21 Single Instance orDistributed? Single environment Distributed Environment Recommended Specs: 6X2 Core CPUs/12GB RAM/800+ IOPs A Splunk install can be one or all roles…
  • 19.
    22 Scales to Hundredsof TBs/Day Enterprise-class Scale, Resilience and Interoperability Collect machine data from thousands sources via Splunk forwarders Compress and store data on Splunk Indexers Initiate searches and visualize results via Search Heads Forwarders Indexer Search Head
  • 20.
    23 Scalability & HighAvailability Forwarders load balance across Indexers Indexed data can be replicated across peers and different physical sites Search Heads can be clustered to eliminate single point of failure and handle large search loads
  • 21.
    24 Over 1000 Apps@ http://splunkbase.splunk.com 2
  • 22.
    25 Time to startSPLUNKING!!! Documentation – http://www.splunk.com/base/Documentation Technical Support – http://www.splunk.com/support Videos – http://www.splunk.com/videos Education – http://education.splunk.com Community – http://answers.splunk.com • Splunk Book – http://splunkbook.com Where do I go for help?
  • 23.
    Copyright © 2015Splunk Inc. SEPT 26-29, 2016 WALT DISNEY WORLD, ORLANDO SWAN AND DOLPHIN RESORTS • 5000+ IT & Business Professionals • 3 days of technical content • 165+ sessions • 80+ Customer Speakers • 35+ Apps in Splunk Apps Showcase • 75+ Technology Partners • 1:1 networking: Ask The Experts and Security Experts, Birds of a Feather and Chalk Talks • NEW hands-on labs! • Expanded show floor, Dashboards Control Room & Clinic, and MORE! The 7th Annual Splunk Worldwide Users’ Conference PLUS Splunk University • Three days: Sept 24-26, 2016 • Get Splunk Certified for FREE! • Get CPE credits for CISSP, CAP, SSCP • Save thousands on Splunk education! #splunkconf2016
  • 24.

Editor's Notes

  • #2 Intro Mention to people to start downloading Splunk
  • #5 What is machine data? Machine data is the definitive record of what’s happening or has happened in your technology infrastructure Often machine data is linked through common information or fields Valuable because it contains records of user behavior, infrastructure, application and service health and customer experience. All technology creates machine data and its projected to grow 40-60% compounded annually At the same time, these massive streams of data come in an array of unpredictable formats that are difficult to process and analyze in a timely manner by traditional methods (data warehouse in database)
  • #6 Machine data has lots of VARIETY and high volume. You’ll notice that machine data events are also typically time-stamped – or time-series data.   Take this example of purchasing a product on your tablet or smartphone: the purchase transaction fails you call the call center then tweet about your experience all these events are captured as they occur in the machine data generated by the different systems supporting these different interactions. each of the underlying systems can generate millions of machine data events daily that can be very difficult to understand using traditional methods.
  • #7 When we look more closely at the data we see that it contains valuable information – customer id, order id, time waiting on hold, twitter id … what was tweeted. If you can correlate and visualize related events across these disparate sources, you can build a picture of activity, behavior and experience. That’s exactly what Splunk is designed to do
  • #8 And that’s exactly what Splunk empowers organizations to do. Correlating high volume machine data from disparate data sources in real-time without the need to transform the data so organizations can make more informed decisions at the pace of their business.
  • #9 All of this is accomplished with: No backend database No custom connectors Without filtering data – no need to filter data for questions that you have now, index all your data to support questions that you’ll have in the future. Without knowing the questions before hand. While Providing a quick time to value With agile reporting and analytics All in real-time
  • #10  The Splunk platform consists of multiple products and deployment models to fit your needs. At the core we have the universal machine data platform that underpins our four key technology offerings. These include: Splunk Enterprise – for on-premise deployments Splunk Cloud – Fully managed service with 100% SLA and all the capabilities of Splunk Enterprise…in the Cloud (currently only available in North America) Hunk – for analytics on data in Hadoop Splunk Mint – to get insights into data from Mobile devices The products can pull in data from virtually any source to support multiple use cases. On top of deployments for Splunk Enterprise and Splunk Cloud, Splunk Apps extend and simplify deployments by providing pre-packaged content designed for specific use cases and data types. There is a rich ecosystem of apps that come from the Splunk community. And there are premium apps that are developed and fully supported by Splunk. You see those at the top here. For the rest of the presentation, we’ll focus on explaining Splunk Enterprise.
  • #12 It only takes minutes to download and install Splunk on the platform of your choice, bringing you fast time to value. Once Splunk has been downloaded and installed the next step is to get data into a Splunk instance. The data then becomes searchable from a single place! Since Splunk stores only a copy of the raw data, searches won’t affect the end devices data comes from. Having a central place to search your data not only simplifies things, it also decreases risk since a user doesn’t have to log into the end devices. Splunk can be installed on a single small instance, such as a laptop, or installed on multiple servers to scale as needed. The ability to scale from a single desktop to an enterprise is another of our key differentiators. When installed on multiple servers the functions can be split up to meet any performance, security, or availability requirements.
  • #23 Splunk Forwarders are lightweight components which collect Machine data throughout your environment. Forwarder deployment is highly customizable, you can have the forwarder remotely collect data or place the forwarder locally on hundreds of thousands of devices as some of our customers do. Forwarders automatically load-balance their collected machine data across a pool of Indexers, which scale horizontally on commodity hardware to adjust to your growing pool of Machine Data. Search Heads initiate map-reduced searches across the indexer tier, combine and return the results to the Splunk console or your interface of choice. Like Indexers, Search Heads can scale horizontally to meet your needs on commodity hardware.
  • #25 More than 1000 are available for download on our Website. These Apps significantly reduce the time to value and make it easy for customers to extend their visibility across common sources or use cases.
  • #27 We’re headed to the East Coast! 2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics! 165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE! 30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you! Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers. Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja! REGISTRATION OPENS IN MARCH 2016 – STAY TUNED FOR NEWS ON OUR BEST REGISTRATION RATES – COMING SOON!