SlideShare a Scribd company logo

Power of SPL

•
Splunk, profile picture
Splunk

Splunk's Search Processing Language (SPL) allows users to search, filter, modify, manipulate, enrich, and visualize machine data. SPL includes over 140 search commands for tasks like searching, filtering, modifying fields, calculating statistics, charting data over time, and identifying anomalies. During the presentation, the speaker will provide examples and demonstrations of using SPL commands to perform common search, analytics, and visualization tasks.

Technology
1 of 88
Downloaded 53 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
© 2017 SPLUNK INC.© 2017 SPLUNK INC. Power of Splunk Search Processing Language (SPL™) Mike Walker | Sales Engineer August 23, 2017 | Detroit, MI
© 2017 SPLUNK INC. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2017 Splunk Inc. All rights reserved. Forward-Looking Statements
© 2017 SPLUNK INC. Set Up Before You Can Play Download the following at splunk.com and splunkbase.com ▶ Splunk Enterprise: • https://www.splunk.com/download ▶ Power of SPL App: • https://splunkbase.splunk.com/app/3353/
© 2017 SPLUNK INC. Set Up Before You Can Play ▶ Install Power of SPL App ▶ Now we’re ready to go!
© 2017 SPLUNK INC. ▶ License expired (already had older version installed) • Close browser, empty cache, open browser. If that doesn’t work: • Stop Splunk. • Uninstall all Splunk versions • Windows Control Panel->Uninstall programs->Splunk • OS X. Finder->Applications->Right click Splunk, Move to trash • Reinstall • Start Splunk ▶ Can’t start Splunk • Windows, Search Control panel ->Services->Splunk start • Linux; cd <SPLUNK dir>/splunk/bin;./splunk start Common problems at this point
© 2017 SPLUNK INC. 1. Installation and Setup (~15min) 2. Power of SPL Walkthrough (~1h 30min) • Overview & Anatomy of a Search • SPL Commands & Examples for Searching, charting, converging, mapping, transactions, anomalies, exploring data, custom 3. Custom Visualizations (~30min) 4. SPL and the Machine Learning Toolkit (~45min) Agenda
© 2017 SPLUNK INC. SPL Overview
© 2017 SPLUNK INC. ▶ Over 140 search commands ▶ Syntax was originally based upon the Unix pipeline and SQL and is optimized for time-series data ▶ The scope of SPL includes data searching, filtering, modification, manipulation, enrichment, insertion and deletion ▶ Includes machine learning such as anomaly detection SPL Overview Disk Intermediate results table Intermediate results table Final results table
© 2017 SPLUNK INC. ▶ Flexibility and effectiveness on small and big data ▶ Late-binding schema ▶ More/better methods of correlation ▶ Not just analyze, but visualize Why Create a New Query Language? Data BIG Data
© 2017 SPLUNK INC. search and filter | munge | report | cleanup | rename sum(KB) AS "Total KB" dc(clientip) AS "Unique Customers" | eval KB=bytes/1024 sourcetype=access* | stats sum(KB) dc(clientip) SPL Basic Structure
© 2017 SPLUNK INC. SPL Examples
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time (We’ll add some custom viz here!) ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
© 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
© 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
© 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
© 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
© 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
© 2017 SPLUNK INC. Eval – Just Getting Started! Splunk Search Quick Reference Guide
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
© 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
© 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
© 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
© 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
© 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
© 2017 SPLUNK INC. ▶ Cumulative/Running Totals index=power_of_spl | reverse | streamstats sum(bytes) AS sum_bytes | timechart latest(sum_bytes) as "Total Bytes" ▶ Summary Statistics index=power_of_spl | eventstats avg(bytes) AS overall_avg_bytes | stats avg(bytes) as clientip_avg_bytes by clientip overall_avg_bytes Streamstats – Cumulative/Running Totals Statistics Examples
© 2017 SPLUNK INC. ▶ Cumulative/Running Totals index=power_of_spl | reverse | streamstats sum(bytes) AS sum_bytes | timechart latest(sum_bytes) as "Total Bytes" ▶ Summary Statistics index=power_of_spl | eventstats avg(bytes) AS overall_avg_bytes | stats avg(bytes) as clientip_avg_bytes by clientip overall_avg_bytes Streamstats – Cumulative/Running Totals Statistics Examples
© 2017 SPLUNK INC. Stats/Timechart – But Wait, There’s More! Splunk Search Quick Reference Guide
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. Converging Data Sources Index Untapped Data: Any Source, Type, Volume Ask Any Question Application Delivery Security, Compliance and Fraud IT Operations Business Analytics Industrial Data and the Internet of Things On-Premises Private Cloud Public Cloud Storage Online Shopping Cart Telecoms Desktops Security Web Services Networks Containers Web Clickstreams RFID Smartphones and Devices Servers Messaging GPS Location Packaged Applications Custom Applications Online Services DatabasesCall Detail Records Energy Meters Firewall Intrusion Prevention
© 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes lookup – Converging Data Sources Examples
© 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes Converging Data Sources Examples
© 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes appendcols – Converging Data Sources Examples
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time (Let’s add some viz!) ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId iplocation Geographic Data Examples
© 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId geostats – Geographic Data Examples
© 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId geom – Geographic Data Examples
© 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId table – Geographic Data Examples
© 2017 SPLUNK INC. Custom Visualizations
© 2017 SPLUNK INC. ▶ Native charts and maps • Bar / Line / Area charts • Bubble / Scatter plots • Gauges • Maps • Single Value Displays • Tables ▶ Generalized to fit use cases across many different areas ▶ Can be customized to some extent to cover specific use cases Native Visualizations In Splunk 42
© 2017 SPLUNK INC. ▶ Many use cases require a more specific visualization ▶ Specific custom appearance ▶ Represent data where native visualizations are not suitable • You can Splunk everything! • We won’t be able to predict every possible use case • Still uses SPL to drive visualizations Custom Visualizations FTW!
© 2017 SPLUNK INC. ▶ Platform extensibility framework and API ▶ Targeted at internal and external developers with web development / JS skills and basic knowledge of the Splunk platform ▶ Developers can make use of any third party libraries (d3.js, three.js, highcharts.js, etc…) that run in the browser* * with minor adjustments, and if third party license permits such use Custom Visualizations
© 2017 SPLUNK INC. Custom Visualizations For Admins In-product • Packaged as an app! • Installed like any other app • Users can search for visualizations on Splunkbase and directly in the product Installation
© 2017 SPLUNK INC. ▶ Choose from potentially dozens of installed visualizations! ▶ Appears as a first-class citizen alongside native visualizations • Looks and works just like packaged native visualizations ▶ Customize functionality and appearance of the visualization without touching any code, straight from the UI ▶ SPL Example provided as you hover over each visualization option. Custom Visualizations How-to
© 2017 SPLUNK INC. New Splunk Visualizations Treemap Sankey Diagram Punchcard Calendar Heat Map Parallel Coordinates Bullet GraphLocation Tracker Horseshoe Meter Machine Learning Charts Timeline Horizon Chart Multiple use cases across IT, security, IoT, and business analytics
© 2017 SPLUNK INC. Box Plot 3D scatter plot New Partner/Community Visualizations Wordcloud Donut Chart Heat Map
© 2017 SPLUNK INC. New Partner/Community Visualizations Geo Heatmap Custom Cluster Map Clustered Single Value Map Missile Map
© 2017 SPLUNK INC. Custom Visualizations – Demo!
© 2017 SPLUNK INC. Demo Screenshot #1
© 2017 SPLUNK INC. Demo Screenshot #2
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Find anomalies | inputlookup car_data.csv | anomalydetection ▶ Summarize anomalies | inputlookup car_data.csv | anomalydetection action=summary ▶ Use IQR and remove outliers | inputlookup car_data.csv | anomalydetection method=iqr action=remove Anomaly Detection – Find anomalies in your data Examples
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
© 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
© 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. Data Exploration | analyzefields | anomalies | arules | associate | cluster | contingency | correlate | fieldsummary
© 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status Cluster – Exploring Your Data Examples
© 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status fieldsummary – Exploring Your Data Examples
© 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status correlate – Exploring Your Data Examples
© 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status contingency – Exploring Your Data Examples
© 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status analyzefields – Exploring Your Data Examples
© 2017 SPLUNK INC. ▶ Predict Numeric Fields ▶ Predict Categorical Fields ▶ Detect Numerical Outliers ▶ Detect Categorical Outliers ▶ Forecast Time Series ▶ Cluster Events Machine Learning Toolkit and Showcase Examples
© 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
© 2017 SPLUNK INC. ▶ What is a Custom Command? • “| haversine origin="47.62,-122.34" outputField=dist lat lon” ▶ Why do we use Custom Commands? • Run other/external algorithms on your Splunk data • Save time munging data (see Timewrap!) • Because you can! ▶ Create your own or download as Apps • Haversine (Distance between two GPS coords) • Timewrap (Enhanced Time overlay) • Levenshtein (Fuzzy string compare) • Base64 (Encode/Decode) Custom Commands
© 2017 SPLUNK INC. ▶ Download and install App Haversine ▶ Read documentation then use in SPL! sourcetype=access* | iplocation clientip | search City=A* | haversine origin="47.62,-122.34" units=mi outputField=dist lat lon | table clientip, City, dist, lat, lon Custom Commands – Haversine Examples
© 2017 SPLUNK INC. ▶ Download and install App Haversine ▶ Read documentation then use in SPL! sourcetype=access* | iplocation clientip | search City=A* | haversine origin="47.62,-122.34" units=mi outputField=dist lat lon | table clientip, City, dist, lat, lon Custom Commands – Haversine Examples
© 2017 SPLUNK INC. SPL & The Machine Learning Toolkit
© 2017 SPLUNK INC. ▶ Predict Numeric Fields ▶ Predict Categorical Fields ▶ Detect Numerical Outliers ▶ Detect Categorical Outliers ▶ Forecast Time Series ▶ Cluster Events Machine Learning Toolkit and Showcase Examples
© 2017 SPLUNK INC. Machine Learning with the Splunk Platform Visualize Share Clean Transform Operationalize Monitor Alert Build Model Search Explore Collect Data Test, Improve Models Ecosystem MLTK Choose Algorithm Ecosystem Splunk Splunk Splunk Splunk MLTK Splunk MLTK Splunk MLTK Splunk Ecosystem Splunk Real-time Data Science Pipeline Ecosystem MLTK Splunk Splunk’s App Ecosystem contains 1000’s of free add-ons for getting data in, applying structure and visualizing your data giving you faster time to value. The Machine Learning Toolkit delivers new SPL commands, custom visualizations, assistants, and examples to explore a variety of ml concepts. Splunk Enterprise is the mission-critical platform for indexing, searching, analyzing, alerting and visualizing machine data. Packaged: UBA, ITSI
© 2017 SPLUNK INC. ML SPL Visualize Share Correlate Clean Munge Operationalize Monitor Alert Build Model Search Explore Universal Indexing Test, Improve Models Ecosystem MLTK Choose Algorithm Ecosystem Splunk Splunk Splunk Splunk MLTK Splunk MLTK Splunk MLTK Splunk Ecosystem Splunk fit sample apply listmodels deletemodel summary eval rex stats eventstats streamstats table … timechart chart stats geostats geom sendalert sendemail table … MLTK Library predict (cmd) anomalydetection (cmd) analyzefields anomalies arules associate cluster contingency correlate fieldsummary …
© 2017 SPLUNK INC. MLTK Commands The Machine Learning Toolkit contains several custom search commands that implement classic machine learning and statistical learning tasks: • fit: Fit and apply a machine learning model to search results. • apply: Apply a machine learning model that was learned using the fit command. • summary: Return a summary of a machine learning model that was learned using the fit command. • listmodels: Return a list of machine learning models that were learned using the fit command. • deletemodel: Delete a machine learning model that was learned using the fit command. • sample: Randomly sample or partition events.
© 2017 SPLUNK INC. ML-SPL Demo
© 2017 SPLUNK INC. Set Up Before You Can Play Download the following at splunkbase.com ▶ Machine Learning Toolkit: • https://splunkbase.splunk.com/app/2890/ ▶ Python for Scientific Computing: • https://splunkbase.splunk.com/app/2881/ *Note – For the Python for Scientific Computing App you need to download the platform specific version – Mac, Linux, Windows
© 2017 SPLUNK INC. | fit
© 2017 SPLUNK INC. | sample
© 2017 SPLUNK INC. | apply
© 2017 SPLUNK INC. | listmodels
© 2017 SPLUNK INC. | summary
© 2017 SPLUNK INC. | deletemodel
© 2017 SPLUNK INC. ▶Additional information can be found in: • Power of SPL App! • Docs - Search Manual • Docs - MLTK Search Commands • MLTK Quick Reference Guide • Blogs • Answers • Exploring Splunk For More Information
© 2017 SPLUNK INC. • SPL Examples App Other Useful Apps to download! • Splunk 6.x Dashboard Examples • Splunk 6.5 Overview App
© 2017 SPLUNK INC. ▶ Benefit of using your own laptop is a ‘Take it with you after’ approach – and bang on it without messing up production Splunk • Promote Personalized 50GB 6 month Dev/Test license to use for Workshop (vs. 30 day download trial) • Encourages long term playing with Splunk and continuously testing out new data sources How about an extra 50GB?
© 2017 SPLUNK INC. Q & A
© 2017 SPLUNK INC.© 2017 SPLUNK INC. Thank You

More Related Content

PPTX
Hitchhikers Guide to Service Intelligence
Splunk
 
PDF
Splunk workshop-2017-Power-of-SPL
Splunk
 
PDF
Splunk workshop-Service Intelligence
Splunk
 
PDF
Machine Data 101
Splunk
 
PDF
The Power of SPL
Splunk
 
PPTX
Power of SPL
Splunk
 
PDF
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
PDF
Hitchhikers Guide to Service Intelligence
Splunk
 
Hitchhikers Guide to Service Intelligence
Splunk
 
Splunk workshop-2017-Power-of-SPL
Splunk
 
Splunk workshop-Service Intelligence
Splunk
 
Machine Data 101
Splunk
 
The Power of SPL
Splunk
 
Power of SPL
Splunk
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
Hitchhikers Guide to Service Intelligence
Splunk
 

What's hot (20)

PPTX
The Hitchhiker's Guide to Service Intelligence
Splunk
 
PDF
Power of SPL Workshop
Splunk
 
PDF
The Power of SPL
Splunk
 
PDF
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
PPTX
Machine Data 101
Splunk
 
PDF
Splunk Discovery Indianapolis - October 10, 2017
Splunk
 
PPTX
Splunk Ninjas: New Features and Search Dojo
Splunk
 
PPTX
SplunkLive! London 2017 - Splunk Enterprise for IT Troubleshooting
Splunk
 
PPTX
SplunkLive! London 2017 - How to Earn a Seat and the Business Table with Splunk
Splunk
 
PPTX
Machine Data 101: Turning Data Into Insight
Splunk
 
PPTX
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
Splunk
 
PPTX
Splunk
Deep Mehta
 
PDF
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
PPTX
The Hitchhiker's Guide to Service Intelligence
Splunk
 
PPTX
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
Splunk
 
PPTX
SplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
Splunk
 
PPTX
SplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
Splunk
 
PDF
The Hitchhiker's Guide to Service Intelligence Workshop
Splunk
 
PPTX
Splunk Forum Frankfurt - 15th Nov 2017 - Threat Hunting
Splunk
 
PPTX
SplunkLive! Zurich 2017 - Splunk Add-ons and Alerts
Splunk
 
The Hitchhiker's Guide to Service Intelligence
Splunk
 
Power of SPL Workshop
Splunk
 
The Power of SPL
Splunk
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
Machine Data 101
Splunk
 
Splunk Discovery Indianapolis - October 10, 2017
Splunk
 
Splunk Ninjas: New Features and Search Dojo
Splunk
 
SplunkLive! London 2017 - Splunk Enterprise for IT Troubleshooting
Splunk
 
SplunkLive! London 2017 - How to Earn a Seat and the Business Table with Splunk
Splunk
 
Machine Data 101: Turning Data Into Insight
Splunk
 
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
Splunk
 
Splunk
Deep Mehta
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
The Hitchhiker's Guide to Service Intelligence
Splunk
 
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
Splunk
 
SplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
Splunk
 
SplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
Splunk
 
The Hitchhiker's Guide to Service Intelligence Workshop
Splunk
 
Splunk Forum Frankfurt - 15th Nov 2017 - Threat Hunting
Splunk
 
SplunkLive! Zurich 2017 - Splunk Add-ons and Alerts
Splunk
 
Ad

Similar to Power of SPL (20)

PDF
Power of SPL Workshop
Splunk
 
PPTX
Getting Started with Splunk Breakout Session
Splunk
 
PPTX
Getting Started with Splunk Enterprise
Splunk
 
PPTX
Getting Started with Splunk Enterprise
Shannon Cuthbertson
 
PPTX
Getting Started with Splunk Enterprise
Splunk
 
PDF
Splunk Discovery Day Milwaukee 9-14-17
Splunk
 
PDF
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
PPTX
Power of SPL
Tian Chen
 
PPTX
Getting started with Splunk Breakout Session
Splunk
 
PPTX
Getting Started with Splunk Enterprise
Splunk
 
PPTX
Getting Started with Splunk Enterprise Hands-On
Splunk
 
PPTX
SplunkLive! Zurich 2017 - The Power of SPL
Splunk
 
PPTX
Getting Started with Splunk Breakout Session
Splunk
 
PPTX
Power of SPL - Search Processing Language
Splunk
 
PPTX
Getting Started with Splunk Enterprises
Splunk
 
PPTX
Delivering New Visibility and Analytics for IT Operations
Splunk
 
PPTX
Splunk Forum Frankfurt - 15th Nov 2017 - .conf2017 Update
Splunk
 
PDF
SplunkLive! Amsterdam 2015 Breakout - Getting Started with Splunk
Splunk
 
PPTX
Power of SPL
Splunk
 
PPTX
Getting Started with Splunk Breakout Session
Splunk
 
Power of SPL Workshop
Splunk
 
Getting Started with Splunk Breakout Session
Splunk
 
Getting Started with Splunk Enterprise
Splunk
 
Getting Started with Splunk Enterprise
Shannon Cuthbertson
 
Getting Started with Splunk Enterprise
Splunk
 
Splunk Discovery Day Milwaukee 9-14-17
Splunk
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Splunk
 
Power of SPL
Tian Chen
 
Getting started with Splunk Breakout Session
Splunk
 
Getting Started with Splunk Enterprise
Splunk
 
Getting Started with Splunk Enterprise Hands-On
Splunk
 
SplunkLive! Zurich 2017 - The Power of SPL
Splunk
 
Getting Started with Splunk Breakout Session
Splunk
 
Power of SPL - Search Processing Language
Splunk
 
Getting Started with Splunk Enterprises
Splunk
 
Delivering New Visibility and Analytics for IT Operations
Splunk
 
Splunk Forum Frankfurt - 15th Nov 2017 - .conf2017 Update
Splunk
 
SplunkLive! Amsterdam 2015 Breakout - Getting Started with Splunk
Splunk
 
Power of SPL
Splunk
 
Getting Started with Splunk Breakout Session
Splunk
 
Ad

More from Splunk (20)

PDF
Splunk Leadership Forum Wien - 20.05.2025
Splunk
 
PDF
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
PDF
Building Resilience with Energy Management for the Public Sector
Splunk
 
PDF
IT-Lagebild: Observability for Resilience (SVA)
Splunk
 
PDF
Nach dem SOC-Aufbau ist vor der Automatisierung (OFD Baden-Württemberg)
Splunk
 
PDF
Monitoring einer Sicheren Inter-Netzwerk Architektur (SINA)
Splunk
 
PDF
Praktische Erfahrungen mit dem Attack Analyser (gematik)
Splunk
 
PDF
Cisco XDR & Splunk SIEM - stronger together (DATAGROUP Cyber Security)
Splunk
 
PDF
Security - Mit Sicherheit zum Erfolg (Telekom)
Splunk
 
PDF
One Cisco - Splunk Public Sector Summit Germany April 2025
Splunk
 
PDF
.conf Go 2023 - Data analysis as a routine
Splunk
 
PDF
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk
 
PDF
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
Splunk
 
PDF
.conf Go 2023 - Raiffeisen Bank International
Splunk
 
PDF
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
Splunk
 
PDF
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
Splunk
 
PDF
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
Splunk
 
PDF
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
Splunk
 
PDF
.conf go 2023 - De NOC a CSIRT (Cellnex)
Splunk
 
PDF
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
Splunk
 
Splunk Leadership Forum Wien - 20.05.2025
Splunk
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Building Resilience with Energy Management for the Public Sector
Splunk
 
IT-Lagebild: Observability for Resilience (SVA)
Splunk
 
Nach dem SOC-Aufbau ist vor der Automatisierung (OFD Baden-Württemberg)
Splunk
 
Monitoring einer Sicheren Inter-Netzwerk Architektur (SINA)
Splunk
 
Praktische Erfahrungen mit dem Attack Analyser (gematik)
Splunk
 
Cisco XDR & Splunk SIEM - stronger together (DATAGROUP Cyber Security)
Splunk
 
Security - Mit Sicherheit zum Erfolg (Telekom)
Splunk
 
One Cisco - Splunk Public Sector Summit Germany April 2025
Splunk
 
.conf Go 2023 - Data analysis as a routine
Splunk
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
Splunk
 
.conf Go 2023 - Raiffeisen Bank International
Splunk
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
Splunk
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
Splunk
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
Splunk
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
Splunk
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
Splunk
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
Splunk
 

Recently uploaded (20)

PPTX
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
PPTX
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
PPTX
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
PDF
Get More from Fiori Automation - What’s New, What Works, and What’s Next.pdf
Precisely
 
PDF
Structs to JSON: How Go Powers REST APIs
Emily Achieng
 
PDF
Oracle AI Vector Search- Getting Started and what's new in 2025- AIOUG Yatra ...
Sandesh Rao
 
PDF
Responsible AI and AI Ethics - By Sylvester Ebhonu
Sylvester Ebhonu
 
PDF
MASTERDECK GRAPHSUMMIT SYDNEY (Public).pdf
Neo4j
 
PDF
Make GenAI investments go further with the Dell AI Factory
Principled Technologies
 
PDF
How-Cloud-Computing-Impacts-Businesses-in-2025-and-Beyond.pdf
Artjoker Software Development Company
 
PDF
Tea4chat - another LLM Project by Kerem Atam
a0m0rajab1
 
PDF
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
PPTX
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
PDF
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 
PDF
Doc9.....................................
SofiaCollazos
 
PDF
OFFOFFBOX™ – A New Era for African Film | Startup Presentation
ambaicciwalkerbrian
 
PDF
A Strategic Analysis of the MVNO Wave in Emerging Markets.pdf
IPLOOK Networks
 
PPTX
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
PPTX
Applied-Statistics-Mastering-Data-Driven-Decisions.pptx
parmaryashparmaryash
 
PDF
REPORT: Heating appliances market in Poland 2024
SPIUG
 
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
Get More from Fiori Automation - What’s New, What Works, and What’s Next.pdf
Precisely
 
Structs to JSON: How Go Powers REST APIs
Emily Achieng
 
Oracle AI Vector Search- Getting Started and what's new in 2025- AIOUG Yatra ...
Sandesh Rao
 
Responsible AI and AI Ethics - By Sylvester Ebhonu
Sylvester Ebhonu
 
MASTERDECK GRAPHSUMMIT SYDNEY (Public).pdf
Neo4j
 
Make GenAI investments go further with the Dell AI Factory
Principled Technologies
 
How-Cloud-Computing-Impacts-Businesses-in-2025-and-Beyond.pdf
Artjoker Software Development Company
 
Tea4chat - another LLM Project by Kerem Atam
a0m0rajab1
 
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 
Doc9.....................................
SofiaCollazos
 
OFFOFFBOX™ – A New Era for African Film | Startup Presentation
ambaicciwalkerbrian
 
A Strategic Analysis of the MVNO Wave in Emerging Markets.pdf
IPLOOK Networks
 
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
Applied-Statistics-Mastering-Data-Driven-Decisions.pptx
parmaryashparmaryash
 
REPORT: Heating appliances market in Poland 2024
SPIUG
 

Power of SPL

  • 1. © 2017 SPLUNK INC.© 2017 SPLUNK INC. Power of Splunk Search Processing Language (SPL™) Mike Walker | Sales Engineer August 23, 2017 | Detroit, MI
  • 2. © 2017 SPLUNK INC. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2017 Splunk Inc. All rights reserved. Forward-Looking Statements
  • 3. © 2017 SPLUNK INC. Set Up Before You Can Play Download the following at splunk.com and splunkbase.com ▶ Splunk Enterprise: • https://www.splunk.com/download ▶ Power of SPL App: • https://splunkbase.splunk.com/app/3353/
  • 4. © 2017 SPLUNK INC. Set Up Before You Can Play ▶ Install Power of SPL App ▶ Now we’re ready to go!
  • 5. © 2017 SPLUNK INC. ▶ License expired (already had older version installed) • Close browser, empty cache, open browser. If that doesn’t work: • Stop Splunk. • Uninstall all Splunk versions • Windows Control Panel->Uninstall programs->Splunk • OS X. Finder->Applications->Right click Splunk, Move to trash • Reinstall • Start Splunk ▶ Can’t start Splunk • Windows, Search Control panel ->Services->Splunk start • Linux; cd <SPLUNK dir>/splunk/bin;./splunk start Common problems at this point
  • 6. © 2017 SPLUNK INC. 1. Installation and Setup (~15min) 2. Power of SPL Walkthrough (~1h 30min) • Overview & Anatomy of a Search • SPL Commands & Examples for Searching, charting, converging, mapping, transactions, anomalies, exploring data, custom 3. Custom Visualizations (~30min) 4. SPL and the Machine Learning Toolkit (~45min) Agenda
  • 7. © 2017 SPLUNK INC. SPL Overview
  • 8. © 2017 SPLUNK INC. ▶ Over 140 search commands ▶ Syntax was originally based upon the Unix pipeline and SQL and is optimized for time-series data ▶ The scope of SPL includes data searching, filtering, modification, manipulation, enrichment, insertion and deletion ▶ Includes machine learning such as anomaly detection SPL Overview Disk Intermediate results table Intermediate results table Final results table
  • 9. © 2017 SPLUNK INC. ▶ Flexibility and effectiveness on small and big data ▶ Late-binding schema ▶ More/better methods of correlation ▶ Not just analyze, but visualize Why Create a New Query Language? Data BIG Data
  • 10. © 2017 SPLUNK INC. search and filter | munge | report | cleanup | rename sum(KB) AS "Total KB" dc(clientip) AS "Unique Customers" | eval KB=bytes/1024 sourcetype=access* | stats sum(KB) dc(clientip) SPL Basic Structure
  • 11. © 2017 SPLUNK INC. SPL Examples
  • 12. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time (We’ll add some custom viz here!) ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 13. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 14. © 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
  • 15. © 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
  • 16. © 2017 SPLUNK INC. ▶ Keyword search: sourcetype=access* http ▶ Filter: sourcetype=access* http host=webserver-02 ▶ Combined: sourcetype=access* http host=webserver-02 (503 OR 504) Search and Filter Examples
  • 17. © 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
  • 18. © 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
  • 19. © 2017 SPLUNK INC. ▶ Calculation: sourcetype=access* |eval KB=bytes/1024 ▶ Evaluation: sourcetype=access* | eval http_response = if(status != 200, ”Error", ”OK”) ▶ Concatenation: sourcetype=access* | eval connection = device.” - ".clientip Eval – Modify or Create New Fields and Values Examples
  • 20. © 2017 SPLUNK INC. Eval – Just Getting Started! Splunk Search Quick Reference Guide
  • 21. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 22. © 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
  • 23. © 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
  • 24. © 2017 SPLUNK INC. ▶ Calculate stats and rename Index=power_of_spl | stats avg(bytes) AS “Avg Bytes” ▶ Multiple statistics index=power_of_spl | stats avg(bytes) AS bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) ▶ By another field index=power_of_spl | stats avg(bytes) AS avg_bytes sparkline(avg(bytes)) AS Bytes_Trend min(bytes) max(bytes) by clientip | sort - avg_bytes Stats – Calculate Statistics Based on Field Values Examples
  • 25. © 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
  • 26. © 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
  • 27. © 2017 SPLUNK INC. ▶ Visualize stats over time index=power_of_spl | timechart avg(bytes) ▶ Add a trendline index=power_of_spl | timechart avg(bytes) as bytes | trendline sma5(bytes) ▶ Add a prediction overlay index=power_of_spl | timechart avg(bytes) as bytes | predict future_timespan=5 bytes Timechart – Visualize Statistics Over Time Examples
  • 28. © 2017 SPLUNK INC. ▶ Cumulative/Running Totals index=power_of_spl | reverse | streamstats sum(bytes) AS sum_bytes | timechart latest(sum_bytes) as "Total Bytes" ▶ Summary Statistics index=power_of_spl | eventstats avg(bytes) AS overall_avg_bytes | stats avg(bytes) as clientip_avg_bytes by clientip overall_avg_bytes Streamstats – Cumulative/Running Totals Statistics Examples
  • 29. © 2017 SPLUNK INC. ▶ Cumulative/Running Totals index=power_of_spl | reverse | streamstats sum(bytes) AS sum_bytes | timechart latest(sum_bytes) as "Total Bytes" ▶ Summary Statistics index=power_of_spl | eventstats avg(bytes) AS overall_avg_bytes | stats avg(bytes) as clientip_avg_bytes by clientip overall_avg_bytes Streamstats – Cumulative/Running Totals Statistics Examples
  • 30. © 2017 SPLUNK INC. Stats/Timechart – But Wait, There’s More! Splunk Search Quick Reference Guide
  • 31. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 32. © 2017 SPLUNK INC. Converging Data Sources Index Untapped Data: Any Source, Type, Volume Ask Any Question Application Delivery Security, Compliance and Fraud IT Operations Business Analytics Industrial Data and the Internet of Things On-Premises Private Cloud Public Cloud Storage Online Shopping Cart Telecoms Desktops Security Web Services Networks Containers Web Clickstreams RFID Smartphones and Devices Servers Messaging GPS Location Packaged Applications Custom Applications Online Services DatabasesCall Detail Records Energy Meters Firewall Intrusion Prevention
  • 33. © 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes lookup – Converging Data Sources Examples
  • 34. © 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes Converging Data Sources Examples
  • 35. © 2017 SPLUNK INC. ▶ Enrich data with lookups index=power_of_spl status!=200 | lookup customer_info uid | stats count by customer_value ▶ Search Inception! index=power_of_spl [ search index=power_of_spl | stats sum(bytes) as total_bytes by clientip | sort - total_bytes | head 1 | return clientip ] | stats count by clientip status uri | sort - count ▶ Append multiple searches index=power_of_spl | timechart span=15s avg(bytes) as avg_bytes | appendcols [ search index=power_of_spl | stats stdev(bytes) as stdev_bytes] | eval 2stdv_upper = avg_bytes + stdev_bytes*2 | filldown 2stdv_upper | eval 2stdv_lower = avg_bytes - stdev_bytes*2 | filldown 2stdv_lower | eval 2stdv_lower = if('2stdv_lower’ <0,0,'2stdv_lower') | fields - stdev_bytes appendcols – Converging Data Sources Examples
  • 36. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time (Let’s add some viz!) ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 37. © 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId iplocation Geographic Data Examples
  • 38. © 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId geostats – Geographic Data Examples
  • 39. © 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId geom – Geographic Data Examples
  • 40. © 2017 SPLUNK INC. ▶ Assign Lat/Lon to IP addresses … | iplocation clientip ▶ Visualize statistics geographically … | geostats sum(price) by product ▶ Use custom choropleths … | geom <featureCollection> <featureId> ▶ Track object movements … | table _time latitude longitude vehicleId table – Geographic Data Examples
  • 41. © 2017 SPLUNK INC. Custom Visualizations
  • 42. © 2017 SPLUNK INC. ▶ Native charts and maps • Bar / Line / Area charts • Bubble / Scatter plots • Gauges • Maps • Single Value Displays • Tables ▶ Generalized to fit use cases across many different areas ▶ Can be customized to some extent to cover specific use cases Native Visualizations In Splunk 42
  • 43. © 2017 SPLUNK INC. ▶ Many use cases require a more specific visualization ▶ Specific custom appearance ▶ Represent data where native visualizations are not suitable • You can Splunk everything! • We won’t be able to predict every possible use case • Still uses SPL to drive visualizations Custom Visualizations FTW!
  • 44. © 2017 SPLUNK INC. ▶ Platform extensibility framework and API ▶ Targeted at internal and external developers with web development / JS skills and basic knowledge of the Splunk platform ▶ Developers can make use of any third party libraries (d3.js, three.js, highcharts.js, etc…) that run in the browser* * with minor adjustments, and if third party license permits such use Custom Visualizations
  • 45. © 2017 SPLUNK INC. Custom Visualizations For Admins In-product • Packaged as an app! • Installed like any other app • Users can search for visualizations on Splunkbase and directly in the product Installation
  • 46. © 2017 SPLUNK INC. ▶ Choose from potentially dozens of installed visualizations! ▶ Appears as a first-class citizen alongside native visualizations • Looks and works just like packaged native visualizations ▶ Customize functionality and appearance of the visualization without touching any code, straight from the UI ▶ SPL Example provided as you hover over each visualization option. Custom Visualizations How-to
  • 47. © 2017 SPLUNK INC. New Splunk Visualizations Treemap Sankey Diagram Punchcard Calendar Heat Map Parallel Coordinates Bullet GraphLocation Tracker Horseshoe Meter Machine Learning Charts Timeline Horizon Chart Multiple use cases across IT, security, IoT, and business analytics
  • 48. © 2017 SPLUNK INC. Box Plot 3D scatter plot New Partner/Community Visualizations Wordcloud Donut Chart Heat Map
  • 49. © 2017 SPLUNK INC. New Partner/Community Visualizations Geo Heatmap Custom Cluster Map Clustered Single Value Map Missile Map
  • 50. © 2017 SPLUNK INC. Custom Visualizations – Demo!
  • 51. © 2017 SPLUNK INC. Demo Screenshot #1
  • 52. © 2017 SPLUNK INC. Demo Screenshot #2
  • 53. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 54. © 2017 SPLUNK INC. ▶ Find anomalies | inputlookup car_data.csv | anomalydetection ▶ Summarize anomalies | inputlookup car_data.csv | anomalydetection action=summary ▶ Use IQR and remove outliers | inputlookup car_data.csv | anomalydetection method=iqr action=remove Anomaly Detection – Find anomalies in your data Examples
  • 55. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 56. © 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
  • 57. © 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
  • 58. © 2017 SPLUNK INC. ▶ Group by session ID sourcetype=access* | transaction JSESSIONID ▶ Calculate session durations sourcetype=access* | transaction JSESSIONID | stats min(duration) max(duration) avg(duration) ▶ Stats is better sourcetype=access* | stats min(_time) AS earliest max(_time) AS latest by JSESSIONID | eval duration=latest-earliest | stats min(duration) max(duration) avg(duration) Transaction – Group Related Events Spanning Time Examples
  • 59. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 60. © 2017 SPLUNK INC. Data Exploration | analyzefields | anomalies | arules | associate | cluster | contingency | correlate | fieldsummary
  • 61. © 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status Cluster – Exploring Your Data Examples
  • 62. © 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status fieldsummary – Exploring Your Data Examples
  • 63. © 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status correlate – Exploring Your Data Examples
  • 64. © 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status contingency – Exploring Your Data Examples
  • 65. © 2017 SPLUNK INC. ▶ Find most/least common events * | cluster showcount=t t=.1 | table _raw cluster_count ▶ Display Summary of Fields sourcetype=access_combined | fields – date* source* time* | fieldsummary maxvals=5 ▶ Show patterns of co-occurring fields sourcetype=access_combined | fields – date* source* time* | correlate ▶ View field relationships sourcetype=access_combined | contingency uri status ▶ Find predictors of fields sourcetype=access_combined | analyzefields classfield=status analyzefields – Exploring Your Data Examples
  • 66. © 2017 SPLUNK INC. ▶ Predict Numeric Fields ▶ Predict Categorical Fields ▶ Detect Numerical Outliers ▶ Detect Categorical Outliers ▶ Forecast Time Series ▶ Cluster Events Machine Learning Toolkit and Showcase Examples
  • 67. © 2017 SPLUNK INC. ▶ Find the needle in the haystack ▶ Charting statistics and predicting values ▶ Enriching and converging data sources ▶ Map geographic data in real time ▶ Identifying anomalies ▶ Transactions ▶ Data exploration & finding relationships between fields ▶ Custom commands SPL Examples and Recipes
  • 68. © 2017 SPLUNK INC. ▶ What is a Custom Command? • “| haversine origin="47.62,-122.34" outputField=dist lat lon” ▶ Why do we use Custom Commands? • Run other/external algorithms on your Splunk data • Save time munging data (see Timewrap!) • Because you can! ▶ Create your own or download as Apps • Haversine (Distance between two GPS coords) • Timewrap (Enhanced Time overlay) • Levenshtein (Fuzzy string compare) • Base64 (Encode/Decode) Custom Commands
  • 69. © 2017 SPLUNK INC. ▶ Download and install App Haversine ▶ Read documentation then use in SPL! sourcetype=access* | iplocation clientip | search City=A* | haversine origin="47.62,-122.34" units=mi outputField=dist lat lon | table clientip, City, dist, lat, lon Custom Commands – Haversine Examples
  • 70. © 2017 SPLUNK INC. ▶ Download and install App Haversine ▶ Read documentation then use in SPL! sourcetype=access* | iplocation clientip | search City=A* | haversine origin="47.62,-122.34" units=mi outputField=dist lat lon | table clientip, City, dist, lat, lon Custom Commands – Haversine Examples
  • 71. © 2017 SPLUNK INC. SPL & The Machine Learning Toolkit
  • 72. © 2017 SPLUNK INC. ▶ Predict Numeric Fields ▶ Predict Categorical Fields ▶ Detect Numerical Outliers ▶ Detect Categorical Outliers ▶ Forecast Time Series ▶ Cluster Events Machine Learning Toolkit and Showcase Examples
  • 73. © 2017 SPLUNK INC. Machine Learning with the Splunk Platform Visualize Share Clean Transform Operationalize Monitor Alert Build Model Search Explore Collect Data Test, Improve Models Ecosystem MLTK Choose Algorithm Ecosystem Splunk Splunk Splunk Splunk MLTK Splunk MLTK Splunk MLTK Splunk Ecosystem Splunk Real-time Data Science Pipeline Ecosystem MLTK Splunk Splunk’s App Ecosystem contains 1000’s of free add-ons for getting data in, applying structure and visualizing your data giving you faster time to value. The Machine Learning Toolkit delivers new SPL commands, custom visualizations, assistants, and examples to explore a variety of ml concepts. Splunk Enterprise is the mission-critical platform for indexing, searching, analyzing, alerting and visualizing machine data. Packaged: UBA, ITSI
  • 74. © 2017 SPLUNK INC. ML SPL Visualize Share Correlate Clean Munge Operationalize Monitor Alert Build Model Search Explore Universal Indexing Test, Improve Models Ecosystem MLTK Choose Algorithm Ecosystem Splunk Splunk Splunk Splunk MLTK Splunk MLTK Splunk MLTK Splunk Ecosystem Splunk fit sample apply listmodels deletemodel summary eval rex stats eventstats streamstats table … timechart chart stats geostats geom sendalert sendemail table … MLTK Library predict (cmd) anomalydetection (cmd) analyzefields anomalies arules associate cluster contingency correlate fieldsummary …
  • 75. © 2017 SPLUNK INC. MLTK Commands The Machine Learning Toolkit contains several custom search commands that implement classic machine learning and statistical learning tasks: • fit: Fit and apply a machine learning model to search results. • apply: Apply a machine learning model that was learned using the fit command. • summary: Return a summary of a machine learning model that was learned using the fit command. • listmodels: Return a list of machine learning models that were learned using the fit command. • deletemodel: Delete a machine learning model that was learned using the fit command. • sample: Randomly sample or partition events.
  • 76. © 2017 SPLUNK INC. ML-SPL Demo
  • 77. © 2017 SPLUNK INC. Set Up Before You Can Play Download the following at splunkbase.com ▶ Machine Learning Toolkit: • https://splunkbase.splunk.com/app/2890/ ▶ Python for Scientific Computing: • https://splunkbase.splunk.com/app/2881/ *Note – For the Python for Scientific Computing App you need to download the platform specific version – Mac, Linux, Windows
  • 78. © 2017 SPLUNK INC. | fit
  • 79. © 2017 SPLUNK INC. | sample
  • 80. © 2017 SPLUNK INC. | apply
  • 81. © 2017 SPLUNK INC. | listmodels
  • 82. © 2017 SPLUNK INC. | summary
  • 83. © 2017 SPLUNK INC. | deletemodel
  • 84. © 2017 SPLUNK INC. ▶Additional information can be found in: • Power of SPL App! • Docs - Search Manual • Docs - MLTK Search Commands • MLTK Quick Reference Guide • Blogs • Answers • Exploring Splunk For More Information
  • 85. © 2017 SPLUNK INC. • SPL Examples App Other Useful Apps to download! • Splunk 6.x Dashboard Examples • Splunk 6.5 Overview App
  • 86. © 2017 SPLUNK INC. ▶ Benefit of using your own laptop is a ‘Take it with you after’ approach – and bang on it without messing up production Splunk • Promote Personalized 50GB 6 month Dev/Test license to use for Workshop (vs. 30 day download trial) • Encourages long term playing with Splunk and continuously testing out new data sources How about an extra 50GB?
  • 87. © 2017 SPLUNK INC. Q & A
  • 88. © 2017 SPLUNK INC.© 2017 SPLUNK INC. Thank You