Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Splunk for Developers
Scott Haskell
Principal Sales Engineer
Santa Clara
Grigori Melnik, Principal Product Manager – Splunk Developer Platform2
EMPOWERING DEVELOPERS
Gain
Application
Intelligence
Build Splunk
Apps
Integrate &
Extend
Splunk
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Splunk for Application
Development
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Build
Unit Testing
Code
Check-in Integration
Testing Deploy
Staging
Application Development Challenges
4
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Build
Unit Testing
Code
Check-in Integration
Testing Deploy
Staging
Lack of visibility across the product
development lifecycle
Pressure to increase velocity and
agility with DevOps
Limited insights into behavior and
performance from application logs
Application Development Challenges
5
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Quickly trace and identify errors anywhere
in the codebase with real-time search
and monitoring
Instrument your app logs to gain
application intelligence
Break down dev tool silos with real-time
insights from machine data
GAIN END-TO-END VISIBILITY
ACROSS THE DEV TOOL CHAIN
FIND AND FIX
ISSUES FASTER
PUSH BETTER CODE
USING ANALYTICS
Splunk for Application Lifecycle Intelligence
6
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Real-time dashboards show error rate
in production and impact of pushing
new builds
Developers can search and visualize
web logs, Java logs, eventlogs etc;
trace tx without complex
instrumentation
Alerts notify developers as soon as a
problem arises
Find and Fix Issues Faster
7
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Gain end-to-end visibility to make
informed decisions
Analytics insights without the need for
additional analytics tools
Ask questions while exploring and
collecting data
Push Better Code Using Analytics
8
Grigori Melnik, Principal Product Manager – Splunk Developer Platform9 9
CI / Build
Servers
Project and Issue
Tracking
Code
Repository
QA / Testing
Tools
End-To-End Visibility Across The Dev Tool Chain
Deployment Servers /
Automation
Grigori Melnik, Principal Product Manager – Splunk Developer Platform10
Grigori Melnik, Principal Product Manager – Splunk Developer Platform11
CI / Build
Server
Code
Review
Task
Tracking
What Data Can You Splunk?
Logs – Which code has already been reviewed for this release/sprint? Who has
completed the most code reviews? What code has NOT been reviewed?
Logs/API – Who is changing files? What kinds of files are being changed? What
branches are most active? What types of activities are occurring for a branch?
Version
Control
Logs/API – How many builds completed today/this week/this month? Which
check-in kicked off this build? Which tests ran against this failed build?
Logs – Which tasks are assigned to which developers? What progress is being
made to complete assigned tasks? What tasks remain for this release/sprint?
1
Grigori Melnik, Principal Product Manager – Splunk Developer Platform12
Key Benefits of Application Lifecycle Intelligence
Reduced Time
to Market
Shrink the time it takes
to get code through
dev/test to market
through faster issue
identification and
resolution
Increased
Agility
With real-time visibility
into processes like code
check-ins, builds and
tests to support
DevOps practices like
continuous integration
“Our devs are now able to
find and fix issues five to ten
times faster.”
“We can monitor all the
automation and handoffs it
takes to deploy 5-10 times
a day”
Application
Insights
Instrument customer
application logs to
capture critical
business events and
user behavior
“My code isn’t ready until it’s
Splunk-ready”
1
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Demo:
ADLC
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Touring the Splunk
Development Platform
Grigori Melnik, Principal Product Manager – Splunk Developer Platform15
Evolving the Splunk Platform
Collection
Indexing
Search Processing Language
Core Functions
Inputs, Apps, Other
Content
SDKs & plug-ins
Operational Intelligence Platform
Content
Core Engine
User and Developer Interfaces
Web Framework
REST API
Grigori Melnik, Principal Product Manager – Splunk Developer Platform16
Powerful Platform for Enterprise Developers
1
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript/CSS Extensions C#
JavaScript
Python
Ruby
Java
PHP
Data Models
Search Extensibility
Modular Inputs
SDKs
KV Store
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Log directly to
Splunk via TCP,
UDP, HTTP
Integrate search
results with other
applications using
custom
visualizations
Create and run
searches from
other applications
The REST API and SDKs
17
VisualizeSearch Manage
Add/Delete Users
Manage Inputs
Index
Grigori Melnik, Principal Product Manager – Splunk Developer Platform18
The Splunk REST API
Exposes an API method for every feature in the product
– Whatever you can do in the UI – you can do through the API
– Index, Search, Visualize, Manage
API is RESTful
– Endpoints are served by splunkd
– Requests are GET, POST, and DELETE HTTP methods
– Responses are Atom XML & JSON
– Versioning as of Splunk 5.0
– Search results can be output in CSV/JSON/XML
1
Grigori Melnik, Principal Product Manager – Splunk Developer Platform19
SDKs Overview
19
• Stay true to the semantics of the particular language
• E.g. Keep Python “pythonic”
• E.g. C#: Fully async , PCL, support for Rx
• Provide implementation that feels natural to the developer
• E.g. Project, build, IDE (where applicable) support
• Cover REST API endpoints based on use cases of language
• Namespaces
• owner: splunk username (defaults to current user)
• app: app context (defaults to default app)
• sharing: user | app | global | system
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
A Developer’s Smörgåsbord
 Data ingestion
– Input
 Scripted inputs
 Modular inputs
 HTTP Event Collector
 Custom (trained) source
types
 Custom sources
– Data ingestion pipeline
 Field extractions
 Field transformations
– Indexing
 Custom indexes
 Searching
– Search authoring
 Custom search commands
 Macros (basic, parametrized)
 Saved searches
– Data classification
 Event types
 Transactions
– Data enrichment
 Lookups
 KV store collections
 Workflow actions
– Data normalization
 Tags
 Aliases
– Data mining
 cluster & dedup
 anomalousvalue
 kmeans
 predict commands …
 Processing & reporting
– Search-time mapping
 Data models
– CIM extensions
– Custom UI/visualizations
 Pages, views & dashboards
 JS Extensions
 CSS Extensions
 Custom setup screens
– Scheduled processing
 Scheduled reports
– Alerting
 Scripted alerts
– Branding & navigation
 Custom app navigation &
branding
– Manageability
 Custom splunkweb
controllers
 Custom splunkd endpoints
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Building Splunk Apps
Splunk Developer Guidance


Splunk Reference Apps
Complete, working real-world Splunk solutions
built together with partners (Conducive; Auth0)
̶ 2 (pseudo-) production releases
̶ entire code & test repos on GitHub
̶ under Apache 2.0
Associated Guidance
I. Start-to-Finish Journey Documentary
II. Essentials
dev.splunk.com/goto/devguide
1. Started with a Questions BacklogArchitecture
– What does a typical Splunk application reference architecture look like?
– What common paradigms are applicable to Splunk app development?
– What are the typical deployment topologies? Why should I choose a specific one? What are the confounding factors
on the choice of my topology?
– How do I partition my Splunk solutions?
– What are the tradeoffs of various types of inputs?
– How do I architect my Splunk solution and deployment for a very large scale?
– How do I architect my Splunk solution for the cloud? What are specific considerations for deploying to AWS or Azure?
– What’s the landscape of Splunk extension points?
– How do I integrate data from Splunk into existing applications and systems?
– How do I plan and design a robust alerting and monitoring subsystem on top of Splunk?
– What should I consider for my sizing requirements?
– What are recommended configurations of Splunk deployment to meet my sizing requirements?
– Should I architect my solution to index my data in local data center (zone) or centrally?
– What are things we can automatically degrade so we can make sure our core experience is working?
– When something happens, how do I effectively propagate the info and react to it?
– How are other solutions on Splunk built? What were the challenges? How have they been addressed?
Packaging and Deployment
– How do I piece together various parts of a Splunk app (custom search commands, mod inputs etc.)?
– How do I package a Splunk solution with a single install that automatically rolls out all the necessary dependencies?
– How do I manage my Splunk solution versioning, backward and future compat?
– What's the best way to split up custom apps for deployment?
Development
– How should I set up my development environment to be productive with Splunk?
– What are different ways of how I develop my Splunk app ? Pros and cons of using specific SDK vs REST APIs?
Pros and cons of using SimpleXML vs Advanced XML vs Web Framework …
– How do I analyze a data source for a TA?
– What are the different ways of enriching the data in Splunk? What are their tradeoffs?
– When should I use event types and transactions for data classification?
– How do I extend Splunk to define a custom input capability?
– When should I use modular inputs vs scripted inputs vs..?
– What are streaming vs non-streaming outputs considerations?
– How do I deal with long-running scripts? Handling shutdown/restart of Splunk? Concurrency? State persistence etc.
– Why should I not use transactions?
– When should I use pivot vs tstats?
– Why should I use data models?
– When my data source touches on many data models, should I assume complete separation or heavy inheritance?
– How do I extend an existing data model?
– What does CIM offer and why should I build CIM-compliant apps?
– In the context of CIM, what are the tradeoffs of using my props.conf and transforms.conf and rewriting them on
indexing, completely discarding the vendor supplied field names? How do I reconcile the advantages of a clean
interface & normalisation, but at the cost of losing alignment with published vendor documentation, and a learning
curve for existing users?
– How do I manage my solution declarative configuration? How do I detect/troubleshoot bad config?
– How do I log and analyze data that is not event driven (certain web feeds, html parsing, image meta data)?
– Compare and contrast ad-hoc searching vs background searching
– How do I handle transient faults?
– How do I effectively manage credentials?
– What’s the effect of search head location on my app and the overall user experience?
– How do I develop an integrated mechanism to let me connect Splunk to my MOM (messaging middleware) and index
my messages?
– How do I handle the requirement that app configs must be different across different server types in a distributed
environment (e.g. apps on search heads shouldn't have inputs enabled)?
Quality/Compliance
– What quality gates should I consider? What kind of para-functional characteristics are important to consider?
– What heuristics do I use to bless/block a release?
– How do I test a data model?
– How do I prepare event generation when building/testing an app?
– What kind of perf testing should I do and how?
– How do I test UI?
– How do I security certify my solution?
– How do I design to satisfy my retention and compliance policies?
– How do I architect to design my availability requirements?
– How do I handle geographic disaster recovery / fault tolerance?
– How do I properly instrument my solution so that I know what’s happening?
Sustained Engineering
– How do I maintain/service/support Splunk apps?
– How do my customers handle updating their customized configs once new versions of my app come out?
Business
– Why should I build on Splunk?
– What kind of skill do I need my devs to have to build a Splunk solution?
– What is the community building? How are current devs creating unique experiences using Splunk – I typically want to
see some marketplace success
– Cost and pricing are very important to me as a entrepreneur developer. If I am coming in to build a tool that will be
commercialized I need to know that the cost structure of Splunk won’t cause my service to be economically
unprofitable.
What does a typical Splunk application architecture look like?
How should I set up my dev environment to be productive with
Splunk?
How do I integrate Splunk into existing systems?
How do I prepare my event generation when developing &
testing an app?
How do I package an app? deal with app versioning and updates?
2. Mined business requirements with partner
3. Formulated learning objectives
4. Reconciled 2 & 3 with our designs
…
 Data
 Search language
 Aggregating siloed metrics into
meaningful KPIs
 Data manipulation
 Data normalization
 Sub-searches
 Config-driven
 Persistence with KV store
 Macros
 Viz:
 Dynamic scaling
 Customizing in-the box viz
controls
 General search patterns
 Search optimizations
 Ux Prototyping
 Adapting 3rd party viz library
 Composite charts with interactions
 Dealing with high-volume data sets
 Troubleshooting perf issues
 Post-process or not-post-process –
deployment implications
 Automated UI testing (w.Selenium)
 Setting the stage
 Overall Splunk app structure
 UI technology selection:
Simple XML vs SplunkJS
 Modularity
 Dev & test env
 Dev workflow
 Modularity
 Data onboarding
 CIM compliance
 Tools
 Post-processing
 Integrating with 3rd party
component
 Unit testing (w.Mocha)
 Persisting state (per user)
 Data modeling
 Using lookups
 Building a baseline lookup table
 Windows of time/Custom time ranges
 Overlaying time data
 Using sub-searches to correlate data
 Troubleshooting searches
 Custom nav
 Ux activities permeating all dev
 Data mining:
 Exploration
 Preparation: filtering/deduping/
bucketing
 Using advanced statistics functions
 Threshold-based anomaly detection
 Evaluating goodness /accuracy
Plus non-functional topics:  App versioning
 Packaging Installation
 Security review
 Deployment
 Publishing to splunkbase
 App certification
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Demo:
Building solutions with
Splunk Reference App
Copyright © 2015 Splunk Inc.27
Splunk Reference App comes preinstalled in the Cloud
Sandbox - www.splunk.com/goto/cloud
Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Copyright © 2015 Splunk Inc.
Resources
Grigori Melnik, Principal Product Manager – Splunk Developer Platform29
Splunk Developer License
2
Grigori Melnik, Principal Product Manager – Splunk Developer Platform30
Where to go for more Info
• Tutorials, Code Samples, Getting Started, Downloads
– http://dev.splunk.com
• Splunk Developer Guidance
– http://dev.splunk.com/goto/devguide
• Splunk Base (Apps)
– https://splunkbase.splunk.com
• GitHub
– https://github.com/splunk
• Twitter
– https://twitter.com/splunkdev
• Blogs
– http://blogs.splunk.com/dev
30
Copyright © 2015 Splunk Inc.31
Takeaways
Application development intelligence
Platform, not just an engine
Open & extensible
On-prem and cloud
Developer Guidance : learn and reuse for the win!
Reach out to us (devinfo@splunk.com) and tell us about
your experience
Copyright © 2015 Splunk Inc.
• September 26-29, 2016
• The Disney Swan and Dolphin, Orlando
• 5000+ IT & Business Professionals
• 3 days of technical content
• 165+ sessions
• 3 days of Splunk University
• Sept 24-26, 2016
• Get Splunk Certified for FREE!
• Get CPE credits for CISSP, CAP, SSCP
• Save thousands on Splunk education!
• 80+ Customer Speakers
• 35+ Apps in Splunk Apps Showcase
• 75+ Technology Partners
• 1:1 networking: Ask The Experts and
• Security Experts, Birds of a Feather and Chalk Talks
• NEW hands-on labs!
• Expanded show floor, Dashboards Control Room &
Clinic, and MORE!
.conf2016: The 7th Annual
Splunk Worldwide Users’ Conference
33
We Want to Hear your Feedback!
After the Breakout Sessions conclude
Text Splunk to 20691
And be entered for a chance to win a $100 AMEX gift card!

More Related Content

PPTX
SplunkLive! - Splunk for IT Operations
PPTX
What's New in 6.3 + Data On-Boarding
PPTX
Getting Started with Splunk Enterprise
PPTX
Elevate your Splunk Deployment by Better Understanding your Value Breakfast S...
PPTX
How to Design, Build and Map IT and Business Services in Splunk
PPTX
Data Onboarding Breakout Session
PPTX
Splunk for Developers
PDF
Splunk @ Adobe
SplunkLive! - Splunk for IT Operations
What's New in 6.3 + Data On-Boarding
Getting Started with Splunk Enterprise
Elevate your Splunk Deployment by Better Understanding your Value Breakfast S...
How to Design, Build and Map IT and Business Services in Splunk
Data Onboarding Breakout Session
Splunk for Developers
Splunk @ Adobe

What's hot (20)

PPTX
Distributed Management Console Breakout Session
PPTX
Getting Started with Splunk Enterprise Hands-On
PDF
Herbalife Customer Presentation
PPTX
How to Design, Build and Map IT and Business Services in Splunk
PPTX
Power of Splunk Search Processing Language (SPL)
PPTX
Splunk and Cisco UCS Breakout Session
PPTX
Splunk Ninjas: New Features and Search Dojo
PPTX
Taking Splunk to the Next Level - Architecture
PPTX
Customer Presentation
PPTX
Getting Started with Splunk Enterprise Hands-On
PPTX
Devops Powered by Splunk
PPTX
Splunk for Developers
PPTX
SplunkLive! - Splunk for IT Operations
PPTX
SplunkLive! San Francisco Dec 2012 - Intuit
PPTX
Machine Learning and Analytics Breakout Session
PPTX
Improve the Impact of DevOps
PPTX
Splunk Enterprise 6.3 - Splunk Tech Day
PPTX
Machine Learning and Analytics Breakout Session
PPTX
Sl boston 05_12_15_ener_noc_final_public
PPTX
SplunkLive! London 2016 Splunk for Devops
Distributed Management Console Breakout Session
Getting Started with Splunk Enterprise Hands-On
Herbalife Customer Presentation
How to Design, Build and Map IT and Business Services in Splunk
Power of Splunk Search Processing Language (SPL)
Splunk and Cisco UCS Breakout Session
Splunk Ninjas: New Features and Search Dojo
Taking Splunk to the Next Level - Architecture
Customer Presentation
Getting Started with Splunk Enterprise Hands-On
Devops Powered by Splunk
Splunk for Developers
SplunkLive! - Splunk for IT Operations
SplunkLive! San Francisco Dec 2012 - Intuit
Machine Learning and Analytics Breakout Session
Improve the Impact of DevOps
Splunk Enterprise 6.3 - Splunk Tech Day
Machine Learning and Analytics Breakout Session
Sl boston 05_12_15_ener_noc_final_public
SplunkLive! London 2016 Splunk for Devops
Ad

Viewers also liked (20)

PDF
Konzept LSF Schnittstelle
PDF
Bamboo boards
PPTX
Certificados digitales
PDF
วิธีเลี้ยงสุนัขที่ถูกต้อง
PDF
Скидки и акции в супермаркетах Билла с 21 по 27 января 2016 г.
PDF
PPTX
«Interviews: good, bad, efficient. Tips for technical interviewers» by Atrem...
PDF
Np cofidis cierre ejercicio 2010
PPTX
Super control Suite Deloitte Italia
DOCX
Tabla de actividades tpack
PPTX
SplunkLive! Beginner Session
PDF
ежо 3 кв 2015
PDF
Promotional Products
PDF
Case study NUTH
PDF
MAP Compliance
PPTX
SplunkLive! Splunk for IT Operations
PPTX
SplunkLive! Customer Presentation - Dow Jones
PPTX
Splunk 5 Overview Analyst v1.0
PPTX
SplunkLive! Analytics with Splunk Enterprise - Part 1
Konzept LSF Schnittstelle
Bamboo boards
Certificados digitales
วิธีเลี้ยงสุนัขที่ถูกต้อง
Скидки и акции в супермаркетах Билла с 21 по 27 января 2016 г.
«Interviews: good, bad, efficient. Tips for technical interviewers» by Atrem...
Np cofidis cierre ejercicio 2010
Super control Suite Deloitte Italia
Tabla de actividades tpack
SplunkLive! Beginner Session
ежо 3 кв 2015
Promotional Products
Case study NUTH
MAP Compliance
SplunkLive! Splunk for IT Operations
SplunkLive! Customer Presentation - Dow Jones
Splunk 5 Overview Analyst v1.0
SplunkLive! Analytics with Splunk Enterprise - Part 1
Ad

Similar to Splunk for Developers (20)

PPTX
Splunk for Developers Breakout Session
PPTX
Splunk for Developers Breakout Session
PPTX
SplunkLive! Seattle - Splunk for Developers
PDF
SplunkLive Brisbane Splunk for Developers
PDF
SplunkLive Brisbane Splunk for Developers
PDF
SplunkLive Melbourne Splunk for Developers
PPTX
SplunkLive! Developer Session
PPTX
Innovate Better Through Machine data Analytics
PPTX
Splunk Developer Platform
PPTX
SplunkLive London 2014 Developer Presentation
PPTX
SplunkLive! Washington DC May 2013 - Developer Session
PDF
SplunkLive! London 2015 - DevOps Breakout
PDF
Splunk in Nordstrom: IT Operations
PPTX
Customer Presentation - Financial Services Organization
POTX
Using the Splunk Java SDK
PDF
SFBA Splunk Usergroup meeting March 13, 2024
PDF
Throw Your Future Self a Lifeline with a Little DevOps Data!
PPTX
SplunkLive! Developer Breakout
PPTX
Splunk at Sabre
PPTX
Splunk Different
Splunk for Developers Breakout Session
Splunk for Developers Breakout Session
SplunkLive! Seattle - Splunk for Developers
SplunkLive Brisbane Splunk for Developers
SplunkLive Brisbane Splunk for Developers
SplunkLive Melbourne Splunk for Developers
SplunkLive! Developer Session
Innovate Better Through Machine data Analytics
Splunk Developer Platform
SplunkLive London 2014 Developer Presentation
SplunkLive! Washington DC May 2013 - Developer Session
SplunkLive! London 2015 - DevOps Breakout
Splunk in Nordstrom: IT Operations
Customer Presentation - Financial Services Organization
Using the Splunk Java SDK
SFBA Splunk Usergroup meeting March 13, 2024
Throw Your Future Self a Lifeline with a Little DevOps Data!
SplunkLive! Developer Breakout
Splunk at Sabre
Splunk Different

More from Splunk (20)

PDF
Splunk Leadership Forum Wien - 20.05.2025
PDF
Splunk Security Update | Public Sector Summit Germany 2025
PDF
Building Resilience with Energy Management for the Public Sector
PDF
IT-Lagebild: Observability for Resilience (SVA)
PDF
Nach dem SOC-Aufbau ist vor der Automatisierung (OFD Baden-Württemberg)
PDF
Monitoring einer Sicheren Inter-Netzwerk Architektur (SINA)
PDF
Praktische Erfahrungen mit dem Attack Analyser (gematik)
PDF
Cisco XDR & Splunk SIEM - stronger together (DATAGROUP Cyber Security)
PDF
Security - Mit Sicherheit zum Erfolg (Telekom)
PDF
One Cisco - Splunk Public Sector Summit Germany April 2025
PDF
.conf Go 2023 - Data analysis as a routine
PDF
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
PDF
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
PDF
.conf Go 2023 - Raiffeisen Bank International
PDF
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
PDF
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
PDF
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
PDF
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
PDF
.conf go 2023 - De NOC a CSIRT (Cellnex)
PDF
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
Splunk Leadership Forum Wien - 20.05.2025
Splunk Security Update | Public Sector Summit Germany 2025
Building Resilience with Energy Management for the Public Sector
IT-Lagebild: Observability for Resilience (SVA)
Nach dem SOC-Aufbau ist vor der Automatisierung (OFD Baden-Württemberg)
Monitoring einer Sicheren Inter-Netzwerk Architektur (SINA)
Praktische Erfahrungen mit dem Attack Analyser (gematik)
Cisco XDR & Splunk SIEM - stronger together (DATAGROUP Cyber Security)
Security - Mit Sicherheit zum Erfolg (Telekom)
One Cisco - Splunk Public Sector Summit Germany April 2025
.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - De NOC a CSIRT (Cellnex)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)

Recently uploaded (20)

PDF
giants, standing on the shoulders of - by Daniel Stenberg
PDF
Co-training pseudo-labeling for text classification with support vector machi...
PDF
4 layer Arch & Reference Arch of IoT.pdf
PDF
Enhancing plagiarism detection using data pre-processing and machine learning...
PPTX
Custom Battery Pack Design Considerations for Performance and Safety
PPTX
Training Program for knowledge in solar cell and solar industry
PDF
Consumable AI The What, Why & How for Small Teams.pdf
PDF
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
PDF
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
PDF
Data Virtualization in Action: Scaling APIs and Apps with FME
PDF
sustainability-14-14877-v2.pddhzftheheeeee
PDF
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
DOCX
search engine optimization ppt fir known well about this
PDF
Produktkatalog für HOBO Datenlogger, Wetterstationen, Sensoren, Software und ...
PDF
Statistics on Ai - sourced from AIPRM.pdf
PDF
Auditboard EB SOX Playbook 2023 edition.
PDF
NewMind AI Weekly Chronicles – August ’25 Week IV
PDF
Transform-Your-Factory-with-AI-Driven-Quality-Engineering.pdf
PPTX
MuleSoft-Compete-Deck for midddleware integrations
PPT
Galois Field Theory of Risk: A Perspective, Protocol, and Mathematical Backgr...
giants, standing on the shoulders of - by Daniel Stenberg
Co-training pseudo-labeling for text classification with support vector machi...
4 layer Arch & Reference Arch of IoT.pdf
Enhancing plagiarism detection using data pre-processing and machine learning...
Custom Battery Pack Design Considerations for Performance and Safety
Training Program for knowledge in solar cell and solar industry
Consumable AI The What, Why & How for Small Teams.pdf
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
Data Virtualization in Action: Scaling APIs and Apps with FME
sustainability-14-14877-v2.pddhzftheheeeee
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
search engine optimization ppt fir known well about this
Produktkatalog für HOBO Datenlogger, Wetterstationen, Sensoren, Software und ...
Statistics on Ai - sourced from AIPRM.pdf
Auditboard EB SOX Playbook 2023 edition.
NewMind AI Weekly Chronicles – August ’25 Week IV
Transform-Your-Factory-with-AI-Driven-Quality-Engineering.pdf
MuleSoft-Compete-Deck for midddleware integrations
Galois Field Theory of Risk: A Perspective, Protocol, and Mathematical Backgr...

Splunk for Developers

  • 1. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Splunk for Developers Scott Haskell Principal Sales Engineer Santa Clara
  • 2. Grigori Melnik, Principal Product Manager – Splunk Developer Platform2 EMPOWERING DEVELOPERS Gain Application Intelligence Build Splunk Apps Integrate & Extend Splunk
  • 3. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Splunk for Application Development
  • 4. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Build Unit Testing Code Check-in Integration Testing Deploy Staging Application Development Challenges 4
  • 5. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Build Unit Testing Code Check-in Integration Testing Deploy Staging Lack of visibility across the product development lifecycle Pressure to increase velocity and agility with DevOps Limited insights into behavior and performance from application logs Application Development Challenges 5
  • 6. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Quickly trace and identify errors anywhere in the codebase with real-time search and monitoring Instrument your app logs to gain application intelligence Break down dev tool silos with real-time insights from machine data GAIN END-TO-END VISIBILITY ACROSS THE DEV TOOL CHAIN FIND AND FIX ISSUES FASTER PUSH BETTER CODE USING ANALYTICS Splunk for Application Lifecycle Intelligence 6
  • 7. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Real-time dashboards show error rate in production and impact of pushing new builds Developers can search and visualize web logs, Java logs, eventlogs etc; trace tx without complex instrumentation Alerts notify developers as soon as a problem arises Find and Fix Issues Faster 7
  • 8. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Gain end-to-end visibility to make informed decisions Analytics insights without the need for additional analytics tools Ask questions while exploring and collecting data Push Better Code Using Analytics 8
  • 9. Grigori Melnik, Principal Product Manager – Splunk Developer Platform9 9 CI / Build Servers Project and Issue Tracking Code Repository QA / Testing Tools End-To-End Visibility Across The Dev Tool Chain Deployment Servers / Automation
  • 10. Grigori Melnik, Principal Product Manager – Splunk Developer Platform10
  • 11. Grigori Melnik, Principal Product Manager – Splunk Developer Platform11 CI / Build Server Code Review Task Tracking What Data Can You Splunk? Logs – Which code has already been reviewed for this release/sprint? Who has completed the most code reviews? What code has NOT been reviewed? Logs/API – Who is changing files? What kinds of files are being changed? What branches are most active? What types of activities are occurring for a branch? Version Control Logs/API – How many builds completed today/this week/this month? Which check-in kicked off this build? Which tests ran against this failed build? Logs – Which tasks are assigned to which developers? What progress is being made to complete assigned tasks? What tasks remain for this release/sprint? 1
  • 12. Grigori Melnik, Principal Product Manager – Splunk Developer Platform12 Key Benefits of Application Lifecycle Intelligence Reduced Time to Market Shrink the time it takes to get code through dev/test to market through faster issue identification and resolution Increased Agility With real-time visibility into processes like code check-ins, builds and tests to support DevOps practices like continuous integration “Our devs are now able to find and fix issues five to ten times faster.” “We can monitor all the automation and handoffs it takes to deploy 5-10 times a day” Application Insights Instrument customer application logs to capture critical business events and user behavior “My code isn’t ready until it’s Splunk-ready” 1
  • 13. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Demo: ADLC
  • 14. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Touring the Splunk Development Platform
  • 15. Grigori Melnik, Principal Product Manager – Splunk Developer Platform15 Evolving the Splunk Platform Collection Indexing Search Processing Language Core Functions Inputs, Apps, Other Content SDKs & plug-ins Operational Intelligence Platform Content Core Engine User and Developer Interfaces Web Framework REST API
  • 16. Grigori Melnik, Principal Product Manager – Splunk Developer Platform16 Powerful Platform for Enterprise Developers 1 REST API Build Splunk Apps Extend and Integrate Splunk Simple XML JavaScript/CSS Extensions C# JavaScript Python Ruby Java PHP Data Models Search Extensibility Modular Inputs SDKs KV Store
  • 17. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Log directly to Splunk via TCP, UDP, HTTP Integrate search results with other applications using custom visualizations Create and run searches from other applications The REST API and SDKs 17 VisualizeSearch Manage Add/Delete Users Manage Inputs Index
  • 18. Grigori Melnik, Principal Product Manager – Splunk Developer Platform18 The Splunk REST API Exposes an API method for every feature in the product – Whatever you can do in the UI – you can do through the API – Index, Search, Visualize, Manage API is RESTful – Endpoints are served by splunkd – Requests are GET, POST, and DELETE HTTP methods – Responses are Atom XML & JSON – Versioning as of Splunk 5.0 – Search results can be output in CSV/JSON/XML 1
  • 19. Grigori Melnik, Principal Product Manager – Splunk Developer Platform19 SDKs Overview 19 • Stay true to the semantics of the particular language • E.g. Keep Python “pythonic” • E.g. C#: Fully async , PCL, support for Rx • Provide implementation that feels natural to the developer • E.g. Project, build, IDE (where applicable) support • Cover REST API endpoints based on use cases of language • Namespaces • owner: splunk username (defaults to current user) • app: app context (defaults to default app) • sharing: user | app | global | system
  • 20. Grigori Melnik, Principal Product Manager – Splunk Developer Platform A Developer’s Smörgåsbord  Data ingestion – Input  Scripted inputs  Modular inputs  HTTP Event Collector  Custom (trained) source types  Custom sources – Data ingestion pipeline  Field extractions  Field transformations – Indexing  Custom indexes  Searching – Search authoring  Custom search commands  Macros (basic, parametrized)  Saved searches – Data classification  Event types  Transactions – Data enrichment  Lookups  KV store collections  Workflow actions – Data normalization  Tags  Aliases – Data mining  cluster & dedup  anomalousvalue  kmeans  predict commands …  Processing & reporting – Search-time mapping  Data models – CIM extensions – Custom UI/visualizations  Pages, views & dashboards  JS Extensions  CSS Extensions  Custom setup screens – Scheduled processing  Scheduled reports – Alerting  Scripted alerts – Branding & navigation  Custom app navigation & branding – Manageability  Custom splunkweb controllers  Custom splunkd endpoints
  • 21. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Building Splunk Apps
  • 22. Splunk Developer Guidance   Splunk Reference Apps Complete, working real-world Splunk solutions built together with partners (Conducive; Auth0) ̶ 2 (pseudo-) production releases ̶ entire code & test repos on GitHub ̶ under Apache 2.0 Associated Guidance I. Start-to-Finish Journey Documentary II. Essentials dev.splunk.com/goto/devguide
  • 23. 1. Started with a Questions BacklogArchitecture – What does a typical Splunk application reference architecture look like? – What common paradigms are applicable to Splunk app development? – What are the typical deployment topologies? Why should I choose a specific one? What are the confounding factors on the choice of my topology? – How do I partition my Splunk solutions? – What are the tradeoffs of various types of inputs? – How do I architect my Splunk solution and deployment for a very large scale? – How do I architect my Splunk solution for the cloud? What are specific considerations for deploying to AWS or Azure? – What’s the landscape of Splunk extension points? – How do I integrate data from Splunk into existing applications and systems? – How do I plan and design a robust alerting and monitoring subsystem on top of Splunk? – What should I consider for my sizing requirements? – What are recommended configurations of Splunk deployment to meet my sizing requirements? – Should I architect my solution to index my data in local data center (zone) or centrally? – What are things we can automatically degrade so we can make sure our core experience is working? – When something happens, how do I effectively propagate the info and react to it? – How are other solutions on Splunk built? What were the challenges? How have they been addressed? Packaging and Deployment – How do I piece together various parts of a Splunk app (custom search commands, mod inputs etc.)? – How do I package a Splunk solution with a single install that automatically rolls out all the necessary dependencies? – How do I manage my Splunk solution versioning, backward and future compat? – What's the best way to split up custom apps for deployment? Development – How should I set up my development environment to be productive with Splunk? – What are different ways of how I develop my Splunk app ? Pros and cons of using specific SDK vs REST APIs? Pros and cons of using SimpleXML vs Advanced XML vs Web Framework … – How do I analyze a data source for a TA? – What are the different ways of enriching the data in Splunk? What are their tradeoffs? – When should I use event types and transactions for data classification? – How do I extend Splunk to define a custom input capability? – When should I use modular inputs vs scripted inputs vs..? – What are streaming vs non-streaming outputs considerations? – How do I deal with long-running scripts? Handling shutdown/restart of Splunk? Concurrency? State persistence etc. – Why should I not use transactions? – When should I use pivot vs tstats? – Why should I use data models? – When my data source touches on many data models, should I assume complete separation or heavy inheritance? – How do I extend an existing data model? – What does CIM offer and why should I build CIM-compliant apps? – In the context of CIM, what are the tradeoffs of using my props.conf and transforms.conf and rewriting them on indexing, completely discarding the vendor supplied field names? How do I reconcile the advantages of a clean interface & normalisation, but at the cost of losing alignment with published vendor documentation, and a learning curve for existing users? – How do I manage my solution declarative configuration? How do I detect/troubleshoot bad config? – How do I log and analyze data that is not event driven (certain web feeds, html parsing, image meta data)? – Compare and contrast ad-hoc searching vs background searching – How do I handle transient faults? – How do I effectively manage credentials? – What’s the effect of search head location on my app and the overall user experience? – How do I develop an integrated mechanism to let me connect Splunk to my MOM (messaging middleware) and index my messages? – How do I handle the requirement that app configs must be different across different server types in a distributed environment (e.g. apps on search heads shouldn't have inputs enabled)? Quality/Compliance – What quality gates should I consider? What kind of para-functional characteristics are important to consider? – What heuristics do I use to bless/block a release? – How do I test a data model? – How do I prepare event generation when building/testing an app? – What kind of perf testing should I do and how? – How do I test UI? – How do I security certify my solution? – How do I design to satisfy my retention and compliance policies? – How do I architect to design my availability requirements? – How do I handle geographic disaster recovery / fault tolerance? – How do I properly instrument my solution so that I know what’s happening? Sustained Engineering – How do I maintain/service/support Splunk apps? – How do my customers handle updating their customized configs once new versions of my app come out? Business – Why should I build on Splunk? – What kind of skill do I need my devs to have to build a Splunk solution? – What is the community building? How are current devs creating unique experiences using Splunk – I typically want to see some marketplace success – Cost and pricing are very important to me as a entrepreneur developer. If I am coming in to build a tool that will be commercialized I need to know that the cost structure of Splunk won’t cause my service to be economically unprofitable. What does a typical Splunk application architecture look like? How should I set up my dev environment to be productive with Splunk? How do I integrate Splunk into existing systems? How do I prepare my event generation when developing & testing an app? How do I package an app? deal with app versioning and updates?
  • 24. 2. Mined business requirements with partner 3. Formulated learning objectives 4. Reconciled 2 & 3 with our designs …
  • 25.  Data  Search language  Aggregating siloed metrics into meaningful KPIs  Data manipulation  Data normalization  Sub-searches  Config-driven  Persistence with KV store  Macros  Viz:  Dynamic scaling  Customizing in-the box viz controls  General search patterns  Search optimizations  Ux Prototyping  Adapting 3rd party viz library  Composite charts with interactions  Dealing with high-volume data sets  Troubleshooting perf issues  Post-process or not-post-process – deployment implications  Automated UI testing (w.Selenium)  Setting the stage  Overall Splunk app structure  UI technology selection: Simple XML vs SplunkJS  Modularity  Dev & test env  Dev workflow  Modularity  Data onboarding  CIM compliance  Tools  Post-processing  Integrating with 3rd party component  Unit testing (w.Mocha)  Persisting state (per user)  Data modeling  Using lookups  Building a baseline lookup table  Windows of time/Custom time ranges  Overlaying time data  Using sub-searches to correlate data  Troubleshooting searches  Custom nav  Ux activities permeating all dev  Data mining:  Exploration  Preparation: filtering/deduping/ bucketing  Using advanced statistics functions  Threshold-based anomaly detection  Evaluating goodness /accuracy Plus non-functional topics:  App versioning  Packaging Installation  Security review  Deployment  Publishing to splunkbase  App certification
  • 26. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Demo: Building solutions with Splunk Reference App
  • 27. Copyright © 2015 Splunk Inc.27 Splunk Reference App comes preinstalled in the Cloud Sandbox - www.splunk.com/goto/cloud
  • 28. Grigori Melnik, Principal Product Manager – Splunk Developer Platform Copyright © 2015 Splunk Inc. Resources
  • 29. Grigori Melnik, Principal Product Manager – Splunk Developer Platform29 Splunk Developer License 2
  • 30. Grigori Melnik, Principal Product Manager – Splunk Developer Platform30 Where to go for more Info • Tutorials, Code Samples, Getting Started, Downloads – http://dev.splunk.com • Splunk Developer Guidance – http://dev.splunk.com/goto/devguide • Splunk Base (Apps) – https://splunkbase.splunk.com • GitHub – https://github.com/splunk • Twitter – https://twitter.com/splunkdev • Blogs – http://blogs.splunk.com/dev 30
  • 31. Copyright © 2015 Splunk Inc.31 Takeaways Application development intelligence Platform, not just an engine Open & extensible On-prem and cloud Developer Guidance : learn and reuse for the win! Reach out to us ([email protected]) and tell us about your experience
  • 32. Copyright © 2015 Splunk Inc. • September 26-29, 2016 • The Disney Swan and Dolphin, Orlando • 5000+ IT & Business Professionals • 3 days of technical content • 165+ sessions • 3 days of Splunk University • Sept 24-26, 2016 • Get Splunk Certified for FREE! • Get CPE credits for CISSP, CAP, SSCP • Save thousands on Splunk education! • 80+ Customer Speakers • 35+ Apps in Splunk Apps Showcase • 75+ Technology Partners • 1:1 networking: Ask The Experts and • Security Experts, Birds of a Feather and Chalk Talks • NEW hands-on labs! • Expanded show floor, Dashboards Control Room & Clinic, and MORE! .conf2016: The 7th Annual Splunk Worldwide Users’ Conference
  • 33. 33 We Want to Hear your Feedback! After the Breakout Sessions conclude Text Splunk to 20691 And be entered for a chance to win a $100 AMEX gift card!

Editor's Notes

  • #8: Order Flow, message queues, Garbage Collection, Java Heap identify errors by java class, thread alert actions - jira ticket, service now ticket, webhook
  • #9: PM’s love to look at feature usage; are new features being used? How do we allocate developer time to create/enhance features
  • #11: Roll Ubisoft Video http://www.splunk.com/en_us/resources/video.5rcTNqdDpGoBAtUNietLLxtORoC0QW7Y.html
  • #13: Compliance - Developers can’t login to production systems but Splunk allows them to troubleshoot
  • #30: 10GB of indexing/day Full Enterprise Features Free trial for 6 months
  • #32: Finds and traces bugs in real time so you can fix them faster Reduces time to market through faster issue identification and resolution Provides insights into user behavior and application usage Works with the applications, tools and systems that dev teams use every day for full lifecycle visibility
  • #33: We’re headed to the East Coast! 2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics! 165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE! 30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you! Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers. Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja! REGISTRATION OPENS IN MARCH 2016 – STAY TUNED FOR NEWS ON OUR BEST REGISTRATION RATES – COMING SOON!