SlideShare a Scribd company logo
Yassine LASRI - Product Manager
Septembre 26, 2019
Introducing Elastic Stack for
Telecom Analytics
2
Yassine LASRI
Product Manager - Analytics
Synapticiel (www.synapticiel.co)
Telecoms Engineer
+12, Senior Architect BSS/OSS, Analytics
Revenue Assurance & Fraud Management
@ylasri
@lasri
3
Housekeeping & Logistics
Attendees are automatically muted when joining Zoom
Ask questions for us in the Zoom chat during the webinar
Slides & Recording will be available after the webinar
Q/A will be at the end of the webinar
4
Agenda
1) Telecom Data Challenges.
2) Quick Elastic Stack Overview.
3) High Level Architecture Example.
4) Demo
5) Q/A
5
TelecomData Challenges
Network Operations
Subscribers Behavior
Complex & Diverse
Events Data Sources
Valuable
Business insights
Major Challenges
Network Optimizing
QoS & SLA, CEM
Fraud (Assurance)
Smart Marketing
Product Innovation
6
NoSQL is better at
“scaling out” than RDBMS
Issue #2 : Capex & Opex Cost
Example : Oracle's Enterprise Edition Plus Business
Intelligence Suite lists at $221,2500 per processor How do you build
Distributed
H. Scalable,
Search/Analytics
platform ?
Issue #1 : RDBMS fails to support Big Data
Scaling "out" with RDBMS is not possible
Open Source NoSQL
Elastic Stack Overview
7
8
ELASTICSEARCH
High Level Architecture
9
Anatomy of Logstash Pipeline
Multiples sources (S&U)
Network Logs
Mediation Files
Queues
Beats
RDBMS
….
Multiples Transformations
Normalization
Enrichment
Corrélations
Aggregation
Pseudonymization
….
Multiples Outputs
Files Distribution
Elasticsearch Ingest
….
10
Elastic Stack {Demo}
Demo Use Case/Context :
Generated full 24H prepaid voice charging events (Ercicsson CCN node)
Simulate medium Operator
Stages:
High Level example of logstash pipeline
Kibana data discovery, visualization & dashboarding
Elasticsearch data frames & entity-centric index (Starting version 7.3.0)
Yassine LASRI - Product Manager
Septembre 26, 2019
Webinar : Introducing Elastic
Stack for Telecom Analytics
(Q&A Session)
12
Encrypted communications (inside/outside cluster)
Built-in authentication realms & IP-based authorization
Role-based access control (Index, Document & Field level access)
Security is free, starting in
versions 6.8.0 and 7.1.0
Role-based access control (RBAC)
Access can be restricted to resources :
Indices
Aliases
Documents
Fields
Users
Cluster
13
14
X-Pack Open Code
15
Elastic Stack Features
16
Elasticsearch Scaling
16
17
Elasticsearch Resiliency/HA
PUT /cortex_ocs_data
{
"settings" : {
"index.default_pipeline": "cortex_data_ingest",
"index" : {
"number_of_shards" : 5,
"number_of_replicas" : 1,
"refresh_interval" : "10s"
}
},
"mappings": {
"properties": {
"@timestamp": {"type": "date"},
"event_id": {"type": "keyword"},
"event_type": {"type": "keyword"},
"caller_party_number": {"type": "keyword"},
"called_party_number": {"type": "keyword"},
"duration": {"type": "long"}
}
}
}
18
Elasticsearch CCR/DR
19
Elasticsearch CCR/DR
20
Elasticsearch CCR/DR
21
Elasticsearch CCR/DR
22
Analytics for The Network
Dominate Telco Deployments
In order to understand where vendors are gaining traction with
products which included Advanced analytics, AI and Automation
(A3), it is useful to look beyond the vendor PowerPoint ☺ - at
published case studies and press releases.
Main Findings
45% of the press releases/case studies related to the network
36% specifically to network management and assurance
The relative maturity of using ML in campaign management and the
reduction in hype around consumer chatbots meant that there was less
case study materials seen in this 2018/19 analysis period than might be
expected.
Increased activity was seen in field service workforce management, the
contact center and digital channels.
Source : https://charlottepatrick.uk/most-popular-a3-projects-in-2018-19/
23
Beats (Logs & Security Analytics)
Audit data Auditbeat
Log files Filebeat
Cloud data Functionbeat
Availability Heartbeat
Systemd Linux journals Journalbeat
Metrics Metricbeat
Network traffic Packetbeat
Windows event logs Winlogbeat
24
Logstash Input
25
Logstash Filter
26
Logstash Output
27
Logstash Pipeline

More Related Content

PDF
Empowering agencies using Elastic as a Service inside Government
Elasticsearch
 
PPTX
Elastic community Abidjan #225 meetup 08 May 2021
Yassine, LASRI
 
PDF
Monitoring and Securing a Geo-Dispersed Data Center at Hill AFB
Elasticsearch
 
PPTX
IoT meets AI in the Clouds
Dr. Mirko Kämpf
 
PDF
Achieving cyber mission assurance with near real-time impact
Elasticsearch
 
PDF
The Elastic Evolution of CenturyLink’s Network Management System
Elasticsearch
 
PDF
Countering Threats with the Elastic Stack at CERDEC/ARL
Elasticsearch
 
PDF
Building Resilient and Scalable Data Pipelines by Decoupling Compute and Storage
Databricks
 
Empowering agencies using Elastic as a Service inside Government
Elasticsearch
 
Elastic community Abidjan #225 meetup 08 May 2021
Yassine, LASRI
 
Monitoring and Securing a Geo-Dispersed Data Center at Hill AFB
Elasticsearch
 
IoT meets AI in the Clouds
Dr. Mirko Kämpf
 
Achieving cyber mission assurance with near real-time impact
Elasticsearch
 
The Elastic Evolution of CenturyLink’s Network Management System
Elasticsearch
 
Countering Threats with the Elastic Stack at CERDEC/ARL
Elasticsearch
 
Building Resilient and Scalable Data Pipelines by Decoupling Compute and Storage
Databricks
 

What's hot (20)

PDF
Loading Data into Azure SQL DW (Synapse Analytics)
Antonios Chatzipavlis
 
PDF
Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale
Elasticsearch
 
PDF
Search for all with Elastic Enterprise Search
Elasticsearch
 
PDF
Migrate and Modernize Hadoop-Based Security Policies for Databricks
Databricks
 
PPTX
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
DataStax
 
PDF
Keynote: Looping through data, insight, and action
Elasticsearch
 
PDF
Elastic at KPN
Elasticsearch
 
PDF
ECS: Delivering Better Cyber Intelligence and Compliance
Elasticsearch
 
PDF
Meetup introduction to elastic stack - search at scale - skilledfield slide...
Mouaz Alnouri
 
PDF
Infrastructure monitoring made easy, from ingest to insight
Elasticsearch
 
PPTX
Azure synapse analytics overview elasta cloud3
Richard Conway
 
PDF
CSG’s Journey with Elastic
Elasticsearch
 
PDF
Modernizing deployment in any environment with Elastic
Elasticsearch
 
PDF
Kubernetes Jakarta Meetup 010 - Service Mesh Observability with Kiali
Yusuf Hadiwinata Sutandar
 
PDF
How eStruxture Data Centers is Using ECE to Rapidly Scale Their Business
Elasticsearch
 
PPTX
Blockchain in Practice
Codit
 
PDF
Powering Postbank Group’s Data-driven Strategy
Elasticsearch
 
PPTX
CI/CD for a Data Platform
Codit
 
PDF
Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...
Data Con LA
 
PDF
Search for All with Elastic Enterprise Search
Elasticsearch
 
Loading Data into Azure SQL DW (Synapse Analytics)
Antonios Chatzipavlis
 
Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale
Elasticsearch
 
Search for all with Elastic Enterprise Search
Elasticsearch
 
Migrate and Modernize Hadoop-Based Security Policies for Databricks
Databricks
 
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
DataStax
 
Keynote: Looping through data, insight, and action
Elasticsearch
 
Elastic at KPN
Elasticsearch
 
ECS: Delivering Better Cyber Intelligence and Compliance
Elasticsearch
 
Meetup introduction to elastic stack - search at scale - skilledfield slide...
Mouaz Alnouri
 
Infrastructure monitoring made easy, from ingest to insight
Elasticsearch
 
Azure synapse analytics overview elasta cloud3
Richard Conway
 
CSG’s Journey with Elastic
Elasticsearch
 
Modernizing deployment in any environment with Elastic
Elasticsearch
 
Kubernetes Jakarta Meetup 010 - Service Mesh Observability with Kiali
Yusuf Hadiwinata Sutandar
 
How eStruxture Data Centers is Using ECE to Rapidly Scale Their Business
Elasticsearch
 
Blockchain in Practice
Codit
 
Powering Postbank Group’s Data-driven Strategy
Elasticsearch
 
CI/CD for a Data Platform
Codit
 
Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...
Data Con LA
 
Search for All with Elastic Enterprise Search
Elasticsearch
 
Ad

Similar to Webinar elastic stack {on telecom} english webinar part (1) (20)

PDF
What's new at Elastic: Update on major initiatives and releases
Elasticsearch
 
PDF
What's new at Elastic: Update on major initiatives and releases
Elasticsearch
 
PDF
Keynote: Elastic Observability evolution and vision
Elasticsearch
 
PDF
Elastic Stack keynote
Elasticsearch
 
PDF
Elastic Observability keynote
Elasticsearch
 
PDF
Combining Logs, Metrics, and Traces for Unified Observability
Elasticsearch
 
PPTX
ELK Solutions Enablement Session - 17th March'2020
Ashnikbiz
 
PPTX
Elasticsearch meetup final_2014_04
marc_harrison
 
PDF
Public sector keynote
Elasticsearch
 
PDF
Keynote : évolution et vision d'Elastic Observability
Elasticsearch
 
PDF
Capgemini: Observability within the Dutch government
Elasticsearch
 
PDF
Cloud-native application monitoring powered by Riverbed and Elasticsearch
Richard Juknavorian
 
PDF
Explore Elasticsearch and Why It’s Worth Using
Inexture Solutions
 
PDF
Realizing your AIOps goals with machine learning in Elastic
Elasticsearch
 
PDF
The Elastic clients: Recent developments
Elasticsearch
 
PDF
Keynote: Making search better, faster, easier
Elasticsearch
 
PDF
Elasticsearch: From development to production in 15 minutes
Elasticsearch
 
PDF
Observability at scale: Hear from the Elastic Cloud SRE team
Elasticsearch
 
PDF
Managing the Elastic Stack at Scale
Elasticsearch
 
PDF
[Virtual Meetup] Using Elasticsearch as a Time-Series Database in the Endpoin...
Anna Ossowski
 
What's new at Elastic: Update on major initiatives and releases
Elasticsearch
 
What's new at Elastic: Update on major initiatives and releases
Elasticsearch
 
Keynote: Elastic Observability evolution and vision
Elasticsearch
 
Elastic Stack keynote
Elasticsearch
 
Elastic Observability keynote
Elasticsearch
 
Combining Logs, Metrics, and Traces for Unified Observability
Elasticsearch
 
ELK Solutions Enablement Session - 17th March'2020
Ashnikbiz
 
Elasticsearch meetup final_2014_04
marc_harrison
 
Public sector keynote
Elasticsearch
 
Keynote : évolution et vision d'Elastic Observability
Elasticsearch
 
Capgemini: Observability within the Dutch government
Elasticsearch
 
Cloud-native application monitoring powered by Riverbed and Elasticsearch
Richard Juknavorian
 
Explore Elasticsearch and Why It’s Worth Using
Inexture Solutions
 
Realizing your AIOps goals with machine learning in Elastic
Elasticsearch
 
The Elastic clients: Recent developments
Elasticsearch
 
Keynote: Making search better, faster, easier
Elasticsearch
 
Elasticsearch: From development to production in 15 minutes
Elasticsearch
 
Observability at scale: Hear from the Elastic Cloud SRE team
Elasticsearch
 
Managing the Elastic Stack at Scale
Elasticsearch
 
[Virtual Meetup] Using Elasticsearch as a Time-Series Database in the Endpoin...
Anna Ossowski
 
Ad

Recently uploaded (20)

PDF
Advanced LangChain & RAG: Building a Financial AI Assistant with Real-Time Data
Soufiane Sejjari
 
PPTX
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
PPTX
Victory Precisions_Supplier Profile.pptx
victoryprecisions199
 
PPTX
Information Retrieval and Extraction - Module 7
premSankar19
 
DOCX
SAR - EEEfdfdsdasdsdasdasdasdasdasdasdasda.docx
Kanimozhi676285
 
PDF
Top 10 read articles In Managing Information Technology.pdf
IJMIT JOURNAL
 
PDF
FLEX-LNG-Company-Presentation-Nov-2017.pdf
jbloggzs
 
PPTX
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
PDF
Biodegradable Plastics: Innovations and Market Potential (www.kiu.ac.ug)
publication11
 
PPTX
MT Chapter 1.pptx- Magnetic particle testing
ABCAnyBodyCanRelax
 
PDF
top-5-use-cases-for-splunk-security-analytics.pdf
yaghutialireza
 
PPTX
easa module 3 funtamental electronics.pptx
tryanothert7
 
PPTX
Chapter_Seven_Construction_Reliability_Elective_III_Msc CM
SubashKumarBhattarai
 
PPTX
22PCOAM21 Data Quality Session 3 Data Quality.pptx
Guru Nanak Technical Institutions
 
PDF
Traditional Exams vs Continuous Assessment in Boarding Schools.pdf
The Asian School
 
PDF
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
PPTX
database slide on modern techniques for optimizing database queries.pptx
aky52024
 
PPTX
Civil Engineering Practices_BY Sh.JP Mishra 23.09.pptx
bineetmishra1990
 
PDF
Unit I Part II.pdf : Security Fundamentals
Dr. Madhuri Jawale
 
PDF
Chad Ayach - A Versatile Aerospace Professional
Chad Ayach
 
Advanced LangChain & RAG: Building a Financial AI Assistant with Real-Time Data
Soufiane Sejjari
 
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
Victory Precisions_Supplier Profile.pptx
victoryprecisions199
 
Information Retrieval and Extraction - Module 7
premSankar19
 
SAR - EEEfdfdsdasdsdasdasdasdasdasdasdasda.docx
Kanimozhi676285
 
Top 10 read articles In Managing Information Technology.pdf
IJMIT JOURNAL
 
FLEX-LNG-Company-Presentation-Nov-2017.pdf
jbloggzs
 
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
Biodegradable Plastics: Innovations and Market Potential (www.kiu.ac.ug)
publication11
 
MT Chapter 1.pptx- Magnetic particle testing
ABCAnyBodyCanRelax
 
top-5-use-cases-for-splunk-security-analytics.pdf
yaghutialireza
 
easa module 3 funtamental electronics.pptx
tryanothert7
 
Chapter_Seven_Construction_Reliability_Elective_III_Msc CM
SubashKumarBhattarai
 
22PCOAM21 Data Quality Session 3 Data Quality.pptx
Guru Nanak Technical Institutions
 
Traditional Exams vs Continuous Assessment in Boarding Schools.pdf
The Asian School
 
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
database slide on modern techniques for optimizing database queries.pptx
aky52024
 
Civil Engineering Practices_BY Sh.JP Mishra 23.09.pptx
bineetmishra1990
 
Unit I Part II.pdf : Security Fundamentals
Dr. Madhuri Jawale
 
Chad Ayach - A Versatile Aerospace Professional
Chad Ayach
 

Webinar elastic stack {on telecom} english webinar part (1)

  • 1. Yassine LASRI - Product Manager Septembre 26, 2019 Introducing Elastic Stack for Telecom Analytics
  • 2. 2 Yassine LASRI Product Manager - Analytics Synapticiel (www.synapticiel.co) Telecoms Engineer +12, Senior Architect BSS/OSS, Analytics Revenue Assurance & Fraud Management @ylasri @lasri
  • 3. 3 Housekeeping & Logistics Attendees are automatically muted when joining Zoom Ask questions for us in the Zoom chat during the webinar Slides & Recording will be available after the webinar Q/A will be at the end of the webinar
  • 4. 4 Agenda 1) Telecom Data Challenges. 2) Quick Elastic Stack Overview. 3) High Level Architecture Example. 4) Demo 5) Q/A
  • 5. 5 TelecomData Challenges Network Operations Subscribers Behavior Complex & Diverse Events Data Sources Valuable Business insights Major Challenges Network Optimizing QoS & SLA, CEM Fraud (Assurance) Smart Marketing Product Innovation
  • 6. 6 NoSQL is better at “scaling out” than RDBMS Issue #2 : Capex & Opex Cost Example : Oracle's Enterprise Edition Plus Business Intelligence Suite lists at $221,2500 per processor How do you build Distributed H. Scalable, Search/Analytics platform ? Issue #1 : RDBMS fails to support Big Data Scaling "out" with RDBMS is not possible Open Source NoSQL
  • 9. 9 Anatomy of Logstash Pipeline Multiples sources (S&U) Network Logs Mediation Files Queues Beats RDBMS …. Multiples Transformations Normalization Enrichment Corrélations Aggregation Pseudonymization …. Multiples Outputs Files Distribution Elasticsearch Ingest ….
  • 10. 10 Elastic Stack {Demo} Demo Use Case/Context : Generated full 24H prepaid voice charging events (Ercicsson CCN node) Simulate medium Operator Stages: High Level example of logstash pipeline Kibana data discovery, visualization & dashboarding Elasticsearch data frames & entity-centric index (Starting version 7.3.0)
  • 11. Yassine LASRI - Product Manager Septembre 26, 2019 Webinar : Introducing Elastic Stack for Telecom Analytics (Q&A Session)
  • 12. 12 Encrypted communications (inside/outside cluster) Built-in authentication realms & IP-based authorization Role-based access control (Index, Document & Field level access) Security is free, starting in versions 6.8.0 and 7.1.0
  • 13. Role-based access control (RBAC) Access can be restricted to resources : Indices Aliases Documents Fields Users Cluster 13
  • 17. 17 Elasticsearch Resiliency/HA PUT /cortex_ocs_data { "settings" : { "index.default_pipeline": "cortex_data_ingest", "index" : { "number_of_shards" : 5, "number_of_replicas" : 1, "refresh_interval" : "10s" } }, "mappings": { "properties": { "@timestamp": {"type": "date"}, "event_id": {"type": "keyword"}, "event_type": {"type": "keyword"}, "caller_party_number": {"type": "keyword"}, "called_party_number": {"type": "keyword"}, "duration": {"type": "long"} } } }
  • 22. 22 Analytics for The Network Dominate Telco Deployments In order to understand where vendors are gaining traction with products which included Advanced analytics, AI and Automation (A3), it is useful to look beyond the vendor PowerPoint ☺ - at published case studies and press releases. Main Findings 45% of the press releases/case studies related to the network 36% specifically to network management and assurance The relative maturity of using ML in campaign management and the reduction in hype around consumer chatbots meant that there was less case study materials seen in this 2018/19 analysis period than might be expected. Increased activity was seen in field service workforce management, the contact center and digital channels. Source : https://charlottepatrick.uk/most-popular-a3-projects-in-2018-19/
  • 23. 23 Beats (Logs & Security Analytics) Audit data Auditbeat Log files Filebeat Cloud data Functionbeat Availability Heartbeat Systemd Linux journals Journalbeat Metrics Metricbeat Network traffic Packetbeat Windows event logs Winlogbeat