Análisis del roadmap del Elastic Stack
0 likes286 views
The document outlines updates and roadmap features for the Elastic Stack, detailing various components such as data ingestion, management, and advanced machine learning analytics. Key topics include the configuration of Beats agents, index lifecycle management, data transformations, and enhancements in search performance. Additionally, it covers security features in Kibana and customizable alerting functionalities.
!["hits": {
"total": 123456789,
"hits": [ ... ]
}
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"hits": [ ... ]
}
Weak-AND](https://image.slidesharecdn.com/elasticontourstackdic2019-carlosperez-aradros-200227235629/85/Analisis-del-roadmap-del-Elastic-Stack-68-320.jpg)
![Templated Alerts
when [CPU] > [90%]
then alert
[alerts@me.com]](https://image.slidesharecdn.com/elasticontourstackdic2019-carlosperez-aradros-200227235629/85/Analisis-del-roadmap-del-Elastic-Stack-113-320.jpg)
Análisis del roadmap del Elastic Stack
- 1. Shay Banon | Founder & CEO Aaron Katz | CRO Janesh Moorjani | CFO Elastic Overview Elastic Stack Updates and Roadmap Carlos Pérez-Aradros Herce Senior Software Engineer
- 5. Ingest
- 7. Ingest
- 8. Ingest
- 9. Ingest
- 10. Ingest
- 12. Ingest Node: Enrichment Processor source.ip => is_known_botnet? geo.location => city/region/country
- 13. Adding Data
- 15. • What technology? (eg. Nginx) • What to monitor? (eg. logs, metrics, packets) • Where is it? (eg. paths to logfiles) Integrations Manager
- 18. • Beats config • Ingest node config • Index template • First index • Index alias • Index lifecycle management policy • Snapshot lifecycle management policy • Index patterns • Kibana dashboards • Canvas workpads • Machine learning jobs • Alerts Automatically Setup
- 19. • Beats config • Ingest node config • Index template • First index • Index alias • Index lifecycle management policy • Snapshot lifecycle management policy • Index patterns • Kibana dashboards • Canvas workpads • Machine learning jobs • Alerts Automatically Setup
- 21. • Single config language • Installs required Beats • Upgrades Beats • Upgrades itself Beats Agent
- 22. Fleet • Centralized Config Deployment • Centralized Beats Monitoring • Centralized Upgrade Management
- 23. Data Management
- 24. Frozen Indices
- 25. Heap File system cache Disk
- 26. Heap File system cache Disk
- 28. Hot Nodes 1 2 3 Cold NodesWarm Nodes
- 29. 1 2 3 1 2 3 Hot Nodes Cold NodesWarm Nodes
- 30. 1 2 3 1 2 3 Hot Nodes Cold NodesWarm Nodes
- 31. 231 2 3 Hot Nodes Cold NodesWarm Nodes 1
- 32. 1 Hot Nodes Cold NodesWarm Nodes 1 2 3
- 33. 1 Hot Nodes Cold NodesWarm Nodes 1 2 3
- 34. Hot Nodes Cold NodesWarm Nodes 1 2 3
- 35. Hot Nodes Cold NodesWarm Nodes 1 2 3
- 36. (coming soon to X-Pack)
- 38. • Periodic scheduled backups • Retention polices for automatic deletion Snapshot Management
- 41. Data Transforms
- 42. Clickstream Data
- 43. Page views per minute? Clickstream Data
- 44. 99th percentile latency? Clickstream Data
- 45. Most frequent URLs? Clickstream Data
- 46. How long was session 1? Clickstream Data
- 47. How long was session 1? Clickstream Data
- 48. Average session length? Clickstream Data
- 49. Average session length? Session Data
- 50. Average number of pages per session? Session Data
- 51. Most frequent exit page per session? Session Data
- 52. Session Data
- 53. How frequently do users visit the site? Session Data
- 54. How frequently do users visit the site? User Data
- 55. • Pivot • Pattern Matching Data Transformation
- 58. • Outlier detection • Supervised model training for regression & classification • Ingest Prediction Processor Advanced ML Analytics
- 59. Search
- 61. Query Before After Improvement Fuzzy 46 qps Phrase 4 qps Bool AND 9.3 qps Bool OR 3.3 qps Term 33 qps
- 62. Query Before After Improvement Fuzzy 46 qps 59 qps 28% Phrase 4 qps 7 qps 87% Bool AND 9.3 qps 23.5 qps 247% Bool OR 3.3 qps 9.8 qps 292% Term 33 qps 1,160 qps 3,700%
- 63. Magic WAND
- 64. "query" : "elasticsearch and lucene" max_score(and) == 1 max_score(lucene) == 5 max_score(elasticsearch) == 3 Weak-AND
- 65. Min top-10 score and (1) elasticsearch (3) lucene (5) <=1 ✓ ✓ ✓ > 1 and <= 4 ✗ ✓ ✓ > 4 and <= 9 ✗ ✗ ✓ > 9 ✗ ✗ ✗ Weak-AND
- 66. Weak-AND
- 67. Weak-AND "aggs": { ... } "track_total_hits": true
- 68. "hits": { "total": 123456789, "hits": [ ... ] } "hits": { "total": { "value": 10000, "relation": "gte" }, "hits": [ ... ] } Weak-AND
- 69. Search as you type
- 70. index_prefixes: qu, qui, quic, quick br, bro, brow, brown fo, fox, foxe, foxes
- 71. index_phrases: the_quick quick_brown brown_fox fox_jumped jumped_over over_the the_lazy lazy_dog
- 73. Advanced Scoring
- 74. rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 75. score = BM25(Text) + PageRank rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 76. score = BM25(Text) + Saturation(PageRank) rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 77. distance_feature: rank_feature: Advanced Scoring • Date • Geopoint • Numeric • Star Ratings • PageRank • Popularity
- 78. script_score: • Custom scoring, including vectors distance_feature: • Date • Geopoint • Numeric rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 79. Work with WAND script_score: • Custom scoring, including vectors distance_feature: rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity • Date • Geopoint • Numeric
- 80. Result Pinning
- 82. Geoshapes
- 83. • v2.3: 1 dim, for numbers and dates • v5.0: 2 dim, for geopoints • v5.2: 2 dim, for number & date ranges • v6.7: 7 dim, for geoshapes BKD Trees
- 84. BKD Geoshapes
- 85. • Accurate to 1cm, vs 50m • Index is 60% smaller • Indexing 60% faster • Queries 50% faster • Plus BKD GeoPoints 80% faster indexing BKD Geoshapes
- 87. Zen
- 94. New York London Tokyo
- 95. v5.6 v6.7 v7.x Three Major Versions
- 97. New York London Tokyo ldn_sales ldn_sales
- 98. New York London Tokyo tk_salesny_sales
- 99. New York London Tokyo tk_salesny_sales ldn_sales ldn_sales
- 101. Kibana
- 102. Security
- 106. Lens
- 108. New Platform
- 111. SIEM Stack Monitoring Machine Learning Observability
- 112. 112
- 113. Templated Alerts when [CPU] > [90%] then alert [[email protected]]
- 114. Chart-based Alerts
- 116. Guides
- 118. News Feed
- 120. Thank you