Getting Additional Value from Logs and APM Data with AppDynamics Unified Analytics - AppSphere16
8 likes1,277 views
The document discusses the integration of log analytics with application performance monitoring (APM) from AppDynamics, emphasizing their unified monitoring platform that offers end-to-end visibility. It highlights the importance of correlating APM data and log data to quickly resolve performance issues and improve operational efficiency. Key features and future enhancements of their analytics platform are also outlined, focusing on advanced search capabilities, log clustering, and centralized configuration management.
![What is common?
APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 50
*. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *...
IP address](https://image.slidesharecdn.com/t11518-haddadkaveh-iyer-161123000003/85/Getting-Additional-Value-from-Logs-and-APM-Data-with-AppDynamics-Unified-Analytics-AppSphere16-50-320.jpg)
![What is common?
APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 51
*. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *...
Timestamp](https://image.slidesharecdn.com/t11518-haddadkaveh-iyer-161123000003/85/Getting-Additional-Value-from-Logs-and-APM-Data-with-AppDynamics-Unified-Analytics-AppSphere16-51-320.jpg)
![What is common?
APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 52
*. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *...
URL](https://image.slidesharecdn.com/t11518-haddadkaveh-iyer-161123000003/85/Getting-Additional-Value-from-Logs-and-APM-Data-with-AppDynamics-Unified-Analytics-AppSphere16-52-320.jpg)
![We provide a summarized view...
APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 53
INFO ajp-nio-*-exec-* *-*-*/*:*:*
com.appdynamicspilot.action.CartAction ADD ...
log patterncount Pct.
129k
65k 5.2%
…
10.2% *. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *...](https://image.slidesharecdn.com/t11518-haddadkaveh-iyer-161123000003/85/Getting-Additional-Value-from-Logs-and-APM-Data-with-AppDynamics-Unified-Analytics-AppSphere16-53-320.jpg)
Getting Additional Value from Logs and APM Data with AppDynamics Unified Analytics - AppSphere16
- 1. Getting additional value from logs and APM data with AppDynamics Unified Analytics Nima Haddadkaveh Principal Product Manager | AppDynamics Arjun Iyer Senior Engineering Director | AppDynamics
- 2. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 2 Notice The information and materials included in this presentation (collectively, the “Materials”) are the proprietary information of AppDynamics, Inc. (“AppDynamics” or the “Company”). No part of the Materials may be reproduced, distributed, communicated or displayed in any form or by any means, or used to make any derivative work, without prior written permission from AppDynamics. The Materials may contain product roadmap information of AppDynamics. AppDynamics reserves the right to change any product roadmap information at any time, for any reason and without notice. This information is intended to outline AppDynamics' general product direction, it is not a guarantee of future product features, and it should not be relied on in making a purchasing decision. The development, release, and timing of any features or functionality described for AppDynamics' products remains at AppDynamics' sole discretion. AppDynamics reserves the right to change any planned features at any time before making them generally available as well as never making them generally available. All third-party trademarks, including names, logos and brands, referenced by AppDynamics in this presentation are property of their respective owners. All references to third-party trademarks are for identification purposes only and shall be considered nominative fair use under trademark law. © 2016 AppDynamics, Inc. All rights reserved.
- 4. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 4 1. Is the Server ok? Average CPU utilization Server load 2. Is the software ok? Transaction Response Time Transaction breakdown 3. How about customers? Customer breakdown
- 5. Today’s monitoring: Many siloed tools 5 Mobile/Web App Middleware Database Server Network Storage CheckoutTransaction ?? “Network 97%”“Slow SQL query”“JVM perf issues”“Checkout is slow” APPDYNAMICS CONFIDENTIAL AND PROPRIETARY
- 6. AppDynamics Unified Monitoring vision Industry-first application-centric unified monitoring platform APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 6 SaaS On-Prem Flexible Deployment Server Visibility Browser Synthetic Monitoring Mobile Real-User Monitoring Database Visibility Browser Real–User Monitoring Application Performance Management THE APPLICATION INTELLIGENCE PLATFORM Server DB Web Server API APP Mobile Business Transaction One solution: one UI, one data platform, one install
- 7. Unified Monitoring Platform can: • Monitor overall application flow • End-to-end visibility into Application lifecycle • Drill-down into a performance problem all the way to code level details 7APPDYNAMICS CONFIDENTIAL AND PROPRIETARY
- 8. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 8
- 9. Beyond APM • Sometimes knowing where the problem is doesn’t tell you how to solve it • Context and multi dimensions • Full picture • Index huge volume of data • Search & get insight 9APPDYNAMICS CONFIDENTIAL AND PROPRIETARY
- 10. How to fill the gap? 10APPDYNAMICS CONFIDENTIAL AND PROPRIETARY
- 11. Filling the gap requires Log Analytics APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 11 • Log Analytics and APM complement each other • Use APM to monitor your App and find issues in seconds • Use Log Analytics to correlate the issue to a root cause in a single pane of glass • Slice & dice through your data and create advanced searches to find the context • Auto - correlation between business transactions and log data = Faster MTTR • Get visibility over un-instrumented tiers
- 12. Log Analytics
- 13. What is AppDynamics Log Analytics? • Real-time, big data analytics platform (SaaS & on-premises) • Single end-to-end solution APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 13 Collect Data Process Data Visualize Data Act on Data
- 14. Architecture
- 15. Agenda • Agent Architecture • SaaS Architecture • On-premise Architecture • SaaS Statistics • Quality of Service • Road Ahead APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 15
- 16. Agent architecture APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 16
- 17. Event service architecture (SaaS) APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 17
- 18. Event service architecture (on premise) APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 18
- 19. Statistics in SaaS APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 19 • Total number of events/day • Total number of requests/day • Total number of bytes/day • Total number of AWS nodes
- 20. QoS APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 20 • We monitor our Saas Analytics using Analytics • 99 percentile for event publish ~= 100 ms • 99 percentile for queries ~= 5 secs
- 21. Technical challenges ahead APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 21 • Streaming back Query Results (4.3) • Supporting longer data retention • Query Performance/Scalability Improvements • Clustering • Anomaly Detection via Machine Learning
- 22. Key use cases
- 23. IT Operations Management Pinpoint errors and troubleshoot with logs APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 23 Order Service Order Processing Service Inventory Service Fulfillment Service Slow BT IT Ops sees slow BT with the Order Processing Service in their APM flowmap Load balancer logs reveal a misconfiguration of irules, causing the slow transaction F5 Load Balancer Get comprehensive visualization into your infrastructure by correlating and analyzing machine data in real time Gain infrastructure visibility
- 24. Advanced Performance Analysis: Breakdown Log data silos APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 24
- 25. Advanced Performance Analysis contd.
- 27. Business Health Dashboard $377,997 $111,802 $42,591 $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 Normal Slow Very Slow Diamond 11% Platinum 38% Gold 25% Silver 16% Bronze 10% Top Product Categories Customers by Tier Average Response Time Total Revenues: $532,390 Total Revenues by Transaction Speed Top Cities Performance timestamp indicating trend towards problem PERSONAL CARDS SMALL BUSINESS CORPORATECARDS PREPAID CARDS TRAVEL REWARDS MERCHANTS Top product categories generating highest revenue Most of customers experiencing issues are Platinum Revenue impact of poor performance $0 $20,000 $40,000 $60,000 $80,000 $100,000 New York San Francisco Honolulu Bangalore London Paris Revenue by cities
- 28. Key features in future releases
- 29. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 29 ADQL * Math Operations * Advanced Operators * Improved Free-text Search Performance & Metrics Improvements Advanced Search Smarter Insights BT & Log Correlation Relevant Fields * Server & Log correlation * Log Clustering Richer Data set * Query Performance Improvement * Metrics Centralized Management * Centralized Source Log Config Management * Centralized Source –Agent Mapping * Universal Agent Business Transactions Logs Browsers Analytics Mobile Analytics Custom Events Sessions Synthetic
- 30. Math Operations in ADQL APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 30 • Arithmetic on raw fields: SELECT startTime - endTime AS duration FROM logs • Arithmetic within aggregations: SELECT avg(startTime - endTime) FROM transactions • Arithmetic of aggregations: SELECT count(errorMessage) / count(*) FROM transactions • Arithmetic in WHERE clauses: SELECT * FROM transactions WHERE startTime - endTime > 100 • Filter in SELECT clause: SELECT series(eventTimestamp, '1h'), filter(count(*), WHERE action=purchase) / filter(count(*), WHERE action=view) FROM … • ORDER BY: SELECT count(action) as failure FROM logs WHERE action = purchase AND status = ‘404’ ORDER BY failure
- 31. Improved free-text search APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 31 • Removing the first three characters limits for wildcards • Improving wildcard searches over tokens • Camel-case • Non alpha-numeric characters all delimiters
- 32. Metrics APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 32
- 33. Query performance improvements APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 33 • Server side pagination of results • Impacts Data Grid, Table Widget • Streaming results for ADQL queries • Impacts widgets, ADQL and API • Better UX for long running queries
- 34. Centralized log configuration APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 34 Collection Rules Field Extraction Field Customization Agent Mapping • Path • Timestamp • Time range • Time zone • Multi-line • GROK Patterns • Key/value Pairs • Auto-regex • Manual regex • Validation • Mask sensitive data • Remove a field • Rename a field • Change type • Map source configs to agents
- 35. Add source rules APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 35
- 36. Source rules contd. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 36
- 37. Field extraction - GROK APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 37
- 38. Auto field extraction APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 38
- 39. Auto field extraction contd. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 39
- 40. Auto field extraction contd. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 40
- 41. Auto field extraction contd. APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 41
- 42. Manual field extraction APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 42
- 43. Field customization actions • Mask sensitive information – Mask partial value – Eg. Credit card number except last four digits ******7890 – Mask entire value – Eg. Social security number ######## • Remove field • Replace field • Rename field • Change type APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 43
- 44. Field customization APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 44
- 45. Source – agent mapping APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 45
- 46. Adding agent scope APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 46
- 47. Source – agent mapping APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 47
- 48. Log clustering (machine learning) • Log events come in large volume, hard to be analyzed manually • Group log events based on patterns • Show changes in real time • Apply anomaly detection APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 48
- 49. What is common among these log events? APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 49
- 50. What is common? APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 50 *. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *... IP address
- 51. What is common? APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 51 *. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *... Timestamp
- 52. What is common? APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 52 *. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *... URL
- 53. We provide a summarized view... APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 53 INFO ajp-nio-*-exec-* *-*-*/*:*:* com.appdynamicspilot.action.CartAction ADD ... log patterncount Pct. 129k 65k 5.2% … 10.2% *. *. *. * -- [*/*/*:*:*:* +0000] “GET / * HTTP/1.1” 200 *...
- 54. We can define possible anomalies APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 54
- 55. Anomalies (contd.) APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 55 Anomaly
- 56. Unify deployment and config APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 56 • One package to rule them all! • User downloads, deploys, and configures one package.
- 57. Agent installation today APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 57 Manual: Have to install each agent Diverse: Each agent install is unique Repetitive: Similar steps for each agent Config: Each configured separately
- 58. Centralize configuration APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 58 - Use the same config on all machines - Use rules to describe which pieces of config apply to which machines.
- 59. Analytics Agent as part of AppDynamics Universal Agent APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 59 Download Install Upgrade Enable Disable Universal Agent
- 60. Please give us your feedback—Session T11518 • Complete the online survey you'll receive via email later today or via text at: Text this number: 878787 Text this word: APPSPHERE • Every time you submit a session survey, your name will be entered in a random drawing. We're giving away Amazon Echos to 5 lucky winners! • Thank you for your input APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 60 Win!
- 61. Thank you