Enterprise Monitoring and Auditing in Denodo
0 likes•168 views
The document outlines a webinar series focused on the monitoring capabilities of Denodo's data virtualization, detailing tools like Denodo Monitor, the Diagnostic & Monitoring Tool, and the Denodo Dashboard. It emphasizes the importance of data collection, context, and intent in monitoring, showcasing how these tools can help track server resources and queries. A live demo is included to illustrate these functionalities and the document concludes with a summary of key takeaways.
Enterprise Monitoring and Auditing in Denodo
- 1. Denodo TechTalks Product Deep-Dive Series A product deep-dive, webinar series covering the critical capabilities of Denodo’s modern data virtualization
- 2. Enterprise Monitoring and Auditing in Denodo Vijayalakshmi Mani Data Engineer| Denodo
- 3. AGENDA 1. Introduction 2. Monitoring in Denodo 3. Demo 4. Q&A
- 4. Introduction
- 5. 5 What is Monitoring? Introduction ● Data Collection - Taking and storing measurements from a system A car is going at 90 KM/H ● Context - The extra information needed to understand the collected data, such as timing, location, and external factors affecting the data. This drives decision criteria and thresholds. A car is going at 90 KM/H in a city. Therefore, it is going fast ● Intent - The business goals behind the monitoring. This drives actions and policies. The car is going at 90 KM/H in a city. Therefore, it is going fast. We want increase road safety so we are going to raise an alert to the Police Department. INTENT CONTEXT DATA COLLECTION
- 6. 6 Introduction The tools for Monitoring in Denodo Denodo Monitor (DM) Stand-alone data collection process Diagnostics & Monitoring Tool (DMT) Web application for real time monitoring Denodo Dashboard (DD) Operational and FinOps dashboards
- 7. 7 Introduction Denodo Monitor JMX Interface Denodo Platform Diagnostic & Monitoring Tool Denodo Dashboard Visualization Event Generation Data Collection The tools for Monitoring in Denodo
- 8. 8 What is the JMX interface? Introduction Java Management Extensions Management The JMX technology provides a standard way of managing resources. JMX technology can also be used to monitor and manage the Java Virtual Machine (Java VM). 2 Standard The JMX technology is a standard part of the Java Platform, Standard Edition (Java SE platform). 1 Denodo Virtual DataPort implements some additions over the standard to monitor and manage Denodo-specific resources. 4 Independent The way in which resources are instrumented is completely independent from the management infrastructure. It adds no overhead to the JVM 3 service:jmx:rmi:///jndi/rmi://denodo-server-prod.acme.com:9997/jmxrmi
- 10. 10 Denodo Monitor Monitoring in Denodo Cloud Relational Database File ● Collects most of the data exposed by the JMX interface and persists it in a storage solution: Files, Relational Database or Cloud storage ● Managed from Solution Manager for a centralized administration Easy to share across teams Not scalable and not suitable for analytics Suitable for analytics and better scalability Harder to share across teams Best scalability and easy to share Analytics need an execution engine ● Collects data about: ○ Server resources ○ Queries received by Virtual DataPort ○ Cache usage ○ Delegated queries and data source usage
- 11. 11 Diagnostic & Monitoring Tool Monitoring in Denodo ● Web-based application for real-time monitoring of Virtual DataPort Servers ● There is no persistence. Once the session is closed the data is lost ● It has the ability to load Denodo Monitor logs and perform post-mortem analysis
- 12. 12 Denodo Dashboard Monitoring in Denodo ● Custom Apache Superset delivered as part of our DenodoConnects program. ● Offers some out-of-the-box operational dashboards to monitor the Denodo Servers ● Can be extended to include custom enterprise metrics and dashboards
- 13. 13 Live Demo Monitoring in Denodo
- 14. 14 Takeway ● Denodo Provides and allows multiple Methods to monitor the queries being processed by the Denodo Platform servers. ○ Denodo Monitor for storing historical data data ○ Diagnostics & Monitoring Tool for Real time data ○ Using External Clients like JMX ● The all new Denodo Dashboard helps in visualizing the data stored by the Denodo Monitor. ○ Providing information on FinOps ○ Metadata related information ○ Resource Utilization of the server. Reference ● Monitoring Denodo servers with JMX ● Denodo Monitor ● Monitoring ● Diagnostic & Monitoring Tool Guide ● Denodo Dashboard - User Manual
- 15. Q&A