Denodo Data Virtualization Platform: Overview (session 1 from Architect to Architect webinar series)
8 likes6,626 views
The document outlines a series of five webinars focused on the Denodo platform for data virtualization, featuring an architectural overview, performance, scalability, data governance, and security. It highlights key features of Denodo Express, including a user-friendly design tool and extensive data connectivity, as well as advanced query optimization and caching capabilities. Additionally, it emphasizes the platform's support for high availability, metadata management, and customizable security mechanisms.
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Architect webinar series)
- 1. Five In-depth Technology and Architecture Sessions on Data Virtualization Session 1: Denodo Platform – Architecture
- 2. Today’s Speaker ■Paul Moxon Senior Director, Product Management
- 3. Architect-to-Architect Series ■Series of five webinars over next 2 months ■Deeper look into Denodo Platform ■Architectural Overview (today’s sessions) ■Performance ■Scalability ■Data Discovery and Governance ■Security
- 4. Denodo Express ■Denodo Express ■Free to Download ■Fully functioning Data Virtualization Platform ■Single user, supports common data sources ■Many of the same capabilities of Denodo Platform ■Performance, Data Discovery, Governance, internal Security, Publishing, …
- 5. •Intuitive graphical design tool •Drag-and-drop, point-and-click modeling and integration •Connectivity to all types of data sources •Native support for hierarchical data structures via ERM •Extensive on-the-fly data transformation & data cleansing and matching functions •Multiple ‘publishing’ protocols •Metadata exploration •Data lineage, dependency analysis, change impact analysis, etc. •Global Search •Search data and metadata across all data sources •Team/distributed development models •Version Control System integration … Denodo Platform – Design time
- 6. •Proven scalability & performance •Scales from departmental/project to enterprise-wide with geographically distributed nodes •Handles largest transactional (users, transactions, or sources) and analytical loads (data volumes) • Advanced query optimizer •Cost-based & source constraint- based query optimization incl. query delegation, query rewrite, join optimization, data movement, … •Intelligent caching •Full, partial, time-based caching •Fine-grained security model •Integrate with external systems •Enterprise–grade monitoring and auditing •Integration with System Management products e.g. HP BTO, Unicenter, Tivoli, etc. •Full development & deployment lifecycle support Denodo Platform - Runtime
- 7. Denodo Platform – Detailed Architecture
- 8. Performance – Architecture Modules
- 9. Performance Optimization ■Advanced Query Optimization ■Cost and Source Constraint Based Query Plans ■Query Delegation ■Automatic Query Rewriting ■Join Optimizations ■Data Movement ■Asynchronous Multi-threaded Processing ■Server Throttling Mechanisms ■Scalability ■Caching ■Multiple configuration modes – full or partial
- 10. Performance – Optimizer, etc. ■Optimizer ■The Optimizer applies state-of-the-art optimization techniques to relational and non- relational sources. ■Query Plan Generator ■The Plan Generator is in charge of generating possible execution plans for the query and selecting the optimum one. ■Execution Engine ■Responsible for executing the selected query plan, executing the necessary sub-queries on the sources (or collecting data from cache as appropriate) and integrating the results to generate the global response.
- 11. Scalability – Architecture Modules
- 12. Scalability – HA and Clustering ■High Availability Architecture ■Active-Passive ■Active-Active ■Vertical Scalability ■VM Support ■Metadata Sharing ■Horizontal Scalability ■Clustering ■Shared Cache or Individual Cache
- 13. Scalability – HA and Clustering ■Denodo Cache ■The Denodo Platform incorporates an advanced cache system to improve performance ■For data materialization it makes use of an external JDBC database (physical or in-memory) ■Denodo High-Availability ■Set of features provided by the Denodo Platform to support High Availability and Load Balancing ■Denodo Cluster Management ■Tool for monitoring and managing a cluster of Denodo servers
- 14. Data Discovery and Governance – Architecture Modules
- 15. Data Discovery and Governance ■Metadata Repository ■Apache Derby database ■Contains view and data source configuration data ■Relationships between views – dependencies, etc. ■Searchable – Catalog searches, etc. ■Data Lineage ■Trace how data changes between source and consumer ■Change Impact Analysis ■What is the impact of a change in a data source schema?
- 16. Data Discovery and Governance ■RESTful Web Services ■Supports Global Search functionality ■Index and search data sources ■‘Google’ like search
- 17. Security – Architecture Modules
- 18. Security ■Authentication & Authorization ■Built-in User/Role Management Module ■Integration with external entitlement servers (LDAP/AD) ■Multi-level access controls ■Database, View, Row, Column, and Cell ■Policy-based Security and Workload Management ■Enforcement of custom policies for query execution according to security / workload considerations
- 19. Extensibility – Architecture Modules
- 20. Extensibility ■Stored Procedures ■Extension point for custom behavior (based on a Java SDK) ■Eclipse Plugin for ease-of-development ■Custom Functions ■Extension point for custom transformation and data quality functions ■Custom Wrappers ■Extension point for custom data sources ■Denodo API ■Integration point for external tools and utilities
- 21. Q&A
- 22. Data Virtualization – Next Steps Move forward at your own pace Download Denodo Express – The fastest way to Data Virtualization Denodo Community: Documents, Videos, Tutorials, more. Attend Architect-to-Architect Series Performance Scalability Move forward with one of our Data Virtualization experts Phone: (+1) 877-556-2531 (NA) Phone: (+44) (0)20 7869 8053 (EMEA) Email: [email protected] | www.denodo.com Data Discovery and Governance Security
- 23. Five In-depth Technology and Architecture Sessions on Data Virtualization Thank You! Next Session Session 2 Denodo Platform: Performance