Parallel In-Memory Processing and Data Virtualization Redefine Analytics Architectures
0 likes177 views
The document outlines the benefits of data virtualization as presented in a webinar series, focusing on Denodo's platform that simplifies data integration and analytics through parallel in-memory processing, logical architectures, and advanced optimization techniques. It highlights how Denodo facilitates quick data access, security governance, and self-service capabilities while emphasizing its ability to integrate disparate data sources in real-time. Real-world experiences from users underscore the platform's efficiency and effectiveness in handling complex data requirements.
Parallel In-Memory Processing and Data Virtualization Redefine Analytics Architectures
- 1. DATA VIRTUALIZATION PACKED LUNCH WEBINAR SERIES Sessions Covering Key Data Integration Challenges Solved with Data Virtualization
- 2. Parallel In-Memory Processing and Data Virtualization Redefine Analytics Architectures Alberto Pan CTO, Denodo
- 3. 3 Logical Architectures for Analytics
- 4. 4 Logical Architectures: Data Virtualization
- 5. The Role of Denodo: Data Combination Connect to any type of data source Combine data from several systems Publish it to the desired format …zero-code development of complex data combinations and transformations The Denodo Platform enables us to build and deliver data services, to our internal and external consumers, within a day instead of the 1 – 2 weeks it would take with ETL.” – Manager, DrillingInfo
- 6. The Role of Denodo: Abstraction Abstracts access to disparate data sources. Acts as a single virtual repository. Abstracts data complexities like location, format, protocols …hides data complexity for ease of data access by business Enterprise architects must revise their data architecture to meet the demand for fast data.” – Create a Road Map For A Real-time, Agile, Self- Service Data Platform, Forrester Research
- 7. The Role of Denodo: Logical Views for Self-Service Enables creation of semantic models reflecting business taxonomy Create multiple logical views over the same physical data, adapted to each type of user / Line of Business …enables information discovery and self-service Impressively quick turn around time to "unlock“ data from additional siloes and from legacy systems - Few vendors (if any) can compete with Denodo's support of the Restful/Odata standard - both to provide data (northbound) and to access data from the sources (southbound).” – Business Analyst, Swiss Re
- 8. Centralized metadata, security & governance Abstracts data source security models and enables single-point security and governance. Extends single-point control across cloud and on- premises architectures Provides multiple forms of metadata (technical, business, operational) to facilitate understanding of data. …simplifies data security, privacy, audit Our Denodo rollout was one of the easiest and most successful rollouts of critical enterprise software I have seen. It was successful in handling our initial, security, use case immediately, and has since shown a strong ability to cover additional use cases, in particular acting as a Data Abstraction Layer via it's web service functionality.” – Enterprise Architect, Asurion
- 9. The Role of Denodo: Advanced Optimization Optimizer specifically designed for logical architectures Minimizes data movement and maximizes local processing at the origin systems Cost-based automatic optimization decisions Optimize distributed queries across large repositories Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.” – Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015
- 10. 10 Query Optimization: Example (1) Naive Strategy (BI Tools, BDI Tools, Simple federation engines): join union group by Customers (3M) Sales previous years (3B)Sales this year (290M) 290M rows 300M rows (sales previous year) 3M rows 593M rows through the network Obtain Total Sales By Customer Country in the Last Two Years
- 11. 11 Query Optimization: Example (2) Denodo Strategy join union group by Customers (3M) Sales previous years (3B)Sales this year (290M) 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows 9 M rows through the network Obtain Total Sales By Customer Country in the Last Two Years group by customer group by customer
- 12. Query Optimization: Example (and 3) union group by 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows (customers) Aggregation pushdowngroup by customer group by customer join Integrated MPP processing System Execution Time Optimization Technique No Rewriting 20 min None Denodo 6 51 sec Aggregation push-down Denodo 7 24 sec Aggregation push-down + MPP integration
- 13. Demo Logical Data Warehouse Scenario 1. Create Data Sources and Import Source Tables in Denodo • DW: Sales current year • Hadoop: Sales previous years • Oracle: Customer data 2. Create logical sales view 3. Run example reports and see execution trace 13
- 14. Demo 14
- 15. Q&A
- 16. Next steps Download Denodo Express: www.denodoexpress.com Access Denodo Platform on AWS: www.denodo.com/en/denodo-platform/denodo- platform-for-aws
- 17. 17
- 18. Thank you! © Copyright Denodo Technologies and Daman, Inc. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies and Daman, Inc