Denodo DataFest 2017: Conquering the Edge with Data Virtualization
1 like273 views
The document discusses the importance of data virtualization for managing and analyzing IoT data across edge and cloud environments. It highlights five essential capabilities of data virtualization, including real-time information access and zero data replication, and provides case studies demonstrating its impact on businesses in improving service delivery and operational efficiency. Ultimately, it emphasizes that data virtualization facilitates efficient data integration and enhances decision-making in real-time.
More Related Content
Similar to Denodo DataFest 2017: Conquering the Edge with Data Virtualization (20)
Denodo DataFest 2017: Conquering the Edge with Data Virtualization
- 1. Conquering the Edge with Data Virtualization Lakshmi Randall Director of Product Marketing, Denodo Twitter: @LakshmiLJ
- 2. 2 Source: Gartner Blog (http://blogs.gartner.com/thomas_bittman/2017/03/06/the-edge-will-eat-the- cloud/) The edge will eat the cloud. And this is perhaps as important as the cloud computing trend ever was.”
- 3. Which workloads will need an edge presence? The need for an edge presence will be defined & driven by data requirements … Speed Availability & reliability of access Rate of generation or security …or any combination Source: 451 Research 3
- 4. Which deployment locations do you plan to use to store and analyze IoT data in 2017 Source: 451 Research, Voice of the Enterprise: IoT Budgets and Outlook, 2017. n = 288 4
- 5. 5 IoT Network Architecture • IoT networks will contain multiple ‘layers’ of compute systems • Each layer will persist data • Some storage (e.g. at the sensor) might be temporary • e.g. last hour of readings • Not all data can be transmitted to next layer • Regulatory, privacy, security reasons leaving data where it is • Data moving to higher levels is typically aggregated for analytics and monitoring
- 6. 6 Layer 1 Layer 2 Layer 3 Layer 4 Layered Architecture
- 7. 7 Data Virtualization: Five Essential Capabilities 4. Self-service data services 5. Centralized metadata, security & governance 1. Data abstraction 2. Zero replication, zero relocation 3. Real-time information
- 8. 8 • Data Virtualization provides data access at the edge layer and data center/cloud layer • Connected DV platforms optimize data access by pushing processing to the data • Data Virtualization can be deployed at any intermediary layers between ‘edge’ and data center/cloud layer • e.g. Organizational, Regional, or Country based layers Data Virtualization at the Edge
- 9. 9 Connect rather than Collect Paradigm Case Study Example from Gartner Single Logical View
- 10. 10 DV Layer @ Edge DV Layer @ Data Center/Cloud Layer 1 Layer 2 Layer 3 Layer 4 Layered Architecture
- 11. Business Need Solution Benefits 11 Leading Heavy Equipment Manufacturer Improves Service Delivery and Revenue • Competitive pressure from low-cost Chinese manufacturers • Needed a proactive approach to customer service to differentiate • Sought to improve equipment and services delivery through predictive maintenance • Phased rollout systematically improved asset performance and proactive maintenance • Increased revenue from sale of services and parts • Reduced warranty costs of parts failure • Future – optimize pricing for services and parts among global service providers • Telemetry (IoT) data from sensors embedded in the equipment is stored in Hadoop to perform predictive analytics • Denodo integrates analytics data with parts, maintenance, and dealer information stored in traditional systems • It then feeds the predictive maintenance information to a customer dashboard In business for over 90 years and is the world’s leading manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines and diesel-electric locomotives with 2014 net revenue of $54.6 billion. Denodo @ Layer 4 (Data Center/Cloud)
- 12. 12 Denodo @ Layer 4 (Data Center/Cloud) Telemetry Analytics
- 13. Business Need Solution Benefits 13 World’s Largest Independent Biotechnology Firm Successfully Increases Production Yield and Plant Efficiency • Manufacturing data had to be aligned in real time to identify “weak signals” that could hinder production and cost millions • Streaming data from manufacturing, laboratory facilities, etc. had to be analyzed for optimal plant efficiency • “Golden Batch” had to be defined for increased production yield after combining current/historical data • Improving yield by 1–3% has saved $10 – 30 million per plant • Concerns with operating a new plant at low capacity are eliminated • ROI of Denodo easily justifies all work – will be used for all future applications • Data Virtualization through Denodo is cheaper and more efficient than ETL • Denodo Platform combines current manufacturing process data with historical data and feeds it to analytical systems • The systems analyze virtualized data, determine “golden batch,” and take corrective steps to ensure ideal yield • Denodo combines data from 46 source systems, of which 12 are key systems Headquartered in Thousand Oaks, California, the company is the world’s largest independent biotechnology firm, and sells a number of products treating autoimmune diseases, rheumatoid arthritis, etc. In 2015, the company reported a revenue of $20 billion with its 18,000 employees. Denodo @ Layer 3 (at the Edge)
- 14. 14 Denodo Beyond the Edge • Data Virtualization can also be leveraged beyond the edge – at local data collection level • e.g. factory or production line, retail store, ICU in hospital, etc. • Collection point for real-time data streams from sensors, devices • Stored in local databases, files, etc. • For sensor management and control • DV provides data access for analytics and reporting • Lightweight deployment for lower capability systems • e.g. Denodo in Docker Container
- 15. 15 DV Layer @ Edge DV Layer @ Data Center/Cloud Lightweight DV Layer Beyond Edge Layer 1 Layer 2 Layer 3 Layer 4 Layered Architecture
- 16. Business Need Solution Benefits 16 Large US Telecom Company Improves Inventory Alignment with Virtualized Master Data Views • Business users needed single view of networking device, customer, and site information from systems • The single views were needed for ticket and event enrichment, compliance and audit, and inventory alignment • The information was dispersed across multiple systems with varying levels of completeness and data quality • Customer satisfaction: High quality MDM data helped end customer receive better quality information • Inventory alignment: Improved information availability to internal and external applications • Improved compliance: Better inventory management and analytics helps with better compliance and audit standards • Denodo is used as a virtualized master data management (MDM) system • Denodo connects to multiple systems and Hadoop • Denodo aggregates and exposes the master data views via an integrated API layer • Data is consumed by ticket and event enrichment, dashboard, and compliance / audit applications The company is the second largest cable company in the U.S. by revenue, operating in 29 states, with corporate headquarters located in Midtown Manhattan, New York City. The company recorded revenue of US$23.69 billion in 2015. It’s Network Operations group is a thriving business providing the entire range of telecommunications services to both individual and commercial consumers. Denodo Beyond the Edge
- 17. 17 Denodo and Streaming Data • Integrate with streaming data systems e.g. Spark, Kafka, IBM Streams, etc. • Read data from queues or streams (e.g. Spark data frames) during ingestion process • Also enrich streaming data through Denodo Platform • Access temporal data in specific time window • e.g. last 2 hours of readings/data Read temp window buffers SQL based enrichment Secure + Combine + Enrich
- 18. 18 Complete enterprise information, combining Web, cloud, streaming, and structured data ROI realization within 6 months, with the flexibility to adjust to unforeseen changes An 80% reduction in integration costs, in terms of resources and technology Real-time integration and data access, enabling faster business decisions “Get it Real-time and Get it Fast!” The Benefits of Data Virtualization
- 19. Thank you! © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. #DenodoDataFest