DAS Slides: Data Virtualization – Separating Myth from Reality
0 likes973 views
Data virtualization allows for logical integration of data from diverse sources without physical movement. Despite its appealing prospects, current adoption is low but anticipated to rise significantly in the coming years. It complements existing data solutions while requiring strong data governance and understanding of its specific use cases.
DAS Slides: Data Virtualization – Separating Myth from Reality
- 1. Copyright Global Data Strategy, Ltd. 2020 Data Virtualization Separating Myth from Reality Donna Burbank Global Data Strategy, Ltd. September 24th, 2020 Follow on Twitter @donnaburbank @GlobalDataStrat Twitter Event hashtag: #DAStrategies
- 2. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Donna Burbank 2 Donna is a recognised industry expert in information management with over 20 years of experience in data strategy, information management, data modeling, metadata management, and enterprise architecture. Her background is multi-faceted across consulting, product development, product management, brand strategy, marketing, and business leadership. She is currently the Managing Director at Global Data Strategy, Ltd., an international information management consulting company that specializes in the alignment of business drivers with data-centric technology. In past roles, she has served in key brand strategy and product management roles at CA Technologies and Embarcadero Technologies for several of the leading data management products in the market. As an active contributor to the data management community, she is a long time DAMA International member, Past President and Advisor to the DAMA Rocky Mountain chapter, and was awarded the Excellence in Data Management Award from DAMA International. Donna is also an analyst at the Boulder BI Train Trust (BBBT) where she provides advice and gains insight on the latest BI and Analytics software in the market. She was on several review committees for the Object Management Group’s for key information management and process modeling notations. She has worked with dozens of Fortune 500 companies worldwide in the Americas, Europe, Asia, and Africa and speaks regularly at industry conferences. She has co- authored two books: Data Modeling for the Business and Data Modeling Made Simple with ERwin Data Modeler and is a regular contributor to industry publications. She can be reached at [email protected] Donna is based in Boulder, Colorado, USA. Follow on Twitter @donnaburbank @GlobalDataStrat Twitter Event hashtag: #DAStrategies
- 3. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com DATAVERSITY Data Architecture Strategies • January 23 Emerging Trends in Data Architecture – What’s the Next Big Thing? • February 27 Building a Data Strategy - Practical Steps for Aligning with Business Goals • March 26 Cloud-Based Data Warehousing – What's New and What Stays the Same • April 23 Master Data Management – Aligning Data, Process, and Governance • May 28 Data Governance and Data Architecture – Alignment and Synergies • June 25 Enterprise Architecture vs. Data Architecture • July 22 Best Practices in Metadata Management • August 27 Data Quality Best Practices • September 24 Data Virtualization – Separating Myth from Reality • October 22 Data Architect vs. Data Engineer vs. Data Modeler • December 1 Graph Databases: Practical Use Cases 3 This Year’s Lineup
- 4. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com What We’ll Cover Today 4 • Data virtualization is a practice that logically integrates data from disparate sources without the need to physically move the data. • While this can be an appealing prospect, there is a good deal of confusion around this technology and how to use it to full advantage. • This webinar will explain the pros and cons of data virtualization along with practical use cases for implementation.
- 5. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com What is Data Virtualization? 5 Data Warehouse 011001101 101010010 Data Lake External Data Feeds Databases Etc… Data Virtualization is a logical data layer that integrates disparate data sources across the enterprise without physically moving the data. Data Virtualization Layer Reporting & Query Layer
- 6. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com What is Data Virtualization – Analyst Definition Data virtualization — The utilization of logical views of data, which may or may not be cached in various forms within the data integration application server or systems/memory managed by that application server. Data virtualization may or may not include redefinition of the sourced data. Gartner Magic Quadrant for Data Integration Tools, 18 August 2020 - ID G00450251 6
- 7. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Virtualization Trends 7 • Data virtualization does not have the widespread adoption that other data management trends enjoy (e.g. DW), according to a recent DATAVERSITY survey.* • Adoption has declined significantly with a decrease of nearly 30% between 2020 and 2019. Note: Respondents were able to select more than one response. * Trends in Data Management, a 2020 DATAVERSITY® Report, by Donna Burbank and Michelle Knight
- 8. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Virtualization Trends 8* Sharat Menon, Mark Beyer, Ehtisham Zaidi, Ankush Jain, Gartner Market Guide for Data Virtualization, November 16, 2018 • Although current adoption is low, many see positive adoption trends in the future. • According to the Gartner global research and advisory company, • Through 2022, 60 percent of enterprises will implement some form of data virtualization as one enterprise production option for data integration • Compared to less than 40 percent in 2018 and 11 percent in 2011.
- 9. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Movement vs. Data Virtualization 9 ETL Data Warehouse Cubes / Semantic Layer Source Systems BI / Reporting LayerData Consolidation & Historical Trending Data Transformation & Movement Traditional approaches, such as Data Warehousing, typically rely on moving data from source systems to a centralized location.
- 10. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Movement vs. Data Virtualization 10 Source Systems BI / Reporting Layer Data Virtualization leaves data in place, and provides a virtualization layer across these systems for end user query and access. Data Virtualization Layer
- 11. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Movement vs. Data Virtualization 11 Source Systems BI / Reporting Layer Disparate data sources and data types can be integrated via the virtualization layer – without the need to move the data from its source. Data Virtualization Layer External Data FeedsERP & CRM Systems Cloud Data Platforms Etc..
- 12. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Virtualization vs. ETL 12 Extract, Transform, Load (ETL) • Best when physical movement, integration and storage of the data is needed • Enterprise-grade efforts where business rule planning & enforcement is required. • Batch updates & processing (e.g. nightly, weekly, monthly) • Supports data transformation, cleansing, grouping, etc. The two solutions can complement each other in an organization – it doesn’t need to be an either/or. Data Virtualization (DV) • Best when it is difficult, time-consuming, or expensive to physically move data. • Rapid prototyping or flexible environment where frequent changes required. • Near real-time updates of data • Delegates queries, joins and aggregations to the source system and returns the rows. ETL
- 13. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Virtualization vs. BI Cubes 13 BI Cubes • Easily understandable by business users • Provides user-friendly semantic layer • Specific to a particular BI tool (vendor lock-in) • Facilitates “slicing & dicing” of data The two solutions can complement each other in an organization – it doesn’t need to be an either/or. Data Virtualization (DV) • Requires SQL query skills – may be difficult for business users • Can be used across multiple BI tools and solutions – avoid vendor lock-in • Good solution for data scientists, BI developers, etc.
- 14. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Data Virtualization - What’s “Behind the Curtain”? 14 Data Virtualization isn’t “magic” and is generally supported by platform vendors who provide a wide array of component services to manage the virtualization layer. Security Query Optimization Data Catalog Data Federation Data Quality Lifecycle Management Data Governance Data Services & Interoperability Infrastructure Management Etc.
- 15. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com What Data Virtualization is Not 15 • Data Virtualization is not an easy way out doing the hard work of: • Data Modeling • Data Quality • Data Governance & Stewardship • Privacy & Security • Master & Reference Data • Etc…
- 16. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Use Cases 16 Data Virtualization Use Cases • Integration with disparate data sources – including a data warehouse • Real-time data access • Rapid Protoyping • Data exploration The two solutions can complement each other in an organization – it doesn’t need to be an either/or. Not Data Virtualization Use Cases • Enterprise Data Warehouse • Centralized Master Data Management • Reference Data • Graph Data • Document Management
- 17. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Summary • Data Virtualization provides a flexible way to integrate data from disparate systems without the need to physically move the data • While current adoption is low, expectations are high for future growth • Data virtualization does not obviate the need for strong data governance, security, etc. • It is important to clearly understand the use cases in order to select the correct fit for purpose solution • Data virtualization can be a complementary technology to other enterprise data solutions (e.g. data warehouse)
- 18. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Trends in Data Management: A 2020 Report 18 Whitepaper available for free download on Dataversity.net at: https://content.dataversity.net/DV2020DataManage mentRP_DownloadWP.html
- 19. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com About Global Data Strategy™, Ltd • Global Data Strategy™ is an international information management consulting company that specializes in the alignment of business drivers with data-centric technology. • Our passion is data, and helping organizations enrich their business opportunities through data and information. • Our core values center around providing solutions that are: • Business-Driven: We put the needs of your business first, before we look at any technology solution. • Clear & Relevant: We provide clear explanations using real-world examples. • Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s size, corporate culture, and geography. • High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of technical expertise in the industry. 19 Data-Driven Business Transformation Business Strategy Aligned With Data Strategy Visit www.globaldatastrategy.com for more information
- 20. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com DATAVERSITY Data Architecture Strategies • January 23 Emerging Trends in Data Architecture – What’s the Next Big Thing? • February 27 Building a Data Strategy - Practical Steps for Aligning with Business Goals • March 26 Cloud-Based Data Warehousing – What's New and What Stays the Same • April 23 Master Data Management – Aligning Data, Process, and Governance • May 28 Data Governance and Data Architecture – Alignment and Synergies • June 25 Enterprise Architecture vs. Data Architecture • July 22 Best Practices in Metadata Management • August 27 Data Quality Best Practices – with Nigel Turner • September 24 Data Virtualization – Separating Myth from Reality • October 22 Data Architect vs. Data Engineer vs. Data Modeler • December 1 Graph Databases: Practical Use Cases 20 Join us next month
- 21. Global Data Strategy, Ltd. 2020 www.globaldatastrategy.com Questions? 21 • Thoughts? Ideas?