Cloud and Data Analytics Architecture: Data Everywhere for Everyone
- 1. Cloud and Data Analytics Architecture Data Everywhere for Everyone Big Data Analytics in Power and Utilities Industry Summit Amsterdam 2022
- 2. innogy CZ market position: Market leader in natural gas, fastest-growing in electricity and #1 on the CNG market.
- 3. innogy · Presentation title · DD month YYYY
- 4. 4 Our cloud story year by year
- 5. Cloud will handle all your responsibility on security and data protection
- 6. innogy · Presentation title · DD month YYYY 6
- 7. 7 • AWS OnPremise • Private Cloud (single-tenant): ̶ customer specific development • Public Cloud (multi-tenant): ̶ custom development • Customer AWS account • Vendor AWS account innogy cloud application landscape
- 8. 8 innogy Shared Responsibility Model
- 9. WHY DOESN’T MODERN CLOUD RESULT IN A MODERN DATA EXPERIENCE?
- 10. 10 Analytical Database at its Center • ETL data pipelines that extract all the data to database or datawarehouse at its center Full-blown Data Streaming • A set of microservices commonly configured as a messaging queue Full-blown Data Mesh • A set of microservices that are not product-oriented like streaming architectures, but are domain-focused on the teams and products Data Lake at its Center • ELT pipelines that store everything in a data lake, which gets later integrated into an approved data model.
- 11. innogy · Presentation title · DD month YYYY 11 We integrate so many and focusing more on the integration, administration, and maintenance of those tools instead of building data products. - Who’s going to use the tool? - What data problems is the tool trying to solve? - What are you data governance requirements? Challenges of the Modern Data Stack?
- 12. TOOLS
- 13. 1 Engineering principles testing, code review, CICD 2 SQL well-known language 3 Out-of-the-box testing data integrity, referential constraints, and semantic validity 4 Documentation knowledge repository, data lineage 5 Versioning integrate with git 6 Deployment DataOps practices Source: 7 reasons why we integrated dbt into Keboo
- 14. DATA ACADEMY
- 16. innogy · Presentation title · DD month YYYY 16 Modern Cloud Data Architecture
- 17. DATA MESH As a new approach to sourcing, sharing, accessing, and managing analytical data at scale
- 18. Data & Analytics Modernization Days 2022: How to leverage AWS Modern Data Architecture to Accelerate your Data Strategy 18 Data Producers Domain expertise Data ownership and governance Data Quality Metadata Management Data Mesh Build security controls Build and run the platform Simplify on-boarding Training and community Data Consumers Execute business priorities Business analytics development Data Discovery Data Pipeline development Creation of new insights
- 19. Think Big, start Small and Be Prepared
- 20. Thank you!