Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)
- 1. 2017 - Madrid Overview of Azure Data Factory Carlos Sacristán Data & Analytics Solution Architect, Kabel #GIBMad2017
- 2. Who am I? Carlos Sacristán Data & Analytics Solution Architect, Kabel [email protected] https://twitter.com/sacrisql +34 649 425 928 https://www.linkedin.com/in/csacristan/ #GIBMad2017
- 4. #GIBMad2017
- 5. What is Azure Data Factory ADF is a cloud-based data integration service that orchestrates and automates the movement and transformation of data Think of it like a manufacturing factory running equipment to take the raw materials and transform them into finished goods #GIBMad2017
- 6. What is ADF Mmmm… but we already have things like Integration Services or Stream Analytics #GIBMad2017
- 7. #GIBMad2017
- 9. Evolving approaches to analytics #GIBMad2017
- 10. #GIBMad2017
- 14. Activities
- 16. Just one thing. Scheduling Pipeline Active Periods Activity Schedule Dataset Availability #GIBMad2017
- 17. Just one thing. Scheduling #GIBMad2017
- 18. So, recap: when is executed an Activity? #GIBMad2017
- 19. #GIBMad2017
- 20. Process large-scaled datasets with ADF and Azure Batch
- 21. Customer Churn Azure Blob Storage Game Log Files Customer Table On Premises Data Mart Game Logs Customer Table Azure DB Customer Game Usage Visualize Data Set (Collection of files, DB table, etc) Activity: a processing step (Hadoop job, custom code, ML model, etc) Pipeline: a sequence of activities (logical group) Data Sources Ingest Transform & Analyze Publish Customer Table Geocode Transform, Combine, etc Analyze Move #GIBMad2017