Data quality patterns in the cloud with ADF
Download as PPTX, PDF4 likes776 views
Azure Data Factory can be used to build modern data warehouse patterns with Azure SQL Data Warehouse. It allows extracting and transforming relational data from databases and loading it into Azure SQL Data Warehouse tables optimized for analytics. Data flows in Azure Data Factory can also clean and join disparate data from Azure Storage, Data Lake Store, and other data sources for loading into the data warehouse. This provides simple and productive ETL capabilities in the cloud at any scale.
Data quality patterns in the cloud with ADF
- 1. Data Quality Patterns in the Cloud with Azure Data Factory Mark Kromer Sr. ProgramManager Azure Data Factory
- 2. ADF: Simple and Productive ETL in the Cloud at any Scale
- 3. Modern Data Warehouse Pattern with ADF Mapping Data Flows Applications Dashboards Business/custom apps (structured) Logs, files, and media (unstructured) r Azure Storage/ Data Lake Store Azure Data Factory Load files into data lake on a schedule Azure Data Factory Extract and transform relational data Azure SQL DW Load processed data into tables optimized for analytics Clean and join disparate data Databases Azure Databricks
- 9. list of connectors full list of compute environments data
- 12. Data Flow
- 17. • iif (length(title) == 0,toString(null()),title)
Editor's Notes
- #3: Let’s take a look at the process through the visual user interface Orchestration. Directs other services to execute actions as part of the transformation process. Mapping Data Flows. Develop graphical data transformation logic at scale without writing code using Mapping Data Flows (preview). Mapping Data Flows. Develop graphical data transformation logic at scale without writing code using Mapping Data Flows (preview). Monitoring: Monitor pipeline and activity runs with a simple list view interface