Skip to content

Commit bf9b6b6

Browse files
authored
[AIRFLOW-5013] Add GCP Data Catalog Hook and operators (#7664)
1 parent 29e848d commit bf9b6b6

File tree

9 files changed

+6688
-0
lines changed

9 files changed

+6688
-0
lines changed

airflow/providers/google/cloud/example_dags/example_datacatalog.py

Lines changed: 443 additions & 0 deletions
Large diffs are not rendered by default.

airflow/providers/google/cloud/hooks/datacatalog.py

Lines changed: 1233 additions & 0 deletions
Large diffs are not rendered by default.

airflow/providers/google/cloud/operators/datacatalog.py

Lines changed: 1963 additions & 0 deletions
Large diffs are not rendered by default.

docs/conf.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,7 @@
280280
'google-cloud-bigtable': ('https://googleapis.dev/python/bigtable/latest', None),
281281
'google-cloud-container': ('https://googleapis.dev/python/container/latest', None),
282282
'google-cloud-core': ('https://googleapis.dev/python/google-cloud-core/latest', None),
283+
'google-cloud-datacatalog': ('https://googleapis.dev/python/datacatalog/latest', None),
283284
'google-cloud-datastore': ('https://googleapis.dev/python/datastore/latest', None),
284285
'google-cloud-dlp': ('https://googleapis.dev/python/dlp/latest', None),
285286
'google-cloud-kms': ('https://googleapis.dev/python/cloudkms/latest', None),

docs/howto/operator/gcp/datacatalog.rst

Lines changed: 614 additions & 0 deletions
Large diffs are not rendered by default.

docs/operators-and-hooks-ref.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -602,6 +602,12 @@ These integrations allow you to perform various operations within the Google Clo
602602
- :mod:`airflow.providers.google.cloud.operators.datafusion`
603603
-
604604

605+
* - `Datacatalog <https://cloud.google.com/data-catalog>`__
606+
-
607+
- :mod:`airflow.providers.google.cloud.hooks.datacatalog`
608+
- :mod:`airflow.providers.google.cloud.operators.datacatalog`
609+
-
610+
605611
* - `Dataflow <https://cloud.google.com/dataflow/>`__
606612
-
607613
- :mod:`airflow.providers.google.cloud.hooks.dataflow`

setup.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,7 @@ def write_version(filename: str = os.path.join(*[dirname(__file__), "airflow", "
227227
'google-cloud-bigquery-datatransfer>=0.4.0',
228228
'google-cloud-bigtable>=1.0.0',
229229
'google-cloud-container>=0.1.1',
230+
'google-cloud-datacatalog>=0.5.0',
230231
'google-cloud-dataproc>=0.5.0',
231232
'google-cloud-dlp>=0.11.0',
232233
'google-cloud-kms>=1.2.1',

0 commit comments

Comments
 (0)