-
Notifications
You must be signed in to change notification settings - Fork 466
[GCP] Add dataproc data stream #3789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🌐 Coverage report
|
/test |
|
||
The `dataproc` dataset fetches metrics from [Dataproc](https://cloud.google.com/dataproc/) in Google Cloud Platform. It contains all metrics exported from the [GCP Dataproc Monitoring API](https://cloud.google.com/monitoring/api/metrics_gcp#gcp-dataproc). | ||
|
||
You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, it will return metrics from all buckets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this reference to GCP storage appropriate here?
packages/gcp/changelog.yml
Outdated
@@ -4,6 +4,9 @@ | |||
- description: Remove fields duplicated in ECS fields | |||
type: enhancement | |||
link: https://github.com/elastic/integrations/pull/3609 | |||
- description: Add GCP Dataproc Data stream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs update to latest gcp
package version.
|
||
The `dataproc` dataset fetches metrics from [Dataproc](https://cloud.google.com/dataproc/) in Google Cloud Platform. It contains all metrics exported from the [GCP Dataproc Monitoring API](https://cloud.google.com/monitoring/api/metrics_gcp#gcp-dataproc). | ||
|
||
You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Dataproc does not use zones so `us-central1-a` will return nothing. If no region is specified, it will return metrics from all buckets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After reading this line, I was wondering if we can simply remove the zone
configuration from the data stream. what do you think?
You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Dataproc does not use zones so `us-central1-a` will return nothing. If no region is specified, it will return metrics from all buckets. | |
You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Dataproc is a regional service. If no region is specified, it will return metrics from all buckets. |
What does this PR do?
Adds
dataproc
data stream.Checklist
changelog.yml
file.Author's Checklist
How to test this PR locally
Related issues
Screenshots