[GCP] Add redis data stream #4325

gpop63 · 2022-09-27T15:32:47Z

What does this PR do?

Adds redis data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes [GCP] Collect metrics for Redis #4293

Screenshots

elasticmachine · 2022-09-27T15:53:52Z

🚀 Benchmarks report

Package `gcp` 👍(4) 💚(0) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`dns`	1908.4	1605.14	-303.26 (-15.89%)	💔

To see the full report comment with /test benchmark fullreport

elasticmachine · 2022-09-27T15:54:04Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-10-05T12:10:47.074+0000
Duration: 20 min 57 sec

Test stats 🧪

Test	Results
Failed	0
Passed	84
Skipped	0
Total	84

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2022-09-27T15:54:15Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`5/5`)	💚
Files	100.0% (`5/5`)	💚 2.53
Classes	100.0% (`5/5`)	💚 2.53
Methods	90.099% (`91/101`)	👎 -0.135
Lines	95.752% (`1375/1436`)	👍 4.15
Conditionals	100.0% (`0/0`)	💚

endorama · 2022-09-28T15:05:01Z

packages/gcp/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "2.12.6"


Suggested change

- version: "2.12.6"

- version: "2.12.0"

endorama · 2022-09-28T15:07:16Z

packages/gcp/data_stream/redis/elasticsearch/ingest_pipeline/default.yml

@@ -0,0 +1,131 @@
+---
+description: Pipeline for parsing GCP Redis metrics.
+processors:


All the fields here should have proper suffixes, like mentioned in https://www.elastic.co/guide/en/beats/devguide/current/event-conventions.html

endorama · 2022-09-28T15:09:15Z

packages/gcp/data_stream/redis/fields/fields.yml

+- name: gcp.redis
+  description: Google Cloud Redis metrics
+  type: group
+  fields:


Please add unit, metric_type and dimension where appropriate (see #4099 for more details)

endorama · 2022-09-28T15:12:24Z

packages/gcp/manifest.yml

@@ -1,6 +1,6 @@
 name: gcp
 title: Google Cloud Platform
-version: "2.11.6"
+version: "2.12.6"


Suggested change

version: "2.12.6"

version: "2.12.0"

endorama · 2022-09-29T09:23:49Z

@gpop63 please fix the CI (needs a build)

endorama

Apart from the red CI looks good to me!

endorama · 2022-10-05T08:36:22Z

/test

gpop63 · 2022-10-05T09:38:58Z

Tested data stream with new field units etc and works as expected.

efd6 · 2022-10-06T00:20:45Z

packages/gcp/data_stream/redis/fields/agent.yml

+  group: 2
+  description: Fields related to the cloud or infrastructure the events are coming from.
+  footnote: 'Examples: If Metricbeat is running on an GCP Compute VM and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.'
+  type: group


Some of these fields are duplicates of fields that are defined in ecs.yml. Fixed in #4397. You can check for duplicates in a package if you are running elastic-package v0.65.0 with format version 2.0.0 (do this temporarily before pushing).

add redis data stream

867777a

gpop63 added 3 commits September 28, 2022 16:49

fix field types

3c20d6f

add ingest pipeline

58ea070

update docs

6f10450

gpop63 marked this pull request as ready for review September 28, 2022 14:36

gpop63 requested review from a team as code owners September 28, 2022 14:36

endorama mentioned this pull request Sep 28, 2022

[GCP] support multiple regions when gathering metrics #4197

Merged

11 tasks

endorama requested changes Sep 28, 2022

View reviewed changes

gpop63 added 2 commits September 28, 2022 22:08

fix manifest and changelog version

f1cd1fc

fix sample event metricset name

b58f789

gpop63 force-pushed the add_gcp-redis branch from ad39ca8 to 2cd85c9 Compare September 28, 2022 19:10

add suffixes units and metric types

161f7dd

gpop63 force-pushed the add_gcp-redis branch from 2cd85c9 to 161f7dd Compare September 28, 2022 19:19

Merge remote-tracking branch 'upstream/main' into add_gcp-redis

60944a4

gpop63 requested a review from endorama September 28, 2022 19:24

endorama added the Integration:gcp Google Cloud Platform label Sep 29, 2022

endorama mentioned this pull request Sep 29, 2022

carbonblack_edr,checkpoint,cisco_umbrella,gcp,zeek: remove duplicate fields and fix types #4339

Merged

4 tasks

endorama approved these changes Sep 29, 2022

View reviewed changes

gpop63 and others added 2 commits September 29, 2022 16:06

update docs

9177979

Merge branch 'main' into add_gcp-redis

9b12bab

gpop63 and others added 3 commits October 5, 2022 13:24

Merge remote-tracking branch 'upstream/main' into add_gcp-redis

5f7fc98

fix sample event field name

a06acd1

Merge branch 'main' into add_gcp-redis

19e4ee4

endorama merged commit 2891ec1 into elastic:main Oct 5, 2022

efd6 reviewed Oct 6, 2022

View reviewed changes

andrewkroh mentioned this pull request Oct 20, 2022

[production] Promote many packages from snapshot elastic/package-storage#5929

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[GCP] Add redis data stream #4325

[GCP] Add redis data stream #4325

Uh oh!

gpop63 commented Sep 27, 2022

Uh oh!

elasticmachine commented Sep 27, 2022 •

edited

Loading

Uh oh!

elasticmachine commented Sep 27, 2022 •

edited

Loading

Build stats

Test stats 🧪

Uh oh!

elasticmachine commented Sep 27, 2022 •

edited

Loading

Uh oh!

endorama Sep 28, 2022

Uh oh!

endorama Sep 28, 2022

Uh oh!

endorama Sep 28, 2022

Uh oh!

endorama Sep 28, 2022

Uh oh!

endorama commented Sep 29, 2022

Uh oh!

endorama left a comment

Uh oh!

endorama commented Oct 5, 2022

Uh oh!

gpop63 commented Oct 5, 2022

Uh oh!

efd6 Oct 6, 2022 •

edited

Loading

Uh oh!

Uh oh!

		@@ -1,4 +1,9 @@
		# newer versions go on top
		- version: "2.12.6"

[GCP] Add redis data stream #4325

[GCP] Add redis data stream #4325

Uh oh!

Conversation

gpop63 commented Sep 27, 2022

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Package gcp 👍(4) 💚(0) 💔(1)

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama commented Sep 29, 2022

Uh oh!

endorama left a comment

Choose a reason for hiding this comment

Uh oh!

endorama commented Oct 5, 2022

Uh oh!

gpop63 commented Oct 5, 2022

Uh oh!

efd6 Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticmachine commented Sep 27, 2022 •

edited

Loading

Package `gcp` 👍(4) 💚(0) 💔(1)

elasticmachine commented Sep 27, 2022 •

edited

Loading

elasticmachine commented Sep 27, 2022 •

edited

Loading

efd6 Oct 6, 2022 •

edited

Loading