[GCP] Add GCP Load Balancer Logs Datastream #3493

legoguy1000 · 2022-06-08T23:42:43Z

What does this PR do?

Create a new data stream for GCP Load Balancers logs

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Screenshots

elasticmachine · 2022-06-08T23:45:33Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-08-16T08:50:42.366+0000
Duration: 22 min 11 sec

Test stats 🧪

Test	Results
Failed	0
Passed	74
Skipped	0
Total	74

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

legoguy1000 · 2022-07-11T01:32:35Z

@efd6 can u run tests if u get a chance.

efd6 · 2022-07-11T01:34:08Z

/test

packages/gcp/data_stream/load_balancer/fields/fields.yml

efd6 · 2022-07-12T01:13:27Z

/test

legoguy1000 · 2022-07-12T17:22:43Z

ok, should be fixed

efd6 · 2022-07-12T22:43:47Z

/test

elasticmachine · 2022-07-12T23:03:30Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`5/5`)	💚
Files	100.0% (`5/5`)	💚 2.879
Classes	100.0% (`5/5`)	💚 2.879
Methods	94.186% (`81/86`)	👍 4.128
Lines	95.752% (`1375/1436`)	👍 3.925
Conditionals	100.0% (`0/0`)	💚

packages/gcp/_dev/build/docs/load_balancer.md

packages/gcp/changelog.yml

packages/gcp/manifest.yml

packages/gcp/data_stream/load_balancer/manifest.yml

packages/gcp/data_stream/load_balancer/elasticsearch/ingest_pipeline/default.yml

packages/gcp/changelog.yml

legoguy1000 · 2022-07-19T22:03:13Z

I will update this PR once the LB Metrics datastream is merged

legoguy1000 · 2022-08-09T23:49:22Z

@endorama Rebased and ready for ur review

endorama · 2022-08-12T12:39:21Z

/test

packages/gcp/data_stream/loadbalancing_logs/fields/base-fields.yml

packages/gcp/data_stream/loadbalancing_logs/sample_event.json

packages/gcp/data_stream/loadbalancing_logs/_dev/test/system/test-pubsub-config.yml

legoguy1000 · 2022-08-12T14:14:06Z

I'll fix these.

legoguy1000 · 2022-08-16T00:33:47Z

@endorama Ready when u are

endorama · 2022-08-16T08:50:06Z

/test

packages/gcp/_dev/build/docs/loadbalancing.md

...s/gcp/data_stream/loadbalancing_logs/_dev/test/pipeline/test-load_balancer.log-expected.json

packages/gcp/data_stream/loadbalancing_logs/agent/stream/gcp-pubsub.yml.hbs

sayden

LGTM so let's ask the Grandmaster GCP for his approval

@endorama ? 😬

legoguy1000 · 2022-08-23T03:56:21Z

@endorama any reason this can't be merged

This reverts commit efe5ae5. Bump to 2.7.0 was correct so this PR can be merged after elastic#3493

This reverts commit 41c69fb. Bump to 2.7.0 was correct so this PR can be merged after elastic#3493

endorama · 2022-08-23T08:48:28Z

I'm sorry this slipped through. LGTM

* add storage data stream * fix sample event * fix storage docs * target latest changelog version * bump to 2.7.0 * Update manifest.yml * Update changelog.yml * Revert "Update manifest.yml" This reverts commit efe5ae5. Bump to 2.7.0 was correct so this PR can be merged after #3493 * Revert "Update changelog.yml" This reverts commit 41c69fb. Bump to 2.7.0 was correct so this PR can be merged after #3493 * fix broken merge * Fix typo in metric name `authz.acl_based_object_access_.ount` -> `authz.acl_based_object_access_.count` * update docs with fixed metric name * remove underscore in field Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Tom Myers <[email protected]>

legoguy1000 marked this pull request as ready for review June 8, 2022 23:43

legoguy1000 requested review from a team as code owners June 8, 2022 23:43

legoguy1000 force-pushed the gcp-load-balancers branch from 445cfe5 to efe8068 Compare July 11, 2022 01:31

efd6 reviewed Jul 11, 2022

View reviewed changes

packages/gcp/data_stream/load_balancer/fields/fields.yml Outdated Show resolved Hide resolved

legoguy1000 requested a review from efd6 July 12, 2022 00:34

endorama reviewed Jul 19, 2022

View reviewed changes

legoguy1000 added 8 commits July 19, 2022 22:12

Add GCP Load Balancer Datastream

aef6d7d

update changelog

c1619ae

add additional fields, fix tests

19b9832

fix sample event

f94ebbe

fix formatting

b7b276d

rebuild docs

b37e536

change to Load Balancing

6ae406b

rebase

2fc443c

legoguy1000 force-pushed the gcp-load-balancers branch from 6055e52 to 2fc443c Compare July 19, 2022 22:15

legoguy1000 changed the title ~~[GCP] Add GCP Load Balancer Datastream~~ [GCP] Add GCP Load Balancer Logs Datastream Jul 20, 2022

Merge branch 'main' into gcp-load-balancers

2d17789

legoguy1000 requested a review from endorama July 25, 2022 16:27

Merge branch 'main' into gcp-load-balancers

e0953af

endorama added Integration:gcp Google Cloud Platform Team:Cloud-Monitoring Label for the Cloud Monitoring team labels Aug 1, 2022

legoguy1000 added 3 commits August 9, 2022 23:43

Merge remote-tracking branch 'upstream/main' into gcp-load-balancers

c634df6

update docs

b7c9b43

update manifest

b49ce00

endorama reviewed Aug 12, 2022

View reviewed changes

packages/gcp/data_stream/loadbalancing_logs/fields/base-fields.yml Outdated Show resolved Hide resolved

endorama reviewed Aug 12, 2022

View reviewed changes

packages/gcp/data_stream/loadbalancing_logs/sample_event.json Outdated Show resolved Hide resolved

endorama reviewed Aug 12, 2022

View reviewed changes

packages/gcp/data_stream/loadbalancing_logs/_dev/test/system/test-pubsub-config.yml Outdated Show resolved Hide resolved

resolve comments

cf4bf7d

legoguy1000 requested a review from endorama August 12, 2022 14:39

sayden reviewed Aug 16, 2022

View reviewed changes

packages/gcp/_dev/build/docs/loadbalancing.md Show resolved Hide resolved

sayden reviewed Aug 16, 2022

View reviewed changes

...s/gcp/data_stream/loadbalancing_logs/_dev/test/pipeline/test-load_balancer.log-expected.json Show resolved Hide resolved

sayden reviewed Aug 16, 2022

View reviewed changes

packages/gcp/data_stream/loadbalancing_logs/agent/stream/gcp-pubsub.yml.hbs Show resolved Hide resolved

sayden reviewed Aug 16, 2022

View reviewed changes

endorama added a commit to gpop63/integrations that referenced this pull request Aug 23, 2022

Revert "Update manifest.yml"

c6ace1e

This reverts commit efe5ae5. Bump to 2.7.0 was correct so this PR can be merged after elastic#3493

endorama added a commit to gpop63/integrations that referenced this pull request Aug 23, 2022

Revert "Update changelog.yml"

bcf733d

This reverts commit 41c69fb. Bump to 2.7.0 was correct so this PR can be merged after elastic#3493

endorama approved these changes Aug 23, 2022

View reviewed changes

endorama merged commit 660e504 into elastic:main Aug 23, 2022

andrewkroh mentioned this pull request Aug 25, 2022

[production] Promote many packages from snapshot elastic/package-storage#5402

Merged

[GCP] Add GCP Load Balancer Logs Datastream #3493

[GCP] Add GCP Load Balancer Logs Datastream #3493

Uh oh!

Conversation

legoguy1000 commented Jun 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Jun 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

legoguy1000 commented Jul 11, 2022

Uh oh!

efd6 commented Jul 11, 2022

Uh oh!

Uh oh!

efd6 commented Jul 12, 2022

Uh oh!

legoguy1000 commented Jul 12, 2022

Uh oh!

efd6 commented Jul 12, 2022

Uh oh!

elasticmachine commented Jul 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

legoguy1000 commented Jul 19, 2022

Uh oh!

legoguy1000 commented Aug 9, 2022

Uh oh!

endorama commented Aug 12, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

legoguy1000 commented Aug 12, 2022

Uh oh!

legoguy1000 commented Aug 16, 2022

Uh oh!

endorama commented Aug 16, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sayden left a comment

Choose a reason for hiding this comment

Uh oh!

legoguy1000 commented Aug 23, 2022

Uh oh!

endorama commented Aug 23, 2022

Uh oh!

Uh oh!

legoguy1000 commented Jun 8, 2022 •

edited

Loading

elasticmachine commented Jun 8, 2022 •

edited

Loading

elasticmachine commented Jul 12, 2022 •

edited

Loading