Skip to content

Conversation

@shahzad31
Copy link
Contributor

@shahzad31 shahzad31 commented Sep 16, 2025

Summary

Clean up extra synthetics package policies created via #235021 !!

Clean up

There are three basic tests for clean up

  1. SpaceIds being added in a processor where there should only be one
  2. Multiple package policies exists for same configId and locationId
  3. ConfigId is missing altogether from monitors for a package policy

The bug meant multiple spaceIds were being added in processor and also if multiple package policies exists for a locationId and configId, that also passes the test, so we clean up those policies.

Testing

Revert changes in PR #235021, it will create extra package policies once you have monitors in multiple places, switching to this branch should clean up those.

@github-actions github-actions bot added the author:obs-ux-management PRs authored by the obs ux management team label Sep 16, 2025
@shahzad31 shahzad31 force-pushed the clean-up-package-policies branch from ada5a86 to 46acf7d Compare September 18, 2025 11:07
encryptedSavedObjects: EncryptedSavedObjectsPluginStart;
}) {
const { privateLocationAPI } = this.syntheticsMonitorClient;
const privateConfigs: Array<{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shahzad31 As far as I understand you fixed synthetics policies being created in multiple places in this PR. With this change you undo the fix. Could it be that you pushed this change by mistake in order to create a broken environment with duplicated policies?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reapplied the fix in this commit. This change fixes Synthetics API Tests SyncGlobalParamsSpaces number of package policies matches number of monitors failing tests. Will look into the rest ones

@mgiota
Copy link
Contributor

mgiota commented Sep 23, 2025

@elasticmachine merge upstream

@shahzad31 shahzad31 marked this pull request as ready for review September 24, 2025 08:11
@shahzad31 shahzad31 requested review from a team as code owners September 24, 2025 08:11
@shahzad31 shahzad31 added backport:version Backport to applied version labels v8.19.5 v9.1.5 labels Sep 24, 2025
Copy link
Contributor

@csr csr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AppEx QA codeowner changes LGTM 👍

@botelastic botelastic bot added Team:Fleet Team label for Observability Data Collection Fleet team Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. labels Sep 24, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@mgiota
Copy link
Contributor

mgiota commented Sep 24, 2025

@shahzad31 I tested it and here are the results:

Before

default_space

After

Screenshot 2025-09-24 at 14 13 14

I confirm that the extra policy was successfully deleted. Is it ok though that only one of the automatically generated BROWSER_SYNTHETICS_TEST_NOW_RUN policies was deleted?

Another observation I have is that the 2nd space (test) doesn't list any policies under Fleet > Agent policies (this is also on main, nothing to do with the clean up task)

Screenshot 2025-09-24 at 13 56 36

user?: AuthenticatedUser;
skipUnassignFromAgentPolicies?: boolean;
force?: boolean;
spaceIds?: string[];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if we could use the same pattern as getByIds

 ids: Array<
      | string
      | {
          id: string;
          /**
           * The space id associated with the agent policy. When using an un-scoped SO client, the
           * space id can be set to `*` which will look for that agent policy across all spaces
           */
          spaceId?: string;
        }
    >,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i updated types

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this ask is in reference to the getByIds definition on AgentPolicyService
https://github.com/elastic/kibana/blob/main/x-pack/platform/plugins/shared/fleet/server/services/agent_policy.ts#L719-L726

That said, the PackagePolicyClientImpl.getByIDs follows a different pattern, which this PR adopts, using the WithSpaceIdsOption interface, which is also used in other parts of this client.

@shahzad31 shahzad31 force-pushed the clean-up-package-policies branch from 65d2cc4 to 03955be Compare September 25, 2025 19:04
@shahzad31 shahzad31 requested a review from nchaulet September 25, 2025 19:58
@elasticmachine
Copy link
Contributor

💔 Build Failed

Failed CI Steps

History

@dominiqueclarke dominiqueclarke self-requested a review September 29, 2025 23:21
@dominiqueclarke dominiqueclarke self-requested a review September 30, 2025 01:22
Copy link
Contributor

@dominiqueclarke dominiqueclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚢

Copy link
Contributor

@MichelLosier MichelLosier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

With regards to this ask: #235200 (review)

Looks like the changes in this PR are consistent with other parts of the client, if we wanted to adopt later the pattern that was suggested, the changes here would not prevent that. So not a blocker for ✅

@dominiqueclarke dominiqueclarke merged commit ced929a into elastic:main Sep 30, 2025
12 checks passed
@dominiqueclarke dominiqueclarke deleted the clean-up-package-policies branch September 30, 2025 19:46
@kibanamachine
Copy link
Contributor

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.19 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.19:
- [UII] Supporting input-level deployment_modes for integration policies (#226086)
9.1 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 235200

Questions ?

Please refer to the Backport tool documentation

@dominiqueclarke
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
9.1
8.19 An unhandled error occurred. Please see the logs for details

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 235200

Questions ?

Please refer to the Backport tool documentation

dominiqueclarke pushed a commit to dominiqueclarke/kibana that referenced this pull request Sep 30, 2025
…35200)

## Summary

Clean up extra synthetics package policies created via
elastic#235021 !!

### Clean up

There are three basic tests for clean up

1. SpaceIds being added in a processor where there should only be one
2. Multiple package policies exists for same configId and locationId
3. ConfigId is missing altogether from monitors for a package policy

The bug meant multiple spaceIds were being added in processor and also
if multiple package policies exists for a locationId and configId, that
also passes the test, so we clean up those policies.

### Testing

Revert changes in PR elastic#235021, it
will create extra package policies once you have monitors in multiple
places, switching to this branch should clean up those.

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit ced929a)

# Conflicts:
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.test.ts
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.ts
#	x-pack/solutions/observability/test/api_integration_deployment_agnostic/services/synthetics_private_location.ts
dominiqueclarke pushed a commit to dominiqueclarke/kibana that referenced this pull request Oct 1, 2025
…35200)

## Summary

Clean up extra synthetics package policies created via
elastic#235021 !!

### Clean up

There are three basic tests for clean up

1. SpaceIds being added in a processor where there should only be one
2. Multiple package policies exists for same configId and locationId
3. ConfigId is missing altogether from monitors for a package policy

The bug meant multiple spaceIds were being added in processor and also
if multiple package policies exists for a locationId and configId, that
also passes the test, so we clean up those policies.

### Testing

Revert changes in PR elastic#235021, it
will create extra package policies once you have monitors in multiple
places, switching to this branch should clean up those.

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit ced929a)

# Conflicts:
#	x-pack/platform/plugins/shared/fleet/server/services/package_policy.ts
#	x-pack/platform/plugins/shared/fleet/server/services/package_policy_service.ts
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.test.ts
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.ts
#	x-pack/solutions/observability/test/api_integration_deployment_agnostic/apis/synthetics/index.ts
#	x-pack/solutions/observability/test/api_integration_deployment_agnostic/services/synthetics_private_location.ts
@dominiqueclarke
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@shahzad31
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

shahzad31 added a commit to shahzad31/kibana that referenced this pull request Oct 1, 2025
…35200)

## Summary

Clean up extra synthetics package policies created via
elastic#235021 !!

### Clean up

There are three basic tests for clean up

1. SpaceIds being added in a processor where there should only be one
2. Multiple package policies exists for same configId and locationId
3. ConfigId is missing altogether from monitors for a package policy

The bug meant multiple spaceIds were being added in processor and also
if multiple package policies exists for a locationId and configId, that
also passes the test, so we clean up those policies.

### Testing

Revert changes in PR elastic#235021, it
will create extra package policies once you have monitors in multiple
places, switching to this branch should clean up those.

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit ced929a)

# Conflicts:
#	x-pack/platform/plugins/shared/fleet/server/services/package_policy.ts
#	x-pack/platform/plugins/shared/fleet/server/services/package_policy_service.ts
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.test.ts
#	x-pack/solutions/observability/plugins/synthetics/server/tasks/sync_private_locations_monitors_task.ts
#	x-pack/solutions/observability/test/api_integration_deployment_agnostic/apis/synthetics/index.ts
#	x-pack/solutions/observability/test/api_integration_deployment_agnostic/services/synthetics_private_location.ts
dominiqueclarke added a commit that referenced this pull request Oct 1, 2025
…5200) (#237032)

# Backport

This will backport the following commits from `main` to `9.1`:
- [[Synthetics] Clean up extra synthetics package policies !!
(#235200)](#235200)

<!--- Backport version: 10.0.2 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"Shahzad","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-30T19:46:07Z","message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Fleet","Team:obs-ux-management","backport:version","author:obs-ux-management","v9.2.0","v8.19.5","v9.1.5"],"title":"[Synthetics]
Clean up extra synthetics package policies
!!","number":235200,"url":"https://github.com/elastic/kibana/pull/235200","mergeCommit":{"message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/235200","number":235200,"mergeCommit":{"message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: Shahzad <[email protected]>
dominiqueclarke added a commit that referenced this pull request Oct 1, 2025
…35200) (#237050)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Synthetics] Clean up extra synthetics package policies !!
(#235200)](#235200)

<!--- Backport version: 10.0.2 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"Shahzad","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-30T19:46:07Z","message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Fleet","Team:obs-ux-management","backport:version","author:obs-ux-management","v9.2.0","v8.19.5","v9.1.5"],"title":"[Synthetics]
Clean up extra synthetics package policies
!!","number":235200,"url":"https://github.com/elastic/kibana/pull/235200","mergeCommit":{"message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/235200","number":235200,"mergeCommit":{"message":"[Synthetics]
Clean up extra synthetics package policies !! (#235200)\n\n##
Summary\n\nClean up extra synthetics package policies created
via\nhttps://github.com//pull/235021 !!\n\n### Clean
up\n\nThere are three basic tests for clean up\n\n1. SpaceIds being
added in a processor where there should only be one \n2. Multiple
package policies exists for same configId and locationId\n3. ConfigId is
missing altogether from monitors for a package policy \n\nThe bug meant
multiple spaceIds were being added in processor and also\nif multiple
package policies exists for a locationId and configId, that\nalso passes
the test, so we clean up those policies.\n\n\n### Testing \n\nRevert
changes in PR #235021, it\nwill
create extra package policies once you have monitors in
multiple\nplaces, switching to this branch should clean up
those.\n\n---------\n\nCo-authored-by: kibanamachine
<[email protected]>","sha":"ced929a22bff204846cad448d3603f517cdd240f"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/237032","number":237032,"state":"OPEN"}]}]
BACKPORT-->

---------

Co-authored-by: Shahzad <[email protected]>
rylnd pushed a commit to rylnd/kibana that referenced this pull request Oct 17, 2025
…35200)

## Summary

Clean up extra synthetics package policies created via
elastic#235021 !!

### Clean up

There are three basic tests for clean up

1. SpaceIds being added in a processor where there should only be one 
2. Multiple package policies exists for same configId and locationId
3. ConfigId is missing altogether from monitors for a package policy 

The bug meant multiple spaceIds were being added in processor and also
if multiple package policies exists for a locationId and configId, that
also passes the test, so we clean up those policies.


### Testing 

Revert changes in PR elastic#235021, it
will create extra package policies once you have monitors in multiple
places, switching to this branch should clean up those.

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author:obs-ux-management PRs authored by the obs ux management team backport:version Backport to applied version labels release_note:fix Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. Team:Fleet Team label for Observability Data Collection Fleet team v8.19.5 v9.1.5 v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants