Skip to content

Commit 05afab3

Browse files
authored
fix(pipelines): ensure assets with same hash but different destinations are published separately (#34790)
### Issue Closes #31070 ### Reason for this change Assets are missing to be published in CDK pipelines when stacks with different synthesizers are used for the same account and region. When assets have identical content hashes but need to be published to different destinations (different publishing role ARNs), they were being incorrectly grouped together, causing assets to only be published to one destination instead of all required destinations. ### Description of changes • Modified publishAsset() method in packages/aws-cdk-lib/pipelines/lib/helpers-internal/pipeline-graph.ts • Changed asset tracking key from using only stackAsset.assetId to a composite key: `${stackAsset.assetId}:${stackAsset.assetPublishingRoleArn || 'default'}` • This ensures assets with the same content hash, but different destinations are treated as separate publishing jobs ### Describe any new or updated permissions being added NA ### Description of how you validated changes Checked with the code in #31070 and made sure there are 2 asset stages, locally ran the asset commands and verified that they are being deployed to right buckets: ``` muralikl@b0be83688a18 cdk.out % cdk-assets --path "assembly-pipeline-asset-stack-Staging/pipelineassetstackStagingdevlambdastackEC748226.assets.json" --verbose publish "a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e:current_account-us-east-1" verbose: Loaded manifest from assembly-pipeline-asset-stack-Staging/pipelineassetstackStagingdevlambdastackEC748226.assets.json: 2 assets found verbose: Applied selection: 1 assets selected. info : [0%] start: Publishing LambdaFN/Code (current_account-us-east-1) verbose: [0%] check: Check s3://cdk-dev-assets-123456789012-us-east-1/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip verbose: [0%] build: Zip /Users/muralikl/Downloads/aws-cdk/packages/@aws-cdk-testing/framework-integ/cdk.out/asset.a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e -> assembly-pipeline-asset-stack-Staging/.cache/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip verbose: [0%] upload: Upload s3://cdk-dev-assets-123456789012-us-east-1/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip info : [100%] success: Published LambdaFN/Code (current_account-us-east-1) muralikl@b0be83688a18 cdk.out % cdk-assets --path "assembly-pipeline-asset-stack-Production/pipelineassetstackProductionprdlambdastack4E5ABBC0.assets.json" --verbose publish "a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e:current_account-us-west-2" verbose: Loaded manifest from assembly-pipeline-asset-stack-Production/pipelineassetstackProductionprdlambdastack4E5ABBC0.assets.json: 2 assets found verbose: Applied selection: 1 assets selected. info : [0%] start: Publishing LambdaFN/Code (current_account-us-west-2) verbose: [0%] check: Check s3://cdk-hnb659fds-assets-123456789012-us-west-2/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip verbose: [0%] build: Zip /Users/muralikl/Downloads/aws-cdk/packages/@aws-cdk-testing/framework-integ/cdk.out/asset.a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e -> assembly-pipeline-asset-stack-Production/.cache/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip verbose: [0%] upload: Upload s3://cdk-hnb659fds-assets-123456789012-us-west-2/a26bd817a0dac44954b5caf83f5880a96f831e43b56157224e073b49f236eb4e.zip info : [100%] success: Published LambdaFN/Code (current_account-us-west-2) ``` ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent e400228 commit 05afab3

File tree

45 files changed

+4353
-60
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+4353
-60
lines changed

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-file-system-locations.js.snapshot/PipelinesFileSystemLocations.assets.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-file-system-locations.js.snapshot/PipelinesFileSystemLocations.template.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1768,7 +1768,7 @@
17681768
]
17691769
},
17701770
"Source": {
1771-
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelinesFileSystemLocations-Beta/PipelinesFileSystemLocationsBetaStack18199ECAE.assets.json\\\" --verbose publish \\\"8289faf53c7da377bb2b90615999171adef5e1d8f6b88810e5fef75e6ca09ba5:current_account-current_region\\\"\"\n ]\n }\n }\n}",
1771+
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelinesFileSystemLocations-Beta/PipelinesFileSystemLocationsBetaStack18199ECAE.assets.json\\\" --verbose publish \\\"8289faf53c7da377bb2b90615999171adef5e1d8f6b88810e5fef75e6ca09ba5:current_account-current_region-3d50c90e\\\"\"\n ]\n }\n }\n}",
17721772
"Type": "CODEPIPELINE"
17731773
},
17741774
"VpcConfig": {

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-file-system-locations.js.snapshot/assembly-PipelinesFileSystemLocations-Beta/PipelinesFileSystemLocationsBetaStack18199ECAE.assets.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-file-system-locations.js.snapshot/assembly-PipelinesFileSystemLocations-Beta/manifest.json

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-file-system-locations.js.snapshot/cdkintegcodepipelinewithfilesystemlocationsDefaultTestDeployAssert3E26E748.assets.json

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-vpc.js.snapshot/PipelineStack.assets.json

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-vpc.js.snapshot/PipelineStack.template.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1973,7 +1973,7 @@
19731973
]
19741974
},
19751975
"Source": {
1976-
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelineStack-Beta/PipelineStackBetaStack1E6541489.assets.json\\\" --verbose publish \\\"8289faf53c7da377bb2b90615999171adef5e1d8f6b88810e5fef75e6ca09ba5:current_account-current_region\\\"\"\n ]\n }\n }\n}",
1976+
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelineStack-Beta/PipelineStackBetaStack1E6541489.assets.json\\\" --verbose publish \\\"8289faf53c7da377bb2b90615999171adef5e1d8f6b88810e5fef75e6ca09ba5:current_account-current_region-3d50c90e\\\"\"\n ]\n }\n }\n}",
19771977
"Type": "CODEPIPELINE"
19781978
},
19791979
"VpcConfig": {
@@ -2072,7 +2072,7 @@
20722072
]
20732073
},
20742074
"Source": {
2075-
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelineStack-Beta/PipelineStackBetaStack1E6541489.assets.json\\\" --verbose publish \\\"ac76997971c3f6ddf37120660003f1ced72b4fc58c498dfd99c78fa77e721e0e:current_account-current_region\\\"\"\n ]\n }\n }\n}",
2075+
"BuildSpec": "{\n \"version\": \"0.2\",\n \"phases\": {\n \"install\": {\n \"commands\": [\n \"npm install -g cdk-assets@latest\"\n ]\n },\n \"build\": {\n \"commands\": [\n \"cdk-assets --path \\\"assembly-PipelineStack-Beta/PipelineStackBetaStack1E6541489.assets.json\\\" --verbose publish \\\"ac76997971c3f6ddf37120660003f1ced72b4fc58c498dfd99c78fa77e721e0e:current_account-current_region-8b0ef02d\\\"\"\n ]\n }\n }\n}",
20762076
"Type": "CODEPIPELINE"
20772077
},
20782078
"VpcConfig": {

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-vpc.js.snapshot/assembly-PipelineStack-Beta/PipelineStackBetaStack1E6541489.assets.json

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-vpc.js.snapshot/assembly-PipelineStack-Beta/cdk.out

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-vpc.js.snapshot/assembly-PipelineStack-Beta/manifest.json

Lines changed: 3 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)