fix(core): splitting tree.json produces a lot of files #34718
Conversation
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
| } | ||
|
|
||
| for (const forestFile of forestFiles) { | ||
| fs.writeFileSync(path.join(this.outdir, forestFile.fileName), JSON.stringify(forestFile.file), { encoding: 'utf-8' }); |
There was a problem hiding this comment.
Can we write the files in parallel?
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
|
Comments on closed issues and PRs are hard for our team to see. |
In #34478, we made sure that
tree.jsonwould not exceed 512MB, by splitting parts of the construct tree to separate files.The method we chose led to a lot of files.
In this PR we are combining the various small subtrees into larger "forest" files. Since there are no official readers for the new file format yet, we're not losing a lot of effort by changing the format.
The protocol is that if
treeIdis set in a subtree reference, the target file is a forest file, otherwise it's a tree file.Closes #34627.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license