Lambda Go SDK: upload large inputProps to S3 - #8987
Conversation
- Return the serialization error from constructRenderInternals instead of
calling log.Fatal, so callers stay in control of process exit.
- Wrap invocation, marshalling and response-parsing errors with %w and
include the function name for context.
- Type the internal Crf, ColorSpace and GopSize fields as pointers instead
of interface{} so an unset value serializes to null without a typed
zero-value.
- Fix the example progress output that printed RenderId under the
"bucketName" label and dropped the redundant print.
- Remove the stray semicolon and add the missing trailing newline in
version.go.
Previously the Go SDK rejected inputProps that exceeded the inline Lambda payload limit. Match the behavior of the JS and PHP clients by uploading oversized props to S3 and passing a bucket-url reference instead. - Add s3.go with the bucket resolution and upload helpers: getOrCreateBucket (list remotionlambda- buckets in the region, error on multiple, create one otherwise), SHA256 object hashing, the remotionlambda-<region>-<hash> bucket naming, the input-props/<hash>.json key and a private application/json PutObject. - Rewrite serializeInputProps to return a SerializedInputProps of type "payload" when the props fit inline, or "bucket-url" with the hash and bucket name after an upload. The inline threshold uses the same margin as the reference clients (200KB video, 5MB still). - Honor forceBucketName as the user-specified bucket and forcePathStyle on the S3 client. - Validate the render options before serializing so an invalid request never triggers an S3 upload or bucket creation. - Add tests for inline serialization, the upload threshold and the bucket and key naming.
There was a problem hiding this comment.
Caution
The new S3 upload path will fail on AWS accounts using the default S3 Object Ownership setting (ACLs disabled), because the bucket is created without enabling ACLs and the object upload requests a private ACL.
Reviewed changes — added S3 bucket discovery/creation and oversized inputProps upload to the Go Lambda SDK, plus some error-handling cleanup.
- Add
s3.gowith bucket discovery, creation, and private object upload helpers. - Rewrite
serializeInputPropsto return aSerializedInputPropspayload and upload to S3 when the inline limit is exceeded. - Move inputProps serialization after validation and remove the previous
log.Fatalon serialization errors. - Clean up pointer serialization for
Crf,ColorSpace, andGopSize, and fix theversion.gopackage statement. - Fix the example output in
lambda-go-example/main.goso it printsBucketNameandRenderId.
Note: 1 inline comment(s) dropped because they did not anchor to lines inside the PR diff:
packages/lambda-go/lambda_go_sdk_test.go:22(RIGHT) — line 22 (RIGHT) is not inside a diff hunk
Kimi K2 (free via Pullfrog for OSS) | 𝕏
There was a problem hiding this comment.
ℹ️ No new issues in the latest commits — the delta only resolves merge-time version/dependency drift.
Changes since last review — reviewed the merge commit from 06ba5fd to f32ea1e.
- Version alignment: package.json and go.mod versions were bumped from
4.0.486to4.0.487, matchingversion.go. - Dependency refresh:
go.mod/go.sumupdated toaws-sdk-go v1.55.8, Go directive1.26, toolchaingo1.26.5, and refreshed indirect dependencies. - Source code: the feature code is unchanged;
go test,go vet, andgofmtstill pass.
Note: the feedback from the previous review still needs to be addressed.
Kimi K2 (free via Pullfrog for OSS) | 𝕏
Make private uploads compatible with ACL-disabled buckets, propagate bucket location errors, align payload thresholds and warning rounding, preserve the exported PayloadData name, use unbiased bucket suffixes, and cover upload boundaries.
Add focused coverage for ACL-free uploads, bucket-location error handling, warning rounding, the PayloadData compatibility alias, wrapped upload failures, and random hash formatting.
Replace AWS scalar pointer helpers in the new S3 code with Go 1.26 new expressions.
45daaf4 to
5a848e2
Compare
Replace the end-of-support AWS SDK for Go v1 with supported v2 Lambda, S3, config, and Smithy modules. Preserve shared configuration, region, path-style S3 behavior, and service error handling.
Replace context.TODO with context.Background for AWS configuration and Lambda invocation calls where the existing public API provides no caller context.
# Conflicts: # packages/lambda-go/internalops.go # packages/lambda-go/version.go

Stacked on #8986 (merge that first). Until it lands, this PR's diff also shows the cleanup commit; it will collapse to just the S3 changes once #8986 is merged.
Motivation
The Go SDK currently rejects
inputPropsthat exceed the inline Lambda payload limit, while the JS and PHP clients transparently upload oversized props to S3 and pass a reference. This brings the Go SDK in line with them.Changes
s3.gowith the bucket and upload helpers:getOrCreateBucket: listsremotionlambda-buckets in the region, errors if more than one exists (linking to the multiple-buckets docs), otherwise reuses the single bucket or creates one (with aLocationConstraintoutsideus-east-1).remotionlambda-<region>-<hash>bucket naming, theinput-props/<hash>.jsonkey, and a privateapplication/jsonPutObject.serializeInputPropsto return aSerializedInputPropsof typepayloadwhen the props fit inline, orbucket-urlwith the hash and bucket name after an upload. The inline threshold uses the same margin as the reference clients (200KB video, 5MB still).forceBucketNameas the user-specified bucket andforcePathStyleon the S3 client.Testing
go build ./...,go vet ./...,go test ./...andgofmt -lpass. The S3 upload path itself requires live AWS credentials and was not exercised in CI.