Support refreshable custom gRPC metadata for remotes

[iamthebot]

We run a Dolt remotesapi endpoint behind an identity-aware proxy. Both layers require authentication:

• The proxy expects a short-lived bearer token in Proxy-Authorization.
• Dolt expects its normal credentials in Authorization.

Native gRPC over HTTPS works here. The problem is that the Dolt client has no supported way to attach the second header while preserving its own authentication.

Our current workaround is a local HTTP/2 proxy that obtains and refreshes the identity token, injects Proxy-Authorization, and forwards the gRPC stream unchanged. It works, but adds another process and more operational complexity.

Would the maintainers be open to a credential-helper mechanism for additional gRPC metadata? For example, Dolt could invoke a configured command that returns headers and an expiration time, then refresh them when necessary. Dolt would continue managing its own Authorization header.

Important properties would be:

• Scoped to a specific remote
• Works for unary and streaming RPCs
• Supports expiring credentials
• Does not persist or log returned values
• Cannot accidentally override Dolt's own authentication

A static --header option would help some cases, but would not be sufficient for short-lived tokens.

I'm happy to draft a PR once we agree on the configuration and credential-provider shape. Is PerRPCCredentials the right extension point, or is there an existing mechanism I've missed?

I don't want to just throw up some AI slop without checking first :)

[timsehn]

Thanks for the issue. This is an @reltuk or @coffeegoddd question.

We're certainly not opposed to AI-generated PRs but there is some UI design that we should probably discuss on our end before reviewing code.

[iamthebot]

but there is some UI design that we should probably discuss on our end before reviewing code.

Yep, that's what I wanted to hash out. What're the main considerations on your end re: how to expose this?

[reltuk]

Hi @iamthebot, thanks for the great feature request.

The two credential helpers that I have a bit of familiarity with are docker and bazel. I think something like Bazel's approach might meet these requirements well. We can register an external binary that gets to see the outbound URI, trimmed to scheme + host + port, and returns http headers + expiration time. Then those headers get added to every outbound request matching that schema+host+port on that remote interaction.

Broadly the shape of that solution looks totally workable to me.

For how to best configure it, I have less immediate answers. Some quick overviews:

• Dolt's config hierarchy (local and global, $HOME/.dolt/config_global.json) is currently just a string->string|bool map, so it doesn't currently support structured values.
• Dolt's per-remote config might be a natural place to put it. Currently different remote types can carry some per-remote parameters which are threaded to the places where this would be needed.
• Global config for a single global credential helper could also maybe work well enough. Multiplexing at the layer of the credential helper itself might be sufficient.

One concern with making the config per-remote is any potential security implications of dolt remote add ..., or more likely call dolt_remote("add", "...", "...") letting a user configure Dolt to start calling a local binary.

A concern with a global or per-remote setting is allowing a user who can add a dolt remote to forward along the output of an existing credential helper. As long as that credential helper is configured to only respond for appropriate destinations and/or to scope resulting credentials for the requested destinations, things are fine. But if the credential helper is fast-and-loose with matching the supplied URL, and it mints non-attenuated credentials, then that is bad times.

Another question is which remote types are subject to this interception. I think it makes sense to limit this to remotesapi interactions – so no local process interactions for other types of remotes which happen to use HTTP/S as a transport (GCS, AWS, OSS, some Git interactions, etc.).

Anyway, those are my immediate thoughts. I would love your feedback on them :).

[iamthebot]

Thanks! That makes sense. The Bazel model seems like a good inspiration here we can use. Thinking about the config question, MySQL generally treats auth as server scoped, with db auth handled separately after connecting. So I guess for Dolt, we'd do something like:

• A global helper handles auth to the proxy for the remotesapi endpoint.
• Dolt’s existing credentials handle the remote user.
• Dolt handles db permissions normally after that.

That makes me lean toward a single global helper for the first version. The helper would get the canonical origin and be responsible for only returning credentials for destinations it trusts. In our case, we can just allowlist the exact proxy hosts. This also avoids letting repo or SQL-controlled remote config choose an arbitrary local executable. If we need per-remote selection later, maybe the remote could reference a helper registered globally by name, without defining the executable itself.

I can take a pass at a PR so we have something concrete to discuss/iterate on. Would that be OK?

[reltuk]

I like the suggestion a lot and I think I agree entirely. We would be happy to get a PR which allows a user to configure a global credentials helper which is called on remotesapi interactions and allows the helper to add outbound headers to the resulting HTTP requests.

Please don't hesitate to reach out here or on discord (@aaron) if you have any further questions :).

[iamthebot]

Thanks! Dropped a PR linked above @reltuk that takes the approach we discussed. Diff's a bit big, but half's tests. LMK if you want it broken down further (not sure if you guys use graphite stacks for that).