Skip to content

Can the bundling behavior of gRPC be disabled for pubsub publishers? #1432

@mattnworb

Description

@mattnworb

I've encountered a case where an application that publishes pubsub messages (using google-cloud-pubsub:0.5.1) has encountered OutOfMemoryErrors which I have traced down to an instance of com.google.api.gax.bundling.ThresholdBundler taking up 92% of the heap space. This application purposefully runs with a small heap of ~256mb.

This application publishes all of it's messages to a single topic, so it seems as if all of the requests end up in the same ThresholdBundler instance. In the heap dump that I have, the ThresholdBundler instance has 63984 elements in the closedBundles list.

It looks like the gax library allows for the bundling behavior to be disabled via BundlingSettings$Builder.setIsEnabled(Boolean), but the pubsub layer does not expose any of these options and always leaves the bundling behavior as enabled.

Would it be possible to add flags to PubsubOptions (or wherever is appropriate) to allow for this request batching to be disabled if desired?

Metadata

Metadata

Labels

api: pubsubIssues related to the Pub/Sub API.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions