Skip to content

[format] Support Map<X, Blob> type for Blob File Format#8635

Open
steFaiz wants to merge 4 commits into
apache:masterfrom
steFaiz:map_blob
Open

[format] Support Map<X, Blob> type for Blob File Format#8635
steFaiz wants to merge 4 commits into
apache:masterfrom
steFaiz:map_blob

Conversation

@steFaiz

@steFaiz steFaiz commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Purpose

Supports writing Map<X, Blob> in BlobFormatTable, following #8181

This PR only focus on Blob File Format of Java and Python module, keeping this PR reasonably small.
This PR DO NOT involes:

  1. end-to-end support of Map<X, Blob> Type
  2. Spark/Flink MergeInto support

Key Changes:

  1. extract a new BlobElementSerializer to deal with blob elements: raw, array and map
  2. implement BlobElementSerializer for Map<X, Blob>, where X only supports some simple non-nested type: String and Int family.

Tests

See unit tests

@steFaiz
steFaiz marked this pull request as draft July 15, 2026 03:28
@steFaiz steFaiz changed the title [core] Support Map<X, Blob> type for Blob File Format [format] Support Map<X, Blob> type for Blob File Format Jul 15, 2026
@steFaiz
steFaiz marked this pull request as ready for review July 15, 2026 04:05
@JingsongLi

Copy link
Copy Markdown
Contributor

cc @leaves12138 to take a look.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also support this in PrimaryKeyBlobExternalizer?

@steFaiz

steFaiz commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@JingsongLi Of course! But this PR only focus on file format layer modification. Wiring to Table & Engine layer will be brought in subsequent PRs.

I'm afraid that modifying all layers in one PR is too big to review.

@JingsongLi

JingsongLi commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

@JingsongLi Of course! But this PR only focus on file format layer modification. Wiring to Table & Engine layer will be brought in subsequent PRs.

I'm afraid that modifying all layers in one PR is too big to review.

@steFaiz It is OK in a single PR. It’s important to have a complete implementation; if we run into difficulties during the review, we’ll split the PR then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants