Skip to content

[benchmark] Add DataFusion TPC-DS benchmark harness#542

Merged
JingsongLi merged 3 commits into
apache:mainfrom
JingsongLi:codex/tpcds-benchmark
Jul 19, 2026
Merged

[benchmark] Add DataFusion TPC-DS benchmark harness#542
JingsongLi merged 3 commits into
apache:mainfrom
JingsongLi:codex/tpcds-benchmark

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

What changed

  • Add a non-published benchmarks/tpcds workspace crate with load and run commands.
  • Load external TPC-DS-generated Parquet data into append-only Paimon tables.
  • Run the same external query files against Paimon or native Parquet through DataFusion.
  • Record planning, execution, row-count, spill, and available physical metrics in a versioned JSON report.
  • Add configurable SessionConfig and RuntimeEnv constructors to SQLContext for benchmark memory, parallelism, and spill settings.
  • Document SF10/SF100/SF1000 operation, cache protocol, and result interpretation.

Why

The DataFusion integration has focused functional tests but no reproducible macro benchmark for analytical joins, aggregations, sorts, and spill behavior. This harness provides an end-to-end comparison using the same binary, runtime settings, and external query files for both sources.

The repository does not include TPC tools, generated data, or query text. Reports are explicitly labeled as TPC-DS-derived non-TPC results.

Validation

  • cargo build -p paimon-tpcds-bench
  • cargo test -p paimon-tpcds-bench — 7 unit tests and 4 smoke tests passed
  • cargo test -p paimon-datafusion --test sql_context_tests — 48 tests passed
  • cargo clippy -p paimon-tpcds-bench -p paimon-datafusion --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

@JingsongLi
JingsongLi marked this pull request as ready for review July 19, 2026 01:01

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest changes locally. The benchmark tests, targeted SQLContext test, formatting, clippy checks, and the real Parquet load/overwrite smoke verification all passed. The remaining TPC-DS SQL compatibility work is intentionally handled in a follow-up PR.

@JingsongLi
JingsongLi merged commit 054ce25 into apache:main Jul 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants