Skip to content

Fix left outer merge join wrong results with residual filters - #11358

Merged
angelamayxie merged 11 commits into
mainfrom
angela/leftoutermergejoin
Jul 29, 2026
Merged

Fix left outer merge join wrong results with residual filters#11358
angelamayxie merged 11 commits into
mainfrom
angela/leftoutermergejoin

Conversation

@angelamayxie

@angelamayxie angelamayxie commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Tests added in dolthub/go-mysql-server#3655
Fixes #11350

Based on #11351

LaPingvino and others added 4 commits July 28, 2026 01:45
Fixes two defects in mergeJoinKvIter for left outer merge joins whose
key runs contain duplicates and whose join condition carries a residual
(non-merge-key) filter:

1. matchedLeft was overwritten on every pairing (matchedLeft = ok), so a
   later pairing in the same key run that failed the residual filter
   erased an earlier success, and the key-run transition then emitted a
   spurious null-extended row for a left row that had already matched.
   Now matchedLeft is only ever set within a run (matchedLeft || ok);
   the run-transition reset remains the sole reset point.

2. When the left iterator hit io.EOF while advancing past a completed
   left row in the match loop, the error was returned immediately,
   dropping the null-extended row owed to a final unmatched left row.
   Now the null-extended row for tmpKey is emitted before surfacing EOF,
   mirroring the compare loop's oldLeftKey handling.

Both defects surface as wrong LEFT JOIN ... IS NULL anti-join results
(phantom and missing orphans). Regression tests included; the identical
plan executed by the GMS memory engine already returned correct results.

Fixes #11350

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpTxMfzCXHoM8ojJn5aQiv
@coffeegoddd

coffeegoddd commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@angelamayxie DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 2.35 2.35 0.0
groupby_scan 66.84 66.84 0.0
index_join 1.93 1.93 0.0
index_join_scan 1.32 1.32 0.0
index_scan 204.11 200.47 -1.78
oltp_point_select 0.25 0.25 0.0
oltp_read_only 5.09 5.09 0.0
select_random_points 0.53 0.53 0.0
select_random_ranges 0.67 0.67 0.0
table_scan 200.47 200.47 0.0
types_table_scan 467.3 475.79 1.82
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.09 6.21 1.97
oltp_insert 3.13 3.13 0.0
oltp_read_write 11.24 11.24 0.0
oltp_update_index 3.3 3.3 0.0
oltp_update_non_index 2.97 2.97 0.0
oltp_write_only 6.21 6.21 0.0
types_delete_insert 6.79 6.79 0.0

@coffeegoddd

coffeegoddd commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@angelamayxie DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 45.79 46.63 1.83
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 8eb4c2a 52.66 dolt 11bacab 53.03 0.7

@coffeegoddd

Copy link
Copy Markdown
Contributor

@angelamayxie DOLT

comparing_percentages
100.000000 to 100.000000
version result total
db842b5 ok 5937471
version total_tests
db842b5 5937471
correctness_percentage
100.0

@angelamayxie
angelamayxie requested a review from jycor July 29, 2026 00:03
@angelamayxie
angelamayxie marked this pull request as ready for review July 29, 2026 00:04
@coffeegoddd

Copy link
Copy Markdown
Contributor

@angelamayxie DOLT

comparing_percentages
100.000000 to 100.000000
version result total
c35c4fd ok 5937471
version total_tests
c35c4fd 5937471
correctness_percentage
100.0

@coffeegoddd

Copy link
Copy Markdown
Contributor

@angelamayxie DOLT

comparing_percentages
100.000000 to 100.000000
version result total
11bacab ok 5937471
version total_tests
11bacab 5937471
correctness_percentage
100.0

@angelamayxie
angelamayxie merged commit 677e2f7 into main Jul 29, 2026
32 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kvexec: left outer merge join with residual filter emits spurious null-extended rows and drops final unmatched left row (wrong anti-join results)

4 participants