Skip to content

Optimize depth pipelines and add upstream Seqair backend - #114

Draft
sstadick wants to merge 2 commits into
masterfrom
perf/seqair-depth
Draft

sstadick wants to merge 2 commits into
masterfrom
perf/seqair-depth

Conversation

@sstadick

@sstadick sstadick commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Merge prerequisites

Do not merge until the long-CIGAR fix lands in upstream Seqair. This remains a draft for review.

  • Seqair #95 is merged upstream.
  • Replace the temporary Git pins for both Seqair crates with published versions containing the fix and refresh Cargo.lock before the Perbase release; Git-only dependencies cannot be published to crates.io.
  • Rerun both feature configurations and the long-CIGAR regression against that published dependency.
  • Complete human review of the code and tests.

Scope

Bring the useful application-level results from the Goose/Rust experiments into Perbase. The candidate temporarily pins Seqair and seqair-types to the long-CIGAR fix, not our experimental performance branch. HTSlib remains the default. Build with --features seqair and select --seqair on either depth command to opt in. The release workflow now includes the feature in downloadable binaries, uses Rust 1.98.1, and checks that both depth commands expose --seqair before uploading.

  • Both backends format output in counting workers and send bounded 64 KiB blocks in region order. Readers and scratch allocations are reused. A small -C budget constrains queues rather than reducing counting to one worker.
  • Base-depth reuses same-name groups, uses singleton/pair shortcuts, and preserves stable tie handling for larger groups. Seqair also bypasses mate grouping when read spans prove a name cannot overlap itself, caches filtering decisions, and omits retained auxiliary tags.
  • Only-depth reuses one input record and its difference array, avoids cloning names for existing groups, and replaces interval-query index construction with stable sorting and one-pass union. Existing clipping, deletion, reference-skip, mate, coordinate, and merging behavior is preserved.
  • Seqair only-depth requests core fields and, for mate fixing, names. It uses the raw-record API rather than a pileup store.
  • Safe small-counter formatting and the mate fast paths incorporate Rob's suggestions. No custom allocator, unchecked vector pushes, or new unsafe blocks.

Compatibility and limitations

Seqair's generic depth cap does not match HTSlib's whole-read admission at boundaries; the adapter preserves Perbase's existing rule instead. The comparison tests include low and zero caps, ties, filters, and all nine mate strategies.

The optional backend requires Rust 1.98.1 or newer and an external FASTA for CRAM, including embedded-reference CRAM. The pinned fix expands BAM long CIGARs from CG before pileup counting. The former Perbase rejection guard is removed, and its regression now checks the actual base and insertion counts for a 65,536-operation BAM. Both depth commands support these records.

Validation

  • Revalidated the pinned candidate on Rust 1.98.1 with both test configurations below. Its Linux release build passed both --seqair help checks and the Seqair CLI differential suite. The tag-triggered publishing workflow was not run; no binaries have been released, and the macOS release build has not been exercised in this update.
  • cargo +1.98.1 test --locked: 132 unit tests and one doctest passed.
  • cargo +1.98.1 test --locked --features seqair: 137 unit tests and one doctest passed.
  • cargo fmt --check and git diff --check passed. Clippy completed; existing warnings in examples/older tests and dependency future-compatibility warnings remain.
  • Ran the built release executable through the Goose CLI differential suite with both backends: independent CIGAR/depth oracles, BAM/CRAM, all mate strategies, filters, depth caps, missing sequence, BAI/CSI, BED/VCF regions, empty and quoted references, repeated regions, chunk/worker changes, BGZF output, and output failures. For Seqair's embedded/no-reference CRAM cases, supplied the fixture FASTA to both implementations; the unsupported reference-free mode is not claimed. Skipped only the Goose benchmark-runner's self-test, which depends on that runner's local package cache; it is not a Perbase functionality test.
  • Added a CLI depth test with explicit expected coverage across skips, deletions, mate overlap, filtering, and BED output. Existing depth tests run unchanged against the replacement counter.
  • The long-CIGAR regression now expects the real 1M1I operations: each tested position has depth 1, an A count of 1, and an insertion count of 1, rather than a reference skip from the placeholder.
  • CI tests both default and Seqair-feature builds; both passed on the updated candidate, commit 607f59a.

Performance

Original measurements of this PR's first build versus a clean build of released Perbase 1.4.1. These predate the long-CIGAR pin and are not the experimental fork's R7 numbers. The table is retained as the original matched comparison, not mixed with newer measurements.

Chr20 BAM on the same computer, -t 32 -c 1000000, default queue budgets, uncompressed TSV written to a regular file (no fsync). One warmup followed by two measured runs for each variant/workload, run sequentially in rotating order without concurrent builds or tests. All 27 outputs matched the corresponding workload's SHA-256. These are preliminary two-run measurements, not confidence intervals.

Workflow 1.4.1 median seconds PR + HTSlib PR + upstream Seqair
base-depth -m 49.425 19.875 7.340
only-depth 10.065 4.175 4.140
only-depth -m 10.175 4.150 4.120

The application changes improve both backends. Seqair gives another approximately 2.71× speedup over the optimized HTSlib base-depth path here; only-depth is essentially tied between the two new backends.

Workflow 1.4.1 peak RSS, GiB PR + HTSlib PR + upstream Seqair
base-depth -m 10.24 2.51 5.59
only-depth 1.53 0.31 0.46
only-depth -m 1.52 0.38 0.51

RSS is the maximum over the two timed runs, not an average. Seqair's base-depth speed comes with more memory than the PR's HTSlib path because it retains each active region's reads.

Samples and build provenance

Seconds in measured order:

  • Base + mates: release [49.47, 49.38]; PR HTSlib [19.55, 20.20]; PR Seqair [7.33, 7.35].
  • Depth: release [9.78, 10.35]; PR HTSlib [4.22, 4.13]; PR Seqair [4.18, 4.10].
  • Depth + mates: release [10.19, 10.16]; PR HTSlib [4.16, 4.14]; PR Seqair [4.11, 4.13].

Both source revisions built with Rust 1.98.1, their release profile, RUSTFLAGS='-C target-cpu=native', CC=clang, and CFLAGS='-O3 -march=native'. Candidate built with --features seqair; both PR columns use that same executable, changing only --seqair. Baseline built from clean 1.4.1 with no features. Both commands used --locked and their respective Cargo.lock files.

  • Input SHA-256: a3527928f449355c693275c95fdf2712efaa6279ed57e461652b758d78ff6804.
  • Release executable: 136131baf86623e9d55110318e691878ade0ce98704f6612a57b9ed076044060.
  • PR executable: b10b1722946bea9a8b952bf416579b94ba46c390b86a5b863fd3965f850c1eab.
  • Base + mates TSV: b4e1e41f38b69f5ae99c2a9b5fa652773dde07df41f4467572a6a2bc8de93ed7.
  • Depth TSV: 825dd615bcba2d4e841e46e775a7b4a9ab75d146ddfb15f8b10a1956e5d20012.
  • Depth + mates TSV: 24556d4290684b96765e16bb8b0750f6cd8c1766438475d910224ce07fd61060.

Historical timing recheck

The exact saved PR executable above, verified by SHA-256 and not rebuilt, subsequently measured 5.99 s for mate-fixed base-depth (samples 5.99, 6.03, 5.90), rather than its original 7.34 s. A newer unchanged control measured 5.88 s (5.88, 6.00, 5.87) in the same session. Each had one warmup and three measured runs in alternating order, with the same BAM, 32 threads, 1 Mb chunks, and full TSV output. All outputs matched the recorded checksum. Summed CPU medians were 74.06 and 74.54 seconds respectively.

This rules out a persistent slowdown in the old executable; it does not establish why the earlier environment was slower. Machine or I/O conditions remain hypotheses. Do not attribute the 7.34 → roughly 5.9 second difference to code changes, or substitute the new Seqair number into the old comparison against HTSlib and released Perbase.

Deferred Seqair work

The long-CIGAR correctness fix is now a pinned prerequisite, not deferred optimization work. Other library changes remain outside this application PR:

  1. Consider preparation for sequential pileup without automatic primary-mate linking and random-access reach/window indexes when callers do not need them.
  2. Caller-defined compact entries were tested in Seqair #96, then rejected and the PR closed: the whole-app benefit did not justify the public API. Compact and ordinary entries had identical mate-fixed elapsed medians on that draft. This candidate does not contain the callback API or adapter.
  3. Active-read state, direct common-match contributions, and unused CIGAR summaries are separate library-level investigations, not changes included here. Upstream already supports optional auxiliary retention; this PR uses that API.

The experiment narrative contains the historical checkpoints. These library follow-ups are deliberately separate from this application PR.

AI disclosure

Prepared with Amp, including code and tests. This is a draft for Seth's full human review before merge, as required by CONTRIBUTING.md. No merge or release is requested by this PR.

sstadick and others added 2 commits November 14, 2026 21:58
Reuse ordered output buffers and read-name groups, incorporate safe mate
and formatting fast paths, and stream only-depth from reusable records.
Use published Seqair 0.10.1 without the experimental fork or unchecked
vector operations. Preserve depth admission and reject unsupported BAM
long-CIGAR pileups explicitly.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1ca76-7f80-72d6-ac92-b1b4e5bacdcf
Co-authored-by: Amp <amp@ampcode.com>
Replace the long-CIGAR rejection guard with a counting regression and include optional Seqair support in release binaries while retaining HTSlib as the default.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1ca76-7f80-72d6-ac92-b1b4e5bacdcf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant