Skip to content

Fix stats permutation bugs - #14482

Merged
drammock merged 4 commits into
mne-tools:mainfrom
larsoner:exact-all-perms
Nov 2, 2026
Merged

drammock merged 4 commits into
mne-tools:mainfrom
larsoner:exact-all-perms

Conversation

@larsoner

@larsoner larsoner commented Nov 2, 2026

Copy link
Copy Markdown
Member

Closes #14479
Closes #12216
Closes #11511

Okay this one was a pain.

  1. Fix the off-by-one from n_permutations="all" in the cluster permutation tests does not run the full exact test #14479
  2. Fix a sign convention mismatch (0 and 1 meaning don't flip and flip when the consuming function used them in the opposite way)
  3. Fix a small bug involving use of abs when we actually needed to pick either the min or max depending on tail

Has co-author trailer since this was WIP in #12216 (thanks @kimcoco !)

Diagnosed and code changes drafted with Claude Opus 5.5, I reviewed the code in depth. I also had it run equivalence tests with FieldTrip (installed locally as well) as extra insurance that we're hopefully doing things correctly now. 😅

Co-authored-by: Cora Kim <kimjico@gmail.com>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filename is off-by-one 😆

@drammock drammock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @larsoner. one docstring improvement, one maintainability question.

Comment thread mne/stats/permutations.py Resolved
Comment thread mne/stats/permutations.py Outdated
def _t_max(T, tail):
"""Get the t-max statistic in the direction(s) of the tail."""
T = np.abs(T) if tail == 0 else T * tail # tail == -1 -> positive for min(T)
return np.max(T, axis=-1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this preserves the prior behavior of always returning a positive value, even if the most extreme T-value is negative. Would either of these changes make things easier to maintain?

  1. changing name from _max_stat to _most_extreme_stat or similar (likewise for _t_max)?
  2. returning a signed extremum, and adapting the logic of the consuming code?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think that's cleaner, pushed!

larsoner and others added 2 commits November 2, 2026 13:57
Co-authored-by: Daniel McCloy <dan@mccloy.info>
@drammock
drammock enabled auto-merge (squash) November 2, 2026 19:12
@drammock
drammock merged commit e2f9b20 into mne-tools:main Nov 2, 2026
25 checks passed
@drammock
drammock deleted the exact-all-perms branch November 2, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

n_permutations="all" in the cluster permutation tests does not run the full exact test Permutation test number of tails

2 participants