fix: SpliceIndexer double-applies sample-subset map (spliced+subset sample scramble)

[d-laub]

Root cause

SpliceIndexer.parse_idx in python/genvarloader/_dataset/_indexing.py (around line 451 on main) applied the sample-subset map twice, silently corrupting spliced output whenever a Dataset had been sample-subsetted with a non-identity sample_subset_idxs.

The double-application happened as follows:

1. First application (line 412/420/427): s_idx = self.dsi._s_idx[s_idx] converts output-space sample positions [0..n_subset_samples) → on-disk sample positions via full_sample_idxs[sample_subset_idxs[...]].
2. The ravel/unravel over self.full_shape produces s_idx values already in on-disk space.
3. After r_idx = self.map.full_splice_map[r_idx] (exon expansion) and s_idx = s_idx.repeat(lengths), the code called self.dsi.parse_idx((r_idx, s_idx)).
4. Second application (line 245 of DatasetIndexer.parse_idx): s_idx = self._s_idx[s_idx] applied full_sample_idxs[sample_subset_idxs[...]] again to what were already on-disk positions — mapping wrong samples entirely, or raising IndexError when the double-mapped value fell out of bounds.

Symptom

Only triggered with splice_info enabled and a non-identity sample_subset (i.e. dataset.subset_to(samples=...)). Non-spliced paths were unaffected because they don't call dsi.parse_idx with already-mapped indices.

Concrete MMRF repro: With the MMRF consensus dataset, sample MMRF_2702 (svar position 54, GVL sorted 625) received MMRF_1395's NRAS G12D mutation because sample_subset_idxs[625] mapped to MMRF_1395.

Fix

python/genvarloader/_dataset/_indexing.py, line ~451

Replace the dsi.parse_idx call with a direct ravel_multi_index:

# Before (buggy):
ds_idx, *_ = self.dsi.parse_idx((r_idx, s_idx))

# After (fixed):
r_flat = self.dsi.full_region_idxs[r_idx]
ds_idx = np.ravel_multi_index((r_flat, s_idx), self.dsi.full_shape).ravel()

After the unravel/exon-expand step, r_idx are region positions (indices into full_region_idxs) and s_idx are already-mapped on-disk sample indices. Applying full_region_idxs[r_idx] once and ravelling is all that's needed — no second _s_idx pass.

Regression test

tests/unit/dataset/test_splice_indexer_subset.py — three cases:

1. test_parse_idx_sample_subset_applied_once: constructs a SpliceIndexer with sample_subset_idxs=[2, 4] over 5 on-disk samples, requests all transcripts × all subset samples, and asserts every returned on-disk sample index is 2 or 4 (not a double-mapped value). Failed on main with IndexError: index 4 is out of bounds for axis 0 with size 2. Passes after the fix.
2. test_parse_idx_single_sample_from_subset: requests one specific subset sample by output position and verifies the correct on-disk index. Failed on main with the same IndexError. Passes after the fix.
3. test_parse_idx_no_subset_unaffected: no sample subset; confirms the non-subset path is unchanged. Passed on main and after the fix.

Test execution

Ran locally in the repo's dev pixi environment (Python 3.10, maturin Rust build):

pixi r -e dev pytest tests/unit/dataset/test_splice_indexer_subset.py tests/unit/dataset/test_indexing.py tests/dataset/test_flat_splice.py tests/unit/ tests/dataset/ -v

Results:

• 3/3 new regression tests: FAIL → PASS
• 460 unit tests: 460 passed, 5 skipped, 2 xfailed (no regressions)
• 167 dataset integration tests: 167 passed (no regressions)

🤖 Generated with Claude Code