Skip to content

Add --partition slice:M/N for round-robin test slicing#767

Merged
MatthewMckee4 merged 1 commit into
mainfrom
matt/partition-slice
May 6, 2026
Merged

Add --partition slice:M/N for round-robin test slicing#767
MatthewMckee4 merged 1 commit into
mainfrom
matt/partition-slice

Conversation

@MatthewMckee4
Copy link
Copy Markdown
Owner

Summary

Adds a --partition slice:M/N flag to karva test for splitting a test run across CI jobs. Tests are sorted by qualified name and distributed round-robin: test 1 to slice 1, test 2 to slice 2, ..., test N+1 to slice 1, and so on. Running every slice:1/N through slice:N/N together covers every collected test exactly once.

karva test --partition slice:1/3
karva test --partition slice:2/3
karva test --partition slice:3/3

The slice is applied inside partition_collected_tests after the --last-failed filter and before bin-packing across workers, so it composes with both. Slice membership shifts when the test set changes, which is the documented tradeoff against a hash-based scheme — the upside is no historical data is required.

Closes #587.

Test Plan

ci

Closes #587. Slices the collected test set after sorting by qualified
name, keeping only tests at positions matching slice M of N. Validation
on the M/N values lives in PartitionSelection's FromStr.
@MatthewMckee4 MatthewMckee4 added cli Related to the command-line interface test-running Related to running tests with karva parallelism Related to test parallelism, partitioning, and concurrency labels May 6, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 6, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1 untouched benchmark


Comparing matt/partition-slice (4c9d712) with main (56b23da)

Open in CodSpeed

@MatthewMckee4 MatthewMckee4 merged commit 2cc8ff8 into main May 6, 2026
13 checks passed
@MatthewMckee4 MatthewMckee4 deleted the matt/partition-slice branch May 6, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Related to the command-line interface parallelism Related to test parallelism, partitioning, and concurrency test-running Related to running tests with karva

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slice-based (round-robin) test partitioning

1 participant