Skip to content

test: enforce 10-second default test budget #415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
acgetchell merged 5 commits into from
May 21, 2026
Merged

test: enforce 10-second default test budget #415

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
772b2d0
test: enforce 10-second default test budget
acgetchell May 21, 2026
9b741a2
test: gate slow 4D quality properties
acgetchell May 21, 2026
0541b36
test: gate slow 4D property families
acgetchell May 21, 2026
4c2088a
test: gate Windows-slow 4D property families
acgetchell May 21, 2026
03a646f
test: gate hosted-runner slow 4D property families
acgetchell May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .config/nextest.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ fail-fast = false
# Retry flaky tests (useful for proptests)
retries = 0

# Default-suite tests should stay under the 10-second per-test budget. Longer
# deterministic correctness tests belong behind the slow-tests feature.
slow-timeout = { period = "10s", terminate-after = 1 }

[profile.ci]
# CI profile: optimized for GitHub Actions
# Inherits from default profile
Expand All @@ -29,8 +33,8 @@ fail-fast = false
# Retry flaky tests once in CI (helps with proptests and timing-sensitive tests)
retries = 1

# Show slow tests (> 60s) in CI
slow-timeout = { period = "60s", terminate-after = 2 }
# Enforce the default-suite 10-second per-test budget in CI.
slow-timeout = { period = "10s", terminate-after = 1 }

# JUnit report configuration
[profile.ci.junit]
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/codecov.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,6 @@ jobs:
echo "::group::Running tests with nextest"
# Generate JUnit XML for Codecov Test Analytics
# JUnit path is configured in .config/nextest.toml [profile.ci.junit]
# Release mode is required: exact-predicate arithmetic makes 3D+ proptests
# exceed timeout limits in debug mode (>60s debug vs <1s release).
# Avoid --all-features: it enables slow-tests and count-allocations which
# are special-purpose features not suitable for general test runs.
cargo nextest run --release --profile ci
Expand Down
37 changes: 21 additions & 16 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,43 +84,48 @@ codegen-units = 1
debug = "line-tables-only"

[[bench]]
name = "circumsphere_containment"
path = "benches/circumsphere_containment.rs"
name = "allocation_hot_paths"
path = "benches/allocation_hot_paths.rs"
harness = false

[[bench]]
name = "topology_guarantee_construction"
path = "benches/topology_guarantee_construction.rs"
name = "boundary_uuid_iter"
path = "benches/boundary_uuid_iter.rs"
harness = false

[[bench]]
name = "tds_clone"
path = "benches/tds_clone.rs"
name = "ci_performance_suite"
path = "benches/ci_performance_suite.rs"
harness = false

[[bench]]
name = "boundary_uuid_iter"
path = "benches/boundary_uuid_iter.rs"
name = "circumsphere_containment"
path = "benches/circumsphere_containment.rs"
harness = false

[[bench]]
name = "remove_vertex"
path = "benches/remove_vertex.rs"
name = "cold_path_predicates"
path = "benches/cold_path_predicates.rs"
harness = false

[[bench]]
name = "ci_performance_suite"
path = "benches/ci_performance_suite.rs"
name = "profiling_suite"
path = "benches/profiling_suite.rs"
harness = false

[[bench]]
name = "profiling_suite"
path = "benches/profiling_suite.rs"
name = "remove_vertex"
path = "benches/remove_vertex.rs"
harness = false

[[bench]]
name = "cold_path_predicates"
path = "benches/cold_path_predicates.rs"
name = "tds_clone"
path = "benches/tds_clone.rs"
harness = false

[[bench]]
name = "topology_guarantee_construction"
path = "benches/topology_guarantee_construction.rs"
harness = false

[lints.rust]
Expand Down
9 changes: 6 additions & 3 deletions benches/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ predicates fast across 2D-5D.

| Benchmark | Purpose | Scale | Typical Runtime | Used By |
|-----------|---------|-------|-----------------|---------|
| `allocation_hot_paths.rs` | Allocation contracts for public hot paths | Calibrated 2D-5D canary fixtures | ~1-2 min | Manual allocation checks |
| `boundary_uuid_iter.rs` | Focused boundary-facet and UUID iterator microbenchmarks | 3D small fixed schedules | <1 min | Manual micro-optimization checks |
| `ci_performance_suite.rs` | Public workflow regression contract | Calibrated 2D-5D canaries | ~5-10 min | CI, baselines, `just perf-no-regressions` |
| `circumsphere_containment.rs` | Compare circumsphere predicate methods | 2D-5D fixed, 3D random, edge cases | ~5 min | Predicate tuning, summaries |
| `boundary_uuid_iter.rs` | Focused boundary-facet and UUID iterator microbenchmarks | 3D small fixed schedules | <1 min | Manual micro-optimization checks |
| `cold_path_predicates.rs` | Track hot/cold predicate paths | 2D-5D hot queries, near-boundary cases | ~2-5 min | Predicate optimization work |
| `profiling_suite.rs` | Large-scale construction, memory, query, validation profiling | 2D/3D 10k, 4D 3k, 5D 1k | ~2-3 hr | Manual/monthly |
| `remove_vertex.rs` | Vertex removal and rollback cost | 2D-5D fixed cases | ~1-5 min | Vertex removal |
Expand All @@ -25,17 +26,19 @@ predicates fast across 2D-5D.
| Quick local large-scale wall-clock guard | `just perf-large-scale-smoke` |
| Fast local PR performance guard with temporary same-machine baseline | `just perf-no-regressions` |
| Full CI benchmark suite only | `just bench-ci` |
| Allocation-contract microbenchmarks | `just bench-allocations` |
| Persist/update the default local baseline artifact | `just perf-baseline` |
| Generate a scratch baseline without replacing the default | `just perf-baseline-to <out> [ref]` |
| Persist/update the default local baseline from a release/ref | `just perf-baseline v0.7.5` |
| Compare against an existing baseline | `just perf-compare <file>` |
| Release performance summary | `just bench-perf-summary` |
| Smoke-test benchmark harnesses | `just bench-smoke` |
| Vertex removal mutation baseline | `cargo bench --profile perf --bench remove_vertex -- --noplot` |
| Predicate comparison | `cargo bench --profile perf --bench circumsphere_containment -- --noplot` |
| Allocation hot-path contracts | `cargo bench --profile perf --bench allocation_hot_paths --features count-allocations -- --noplot` |
| Boundary/UUID microbenchmarks | `cargo bench --profile perf --bench boundary_uuid_iter -- --noplot` |
| Predicate comparison | `cargo bench --profile perf --bench circumsphere_containment -- --noplot` |
| Predicate cold-path work | `cargo bench --profile perf --bench cold_path_predicates -- --noplot` |
| Large-scale scaling suite | `cargo bench --profile perf --bench profiling_suite -- --noplot` |
| Vertex removal mutation baseline | `cargo bench --profile perf --bench remove_vertex -- --noplot` |
| One-dimension acceptance/profiling run | `just debug-large-scale-{2,3,4,5}d [n] [repair_every]` |
| Deep profiling | `cargo bench --profile perf --bench profiling_suite --features count-allocations` |

Expand Down
Loading
Loading