Skip to content

test: enforce 10-second default test budget#415

Merged
acgetchell merged 5 commits into
mainfrom
test/256-default-test-budget
May 21, 2026
Merged

test: enforce 10-second default test budget#415
acgetchell merged 5 commits into
mainfrom
test/256-default-test-budget

Conversation

@acgetchell
Copy link
Copy Markdown
Owner

  • Gate default-suite cases at or above the 10-second budget behind slow-tests and remove obsolete high-dimensional periodic validation from routine runs.
  • Move allocation hot-path contracts into a Criterion benchmark over calibrated 2D-5D fixtures, leaving allocation_api as wiring smoke coverage.
  • Document the toroidal validation limits and add the bench-allocations workflow.

Closes #256

- Gate default-suite cases at or above the 10-second budget behind
  slow-tests and remove obsolete high-dimensional periodic validation from
  routine runs.
- Move allocation hot-path contracts into a Criterion benchmark over
  calibrated 2D-5D fixtures, leaving allocation_api as wiring smoke coverage.
- Document the toroidal validation limits and add the bench-allocations
  workflow.

Closes #256
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 21, 2026

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: dd805be7-9759-474c-9ea7-5d5aa31ec170

📥 Commits

Reviewing files that changed from the base of the PR and between 4c2088a and 03a646f.

📒 Files selected for processing (6)
  • tests/README.md
  • tests/proptest_delaunay_triangulation.rs
  • tests/proptest_facet.rs
  • tests/proptest_orientation.rs
  • tests/proptest_simplex.rs
  • tests/triangulation_builder.rs

Walkthrough

This PR enforces a 10s per-test budget in nextest profiles, gates multiple slow/high-dimension proptests behind the slow-tests feature, replaces heavy allocation tests with two smoke checks, and adds a feature-gated allocation hot-path Criterion benchmark plus supporting Cargo, justfile, and documentation updates.

Changes

Allocation Measurement and Test Taxonomy Overhaul

Layer / File(s) Summary
Test execution harness and timing standardization
.config/nextest.toml, .github/workflows/codecov.yml, justfile
Enforces a 10s per-test budget in default and CI nextest profiles, removes a release-mode rationale comment, and switches large-scale debug/test recipes from ci to slow profile.
Slow test feature gating and topology test removal
tests/proptest_delaunay_triangulation.rs, tests/proptest_euler_characteristic.rs, tests/proptest_triangulation.rs, tests/proptest_convex_hull.rs, tests/proptest_serialization.rs, tests/triangulation_builder.rs, tests/README.md
Gates selected 4D/5D and slow correctness proptests behind slow-tests and removes generated 4D/5D toroidal periodic validation tests; updates README notes accordingly.
Allocation API simplification to measurement wiring smoke tests
tests/allocation_api.rs, tests/README.md
Replace extensive allocation-bounded tests with two #[test] smoke checks validating measure_with_result allocation counters under count-allocations.
Allocation hot-path benchmarks and benchmark infrastructure
benches/allocation_hot_paths.rs, Cargo.toml, benches/README.md, justfile
Adds a new feature-gated Criterion benchmark (allocation_hot_paths.rs) exercising deterministic Delaunay fixtures (dims 2–5) and hot-path allocation checks; registers the bench in Cargo.toml, updates benches README, and adds just bench-allocations.
Documentation updates for allocation measurement system
docs/code_organization.md, docs/limitations.md, docs/production_review_remediation_checklist.md, CHANGELOG.md, tests/README.md
Document allocation measurement workflows, clarify toroidal limitations and test-bucket changes, update checklist wording to reference benchmarks, and add Unreleased breaking-change notes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 I count the hops and bytes with care,
Benchmarks planted in a timed-up lair.
Ten seconds whisper, slow tests take their cue,
Hot paths measured, gentle and true.
A rabbit nods — the harness springs anew.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 26.09% which is insufficient. The required threshold is 100.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title concisely and clearly describes the main change: enforcing a 10-second default test budget.
Description check ✅ Passed The description outlines gating slow tests, moving allocation contracts to benchmarks, and documenting limits, all of which are reflected in the changeset.
Linked Issues check ✅ Passed The PR addresses issue #256 by gating slow 4D property tests behind slow-tests and moving allocation hot-paths to benchmarks, directly targeting the 60-second budget constraint.
Out of Scope Changes check ✅ Passed All changes (test gating, nextest config updates, allocation benchmarking, and documentation) are directly aligned with the linked issue objectives.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/256-default-test-budget

Comment @coderabbitai help to get the list of available commands and usage tips.

@acgetchell acgetchell enabled auto-merge (squash) May 21, 2026 06:44
@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented May 21, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

🟢 Coverage ∅ diff coverage · -0.01% coverage variation

Metric Results
Coverage variation -0.01% coverage variation (-1.00%)
Diff coverage diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (1498153) 62504 56738 90.77%
Head commit (03a646f) 62504 (+0) 56730 (-8) 90.76% (-0.01%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#415) 0 0 ∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coderabbitai coderabbitai Bot added documentation Improvements or additions to documentation enhancement New feature or request rust Pull requests that update rust code labels May 21, 2026
- Move 4D triangulation quality properties behind slow-tests so the default
  suite stays within the 10-second per-test budget.
- Document that over-budget triangulation quality variants run through
  just test-slow.
- Refresh the active changelog for the test-bucket updates.
- Move 4D convex-hull, serialization, and insertion-order robustness
  properties behind slow-tests so default CI and Codecov runs stay within the
  10-second per-test budget.
- Document that over-budget convex-hull, serialization, and high-dimensional
  insertion-order variants run through just test-slow.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.75%. Comparing base (1498153) to head (03a646f).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #415      +/-   ##
==========================================
- Coverage   90.76%   90.75%   -0.02%     
==========================================
  Files          71       71              
  Lines       62294    62294              
==========================================
- Hits        56542    56534       -8     
- Misses       5752     5760       +8     
Flag Coverage Δ
unittests 90.75% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

- Move full 4D TDS invariant property tests behind slow-tests with the
  existing 5D variants.
- Gate 4D facet-topology triangulation properties while keeping the
  high-dimensional smoke coverage in the default suite.
- Document that these 4D/5D property families run through just test-slow.
- Move 4D Delaunay, facet, orientation, and simplex property families
  behind slow-tests when they exceed the default 10-second CI budget.
- Gate the 3D toroidal periodic known-limitation validation case with
  the slow-test suite.
- Document the additional 4D/5D slow-test coverage in the test README.
@acgetchell acgetchell merged commit ba15de8 into main May 21, 2026
22 checks passed
@acgetchell acgetchell deleted the test/256-default-test-budget branch May 21, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request rust Pull requests that update rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Predicate performance: re-enable all ignored proptests under 60s

1 participant