Skip to content

report.md

Latest commit

 

History

History
63 lines (51 loc) · 4.89 KB

report.md

File metadata and controls

63 lines (51 loc) · 4.89 KB
Raw

Benchmark reports

Generated by bench/report.ps1 from commit 8e2eb85 (2026-10-31).

  • Toolchain: moon 0.1.20261016 (1af328f 2026-10-16) ~\.moon-0.10.16\bin\moon.exe | moonc v0.10.16+b1a0e846e (2026-10-16) ~\.moon-0.10.16\bin\moonc.exe | moonrun 0.1.20261016 (1af328f 2026-10-16) ~\.moon-0.10.16\bin\moonrun.exe | | Feature flags enabled: rr_moon_mod,rr_moon_pkg
  • Instances: MIPLIB 2017 (https://miplib.zib.de), fetched by bench/fetch-instances.ps1; the data itself is not redistributed (see bench/README.md).
  • Every report below is written by its own script, and every one of those scripts refuses to write unless its run finished, covered every manifest entry, and passed its own checks: no certificate refused, every reported point an integer point of the model. This file is the index of them, and it is the only place that says which of them the current code still backs.

Reports

report scope result generated at code behind it
parse-report.md model file parsing (MPS/LP reader) instances attempted: 32, parsed: 32, failed: 0 8e2eb85 current
solve-report.md continuous kernel (LP relaxation, presolve, certificates), 2200-row budget over the whole manifest instances in manifest: 32, solved to optimality: 28, skipped: 3, other outcomes: 1 8e2eb85 current
mip-report.md branch and bound, 600-node budget over the whole manifest instances in manifest: 32, proven optimal: 5, stopped at the node budget: 10, skipped: 17, relaxations verified by verify: 7081, cuts added and verified by verify_cut: 345 8e2eb85 current
mip-report-small.md branch and bound, 30000-node budget over the small-instance list instances in manifest: 10, proven optimal: 6, stopped at the node budget: 4, relaxations verified by verify: 159910, cuts added and verified by verify_cut: 225 8e2eb85 current

current means no file under the report's own source paths changed between the commit it was generated from and HEAD and none of them is modified in the working tree - so the numbers still describe the code on disk. STALE means they describe an older one: the run happened and the report is its record, but re-running the command would not have to reproduce it. Regenerating a stale report is part of the round that made it stale, and this table is how that is noticed.

The staleness test is deliberately conservative: it compares the files a report's numbers depend on, so a changed comment counts as a change even though it cannot move a number, and an uncommitted edit counts even though the report's own commit is untouched. Untracked files count as well - a new .mbt in one of those directories is part of the package the report measured. That direction is the safe one - a report is flagged when it might no longer describe the current code, never cleared when it might not - and the only thing that settles a flagged report is re-running its command (or measuring that the numbers it would print are unchanged, which is worth writing down where it was measured).

What each report is evidence for

  • parse-report.md: the claim that the reader takes the whole manifest: every instance parses, none fails
  • solve-report.md: the kernel's answer quality on real models: optima found, size refusals, numerical failures
  • mip-report.md: that no run ends on a refused certificate, and that every reported point is an integer point of the model
  • mip-report-small.md: the M5 criterion that small integer instances reach their published optimum; cross-checked by bench/check-mip-objectives.ps1, which compares each proven objective with MIPLIB's own table and requires equality

How to reproduce

Run these in order in the repository root. The first one downloads the instances and writes bench/data/instances/manifest.txt; the rest read them and take minutes to hours, because they solve the whole manifest.

powershell -NoProfile -ExecutionPolicy Bypass -File bench/fetch-instances.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File bench/report-parse.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File bench/report-solve.ps1 -Relax -MaxRows 2200 -MaxIterations 20000 -Presolve
powershell -NoProfile -ExecutionPolicy Bypass -File bench/check-relaxation-bounds.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File bench/report-mip.ps1 -MaxRows 300 -MaxNodes 600 -MaxIterations 5000
powershell -NoProfile -ExecutionPolicy Bypass -File bench/report-mip.ps1 -Manifest bench/data/instances/small.txt -MaxRows 300 -MaxNodes 30000 -MaxIterations 20000 -Output bench/mip-report-small.md
powershell -NoProfile -ExecutionPolicy Bypass -File bench/check-mip-objectives.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File bench/check-mip-objectives.ps1 -Report bench/mip-report-small.md
powershell -NoProfile -ExecutionPolicy Bypass -File bench/report.ps1