Skip to content

feat: delete stored runs, and stop same-day card filename collisions - #326

Merged
SeraphimSerapis merged 2 commits into
mainfrom
feat/delete-runs-card-filenames
Oct 28, 2026
Merged

SeraphimSerapis merged 2 commits into
mainfrom
feat/delete-runs-card-filenames

Conversation

@SeraphimSerapis

Copy link
Copy Markdown
Owner

Two requests from the DGX forum thread (#221, #222).

Problem

Card overwrites. card named its file <model>-<YYYYMMDD>-<fingerprint>.svg, with the fingerprint taken from the tool-call run. A second session on the same day that only added --needle kept that fingerprint, because needle is stored as its own run, so the new card and PNG overwrote the first.

No way to delete runs. Interrupted and failed attempts piled up in the web UI's run list, and nothing in the tool could remove a stored run.

Change

Card filenames now carry the UTC time of the newest run on the card: <model>-<YYYYMMDD-HHMM>-<fingerprint>.svg. A new session gets a new file. Rendering the same session again still writes the same file, so the SVG/PNG pairing and the stale-PNG warning keep working.

Deleting runs removes the database row, its traces and checkpoints, and the Markdown report the run wrote, through one path in application/run_deletion.py:

  • The rows go in one transaction before any file is touched.
  • The report path comes from the database, so a file is removed only when it is shaped like a report the writers produce (YYYY/MM/<run_id>[--label][suffix].md). A path that does not match, or a report another stored run still names, is kept and listed.
  • CLI: tool-eval-bench delete RUN_ID..., or select with --status, --before DATE, and --model. It lists the runs and asks first; --dry-run previews and --yes skips the prompt. Without a terminal it refuses unless --yes is passed. An unknown id deletes nothing, and a bare delete is an error. The flat form is --delete with --delete-status, --delete-before, and --yes.
  • Web UI: a Delete button on each run, and a Select mode on the run list whose "Select all" covers only the runs the current filters show. Both confirm through a native <dialog> with Cancel focused. POST /api/runs/delete goes through the existing token, same-origin, and JSON checks, and returns 409 for a run the server is still executing.

The argument schema moves to version 10 and both compatibility snapshots are regenerated with scripts/update_compat_snapshots.py. README, docs/cli-reference.md, docs/architecture.md, and two changelog fragments are updated. docs/api.md still said schema "8" and now says "10".

Verification

  • ruff check, ruff format --check, mypy, and the full suite with --randomly-seed=104729 pass (10379 tests).
  • tests/test_run_deletion.py covers storage cleanup across all three tables, the report-shape check and its boundaries, kept and shared reports, filters, the endpoint's 400/409/403/415 cases, and the CLI's prompt, --yes, --dry-run, and selection errors.
  • A regression test reproduces the forum's collision: two sessions with fingerprint 9aaa93e4 on one day now get different names.
  • Drove a real serve instance in headless Chromium against a seeded database. Selected two runs, dismissed the dialog with Escape (nothing deleted), confirmed, then deleted from a run's header, and checked the reports were gone from disk. Checked in light and dark mode.

Written with AI assistance; reviewed before opening.

Two sessions of one tool-call configuration on one day shared a default
name, so a rerun that only added --needle overwrote the earlier card and
PNG. The name now carries the UTC time of the newest run on the card;
rendering the same session again still writes the same file.
Deleting a run removes its database row, traces, checkpoints, and the
Markdown report it wrote. The report path comes from the database, so a
file is removed only when it has the shape the report writers produce,
and a report another run still names is kept.

The new delete command selects runs by id, or by --status, --before, and
--model, lists them, and asks before deleting; --dry-run previews and
--yes skips the prompt. The web UI adds a Delete button to each run and
a Select mode on the run list, both confirmed through a native dialog,
and refuses to delete a run it is still executing. The argument schema
moves to version 10.
@SeraphimSerapis
SeraphimSerapis merged commit 887b537 into main Oct 28, 2026
9 checks passed
@SeraphimSerapis
SeraphimSerapis deleted the feat/delete-runs-card-filenames branch October 28, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant