Repository navigation
feat: delete stored runs, and stop same-day card filename collisions - #326
Merged
Merged
Conversation
Two sessions of one tool-call configuration on one day shared a default name, so a rerun that only added --needle overwrote the earlier card and PNG. The name now carries the UTC time of the newest run on the card; rendering the same session again still writes the same file.
Deleting a run removes its database row, traces, checkpoints, and the Markdown report it wrote. The report path comes from the database, so a file is removed only when it has the shape the report writers produce, and a report another run still names is kept. The new delete command selects runs by id, or by --status, --before, and --model, lists them, and asks before deleting; --dry-run previews and --yes skips the prompt. The web UI adds a Delete button to each run and a Select mode on the run list, both confirmed through a native dialog, and refuses to delete a run it is still executing. The argument schema moves to version 10.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two requests from the DGX forum thread (#221, #222).
Problem
Card overwrites.
cardnamed its file<model>-<YYYYMMDD>-<fingerprint>.svg, with the fingerprint taken from the tool-call run. A second session on the same day that only added--needlekept that fingerprint, because needle is stored as its own run, so the new card and PNG overwrote the first.No way to delete runs. Interrupted and failed attempts piled up in the web UI's run list, and nothing in the tool could remove a stored run.
Change
Card filenames now carry the UTC time of the newest run on the card:
<model>-<YYYYMMDD-HHMM>-<fingerprint>.svg. A new session gets a new file. Rendering the same session again still writes the same file, so the SVG/PNG pairing and the stale-PNG warning keep working.Deleting runs removes the database row, its traces and checkpoints, and the Markdown report the run wrote, through one path in
application/run_deletion.py:YYYY/MM/<run_id>[--label][suffix].md). A path that does not match, or a report another stored run still names, is kept and listed.tool-eval-bench delete RUN_ID..., or select with--status,--before DATE, and--model. It lists the runs and asks first;--dry-runpreviews and--yesskips the prompt. Without a terminal it refuses unless--yesis passed. An unknown id deletes nothing, and a baredeleteis an error. The flat form is--deletewith--delete-status,--delete-before, and--yes.<dialog>with Cancel focused.POST /api/runs/deletegoes through the existing token, same-origin, and JSON checks, and returns 409 for a run the server is still executing.The argument schema moves to version 10 and both compatibility snapshots are regenerated with
scripts/update_compat_snapshots.py. README,docs/cli-reference.md,docs/architecture.md, and two changelog fragments are updated.docs/api.mdstill said schema"8"and now says"10".Verification
ruff check,ruff format --check,mypy, and the full suite with--randomly-seed=104729pass (10379 tests).tests/test_run_deletion.pycovers storage cleanup across all three tables, the report-shape check and its boundaries, kept and shared reports, filters, the endpoint's 400/409/403/415 cases, and the CLI's prompt,--yes,--dry-run, and selection errors.9aaa93e4on one day now get different names.serveinstance in headless Chromium against a seeded database. Selected two runs, dismissed the dialog with Escape (nothing deleted), confirmed, then deleted from a run's header, and checked the reports were gone from disk. Checked in light and dark mode.Written with AI assistance; reviewed before opening.