Skip to content

feat(release): archive releases on Zenodo and gate CITATION.cff at tag time - #1036

Draft
Yiminnn wants to merge 1 commit into
benchflow-ai:mainfrom
Yiminnn:chore/zenodo-archiving
Draft

feat(release): archive releases on Zenodo and gate CITATION.cff at tag time#1036
Yiminnn wants to merge 1 commit into
benchflow-ai:mainfrom
Yiminnn:chore/zenodo-archiving

Conversation

@Yiminnn

@Yiminnn Yiminnn commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Sets up Zenodo archiving for public releases, and fixes the metadata drift that
made our citation file unciteable.

Zenodo's GitHub integration archives every published GitHub Release, snapshots
the tag's source tree, and mints a version DOI (plus a concept DOI that always
resolves to the newest version — that is the one to cite). Without a
.zenodo.json in the repo, Zenodo derives authorship from GitHub contributor
statistics
, which is not our author list. So this PR adds that file.

.zenodo.json deliberately omits version and publication_date: Zenodo takes
both from the tag and the Release, so pinning them here would go stale on every
release — exactly the bug the rest of this PR fixes.

CITATION.cff had drifted to 0.6.9 / 2026-08-15 while five 0.7.x releases
shipped, because nothing checked it. It now names 0.7.4 / 2026-08-17, and the
tag-driven release validates it:

  • tools/release_version.py public-release now also reads CITATION.cff and
    fails when its version differs from the release tag, or when
    date-released is in the future.
  • The check runs in the existing "Validate release tag" step, before build
    and publish. A PyPI publish and the GitHub Release that triggers Zenodo
    archiving are irreversible; a failed validation is not.
  • Everyday CI deliberately does not require cff == pyproject.
    CITATION.cff names the last published release, so while main sits on a
    .devN version the citation file is legitimately one release behind. That is
    correct, not drift, and the tag-time gate is what keeps it honest.

docs/release.md gains a ## Zenodo Archiving section (mechanism, plus the
admin-only one-time setup steps), and the public-release step list now covers
bumping CITATION.cff and leaving it on the version just released.

Blocked on: the author list

.zenodo.json's creators is a placeholder, and
tests/test_citation_metadata.py::test_zenodo_creators_are_resolved fails on
purpose
while that placeholder is present. CI is red by design until real
authorship lands — a published DOI cannot be withdrawn, so placeholder authors
must not be able to reach main.

Needed per author: name as Family, Given, the order they should appear in,
ORCID, and affiliation.

Ordering constraint for whoever flips the switch

The Zenodo switch requires admin rights on this repo, and it must be flipped
only after a .zenodo.json with the real author list has merged to main.
Zenodo archives the tagged commit, so with our release cadence the very next tag
would otherwise mint a permanent DOI with contributor-statistics authorship.
Steps are written out in docs/release.md.

Also note Zenodo archives releases published after the switch is flipped, so
the first archived version will be the next release, not v0.7.4.

Test plan

  • uv run --extra dev python -m pytest tests/ — 1184 passed, 6 skipped, and
    the 1 intentional failure above.
  • uv run ruff check ., uv run ruff format --check, uv run ty check src/ tools/.
  • Ran the real workflow command, not just unit tests:
    pyproject bumped to 0.7.5 with CITATION.cff left at 0.7.4 and tag
    v0.7.5 is rejected with CITATION.cff records version '0.7.4' but this release publishes '0.7.5'; a future date-released is rejected; tag
    v0.7.4 on this branch passes and emits version=0.7.4; and
    internal-preview still runs under --with packaging --with pyyaml.
  • CITATION.cff validates against CFF 1.2.0 (cffconvert --validate).
  • .zenodo.json validates against Zenodo's legacy deposit JSON Schema with
    0 errors, and isVariantFormOf was checked against Zenodo's live
    relation-type vocabulary rather than assumed.
  • First real archive: after the next release, confirm on Zenodo that the
    record's authors/title/license came from .zenodo.json.

Follow-up (out of scope here)

The concept DOI does not exist until the first archived release, so adding the
DOI badge to README.md and the DOI to CITATION.cff has to be a second PR.

Notes on the YAML edge cases

Two things the parser has to survive, both covered by tests: YAML parses an
unquoted version: 1.0 as a float (while 0.7.4 is a string), so the
comparison goes through PEP 440 rather than string equality; and an unquoted
date-released arrives as a datetime.date while a quoted one arrives as a
string. The future-date check also tolerates one day of skew, because a releaser
east of UTC tagging after local midnight legitimately writes a date that is
still "tomorrow" for the runner.

…g time

Zenodo's GitHub integration archives each published Release and mints a DOI,
deriving authorship from GitHub contributor statistics unless the repository
ships .zenodo.json. Add that file, with version and publication_date left out
on purpose: Zenodo takes both from the tag and the Release, so pinning them
would go stale on every release.

CITATION.cff had drifted to 0.6.9 while five 0.7.x releases shipped, because
nothing checked it. It now names 0.7.4, and the tag-driven public release
validates it before anything is built or published — a citation file naming a
different version stops the release while it is still reversible, unlike a PyPI
publish or the Release that triggers archiving. Everyday CI deliberately does
not require cff == pyproject: CITATION.cff names the last published release, so
sitting behind main's .devN line is correct, not drift.

The creators list is a placeholder pending the agreed author list, and
tests/test_citation_metadata.py fails while that placeholder is present. A
published DOI cannot be withdrawn, so the switch must not be flipped, and this
must not merge, until real authorship replaces it.
@Yiminnn

Yiminnn commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

@xdotli — this needs the author list from you before it can go anywhere.

.zenodo.json is what Zenodo reads to mint the DOI for every published release.
If it is absent, Zenodo derives authorship from GitHub contributor statistics
(i.e. a list of accounts ranked by commit count), so the file has to carry the
real list instead. Right now creators is a placeholder, and
tests/test_citation_metadata.py::test_zenodo_creators_are_resolved fails on
purpose while it is — CI is red by design so a placeholder cannot reach main.
A published DOI cannot be withdrawn.

Per author, please drop in a comment:

  1. Name as Family, Given (e.g. Li, Xiangyi) — Zenodo's expected form.
  2. The order they should appear in.
  3. ORCID, if they have one.
  4. Affiliation.

Two open questions worth a call while you are here:

  • Individuals, an organizational author, or both? CITATION.cff currently
    says BenchFlow team while pyproject.toml names you and @kywch. Zenodo
    supports an organizational creator, but citations then carry no personal
    names, and nothing links to an ORCID. This PR leaves CITATION.cff
    authorship untouched — whichever way you go, I will make both files say the
    same thing.
  • Should any related paper or preprint be linked? If there is an arXiv ID or
    DOI, I will add it to related_identifiers as isDocumentedBy now rather
    than editing the record's metadata after the fact.

Once you post the list I will drop it in, CI turns green, and I will mark this
ready for review. Note that flipping the Zenodo switch needs repo admin and
must happen after this merges — docs/release.md has the steps and the reason
the order matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant