Open
Conversation
Add a new experimental Changelog check that evaluates whether a project
maintains descriptive logs of changes for its releases, aligning with
OSPS-BR-04 requirements.
The check supports two scoring paths:
Path A - Changelog file exists (max 10/10):
- 3 points for having a changelog file (CHANGELOG, CHANGES, NEWS,
HISTORY, RELEASE-NOTES, or RELEASE_NOTES with common extensions)
- Up to 7 points proportional to how many of the last 5 releases
have corresponding version entries with substantive content
Path B - No file, but releases have notes (max 10/10):
- Up to 10 points proportional to how many of the last 5 releases
have substantive body text (GitHub/GitLab release notes)
- Auto-generated boilerplate (e.g. Full Changelog links) is excluded
If neither a changelog file nor releases are found, the check returns
inconclusive.
Infrastructure changes:
- Add Body field to clients.Release struct
- Map GetBody() in GitHub client and Description in GitLab client
New probes:
- hasChangelogFile: detects changelog files at the repo root
- releasesHaveChangelog: checks release documentation coverage
Gated behind SCORECARD_EXPERIMENTAL like SBOM and Webhooks.
Fixes: ossf#4824
Signed-off-by: Clayton Kehoe <clayton.j.kehoe@boeing.com>
kehoecj
requested review from
jeffmendoza and
spencerschrock
and removed request for
a team
dosubot
Bot
added
the
size:XL
|
This pull request has been marked stale because it has been open for 10 days with no activity |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Feature — a new experimental Changelog check.
What is the current behavior?
Scorecard has no check for whether a project maintains a changelog or descriptive release notes. There is no measurement for OSPS-BR-04 compliance ("all releases MUST provide a descriptive log of functional and security modifications").
What is the new behavior (if this is a feature change)?
A new experimental
Changelogcheck with two scoring paths:Path A — Changelog file exists (max 10/10):
CHANGELOG,CHANGES,NEWS,HISTORY,RELEASE-NOTES, orRELEASE_NOTESwith.md,.txt,.rst,.adocextensions)Path B — No changelog file, but releases have notes (max 10/10):
**Full Changelog**links) is excludedNeither: Inconclusive
Version entries in changelog files are validated for substantive content — empty headers or headers with only sub-headers do not count.
Gated behind
SCORECARD_EXPERIMENTALlike SBOM and Webhooks.Which issue(s) this PR fixes
Fixes #4824
Special notes for your reviewer
This PR includes a small infrastructure change: a
Bodyfield added toclients.Release, mapped from GitHub'sGetBody()and GitLab'sDescription. This is a one-line addition to each client with no behavioral change to existing checks — the field is only read by the new Changelog check.Known limitations:
**Full Changelog**pattern. GitLab and other tool-generated boilerplate may pass through.--local) cannot evaluate release coverage sinceListReleasesis unsupported for the local client.The version parsing is intentionally heuristic-based and may need refinement as the check gets real-world usage, which is what the
experimentallifecycle is for.Does this PR introduce a user-facing change?