Skip to content

feat(lint): add markdown output format for CI job summaries and PR comments (closes #133) - #179

Open
shaw778 wants to merge 2 commits into
laaielichter:mainfrom
shaw778:feat/lint-format-markdown
Open

shaw778 wants to merge 2 commits into
laaielichter:mainfrom
shaw778:feat/lint-format-markdown

Conversation

@shaw778

@shaw778 shaw778 commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

What and why

Closes #133.

gulliver lint currently supports outputting findings in text, json, sarif, github annotations, gitlab, and junit. A readable Markdown report format was missing for appending directly to GitHub Actions job summaries ($GITHUB_STEP_SUMMARY), pull request comments, wikis, or chat pastes without configuring SARIF/code scanning.

This PR adds --format markdown to gulliver lint:

  • Outputs a clean Markdown report with a summary header and findings table:
    • Header: ### Gulliver lint: N file(s), X error(s), Y warning(s), Z info (including (A ignored, B in baseline) when present).
    • Table: | Severity | Rule | Where | Problem | Fix | with file paths wrapped in backticks.
    • Clean runs output a single concise line: No scale problems found in N file(s).
    • Escapes | and newlines inside table cells so formatted messages do not break table layout.
    • Limits report length to ~60,000 characters (MAX_MARKDOWN), appending an omission note when findings exceed the size limit to stay comfortably within the GitHub comment cap of 65,536 characters.
  • Works with -o/--output PATH to write to a Markdown file, as well as printing directly to stdout.
  • Respects --fail-on and returns the same exit codes as other lint formats.
  • Updates documentation in docs/features/lint-ci.md and README.md with recipes for $GITHUB_STEP_SUMMARY.

How it was tested

  • Added unit and CLI tests in tests/test_lint_report.py:

    • test_markdown_format_with_findings: tests header and table rendering with findings across multiple files.
    • test_markdown_format_clean_run: verifies clean run single-line output.
    • test_markdown_format_escapes_pipe_and_newlines: verifies | and escaping within table cells.
    • test_markdown_summary_includes_ignored_and_baselined: verifies ignored and baseline counts in the summary header.
    • test_markdown_truncation_for_long_reports: verifies report truncation at ~60k characters with omission note.
    • test_markdown_cli_stdout_and_file_output: verifies CLI output to stdout, -o file output, and exit codes with --fail-on.
  • Updated CHANGELOG.md under ## Unreleased > ### Added.

  • pytest tests/test_lint_report.py (77 passed).

  • pytest (989 passed, 11 skipped, 3 OS-level symlink privilege tests).

  • ruff check . (All checks passed!).

  • pytest passes

  • ruff check . passes

  • README / AGENTS.md updated if behaviour changed

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shaw778, this is a really solid piece of work. I merged main into it locally. ruff check . is clean and the full pytest run passes (1007 passed, 10 skipped).

I also ran gulliver lint m --format markdown on a small demo. It gives the right header and table, and a clean run prints No scale problems found in 1 file(s). Exit codes match the other formats: 2 on an error, 2 with --fail-on warn, 0 with --fail-on never and -o, and 1 for a missing path. I mutation-checked the new tests too. Removing the | escape, the newline escape, the truncation, the ignored count, the clean-run line or the backticks each makes a test fail. Very nice.

Three small things before I merge:

  1. Please drop the CHANGELOG.md line. Sorry, I know the issue's "Done when" lists it, and that's my mistake. In this repo the maintainer writes the changelog entry after merging, with credit to you. Please revert that file to main.

  2. Angle brackets disappear on GitHub. The GV003 fix text is Add ON <key> = <key> (or USING).... GitHub treats <key> as an HTML tag and strips it, so the job summary and PR comment show Add ON = (or USING) for the most important error. Please escape < and > in _escape_cell as well, for example &lt; / &gt;, or \< / \>. Please also add a test case that checks <key> comes out escaped, either in the pipe/newline test or with a real GV003 finding.

  3. Docs nit. In docs/features/lint-ci.md, the line gulliver lint models/ --format markdown >> $GITHUB_STEP_SUMMARY --fail-on never works in bash, but it reads as if --fail-on never were part of the redirect target. Please put the flags first: gulliver lint models/ --format markdown --fail-on never >> "$GITHUB_STEP_SUMMARY".

That's all. Once those are in, I'll re-run CI and merge.


Generated by Claude Code

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shaw778, the Markdown format works well. On main, gulliver lint --format markdown is rejected by argparse. On this branch it renders the summary and table. I also checked that removing the pipe escape, or the truncation guard, makes the matching new tests fail.

One change before it can merge: this PR adds a line to CHANGELOG.md. The maintainer writes the CHANGELOG entry in this project (see CONTRIBUTING.md), so please remove that line and keep the rest. Once it is gone and CI is green, this is ready to go.

Welcome to Gulliver! CONTRIBUTING.md has the setup steps (pip install -e ".[dev]", pytest, ruff check .). A free next issue is #120 (check --not-null "" passes with nothing checked), labeled good first issue.

If Gulliver is useful to you, a ⭐ helps other people find it.


Generated by Claude Code

Signed-off-by: Masfiquil Islam <shaw778@users.noreply.github.com>
@shaw778

shaw778 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

I have dropped the CHANGELOG.md changes per the maintainer policy (git checkout origin/main -- CHANGELOG.md). The PR is updated and ready for your review. Thank you!

Copy link
Copy Markdown
Owner

Thanks @shaw778. Commit 0a2b06a is meant to drop the CHANGELOG edit, but the PR's file list still shows CHANGELOG.md with 14 added lines, so the change does not seem to have reached the head. Could you check that the push went through? If not, git fetch origin and then git checkout origin/main -- CHANGELOG.md and push. The maintainer will review the new head once it is in.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lint --format markdown: a findings table for GITHUB_STEP_SUMMARY and PR comments (a new report format)

2 participants