Skip to content

feat(rpk-docs): refresh a single plugin's docs with --plugin - #225

Merged
JakeSCahill merged 37 commits into
mainfrom
feat/rpk-plugin-docs
Jul 31, 2026
Merged

feat(rpk-docs): refresh a single plugin's docs with --plugin#225
JakeSCahill merged 37 commits into
mainfrom
feat/rpk-plugin-docs

Conversation

@JakeSCahill

@JakeSCahill JakeSCahill commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

rpk plugins release on their own cadence, independent of rpk itself: connect ships ~weekly from redpanda-data/connect, rpk ai several times a week from cloudv2's adp/v* tags, rpk k8s per operator release, and rpk check from redpanda-data/redpanda-check. Their docs currently refresh only when a Redpanda release triggers a full rpk regeneration.

This adds a --plugin mode so a single plugin's docs can be regenerated on the plugin's own release schedule:

doc-tools generate rpk-docs --plugin connect --plugin-version 4.102.0 --from-json docs-data/rpk-v26.1.12.json

How it works (splice-and-rerender)

  1. Acquire an rpk binary matching the snapshot's rpk_version — official release download for published tags, build-from-source fallback for RC snapshots (RC releases are drafts, so their assets are not publicly downloadable). --rpk-bin skips both for local runs.
  2. Install only the target plugin in an isolated HOME (pinned via the plugin's version flag when --plugin-version is given), run rpk --print-tree, and extract the plugin's top-level node.
  3. Reapply platform markers from the snapshot's recorded linux_only_commands, splice the node into the snapshot (replace-only), record the version in plugin_versions, and save.
  4. Re-render the full tree with normal --from-json semantics. Only the plugin's pages change in git, and stale-file cleanup, the nav rebuild, and override validation all see a complete tree.

Also fixed along the way

  • plugin_versions plumbing (snapshot field → page context → "introduced in {plugin} version X" template) existed end-to-end but was never populated. --plugin records the refreshed version, the from-source path now records manifest versions for plugins that actually installed, and --from-json passes recorded versions through instead of hardcoding {}.
  • rpk's install version validation caps each version segment at two digits (^v?\d{1,2}\.\d{1,2}\.\d{1,2} in pkg/cli/connect/install.go), so pins like connect 4.102.0 are rejected. The installer falls back to latest and records the manifest-resolved version. Upstream bug worth filing against rpk.

Validation (against real docs snapshots)

  • --plugin connect on main's rpk-v26.1.12.json: only the snapshot, nav, and rpk-connect/ pages change; platform markers preserved; second run changes only generated_at.
  • --plugin k8s --plugin-version 26.2.1-beta.3 on beta's rpk-v26.2.1-rc2.json: exercises the source-build fallback and the pre-GA pin, and restores the 5 rpk k8s multicluster pages that the rc2 regen deleted (redpanda-data/docs#1831).
  • --plugin k8s against main's snapshot: clean skip with exit 0 (no k8s command in rpk 26.1).
  • 298 jest tests pass, including new coverage for the splice and shim-detection logic.

Consumer side: redpanda-data/docs gains an update-rpk-plugin-docs workflow that calls this mode, with senders in each plugin's release workflow.

Test it locally

Everything runs against the real committed snapshots in the docs repo — no Docker, no Redpanda build. You need Node 20+, this branch, and a checkout of redpanda-data/docs on main.

# 1. This branch
git clone https://github.com/redpanda-data/docs-extensions-and-macros
cd docs-extensions-and-macros
git checkout feat/rpk-plugin-docs
npm install
DOC_TOOLS=$PWD/bin/doc-tools.js

# 2. From a clean docs-repo checkout (main):
cd /path/to/docs

A. Release diff → What's-new + PR summary (the change-detection overhaul, ~1 min):

node $DOC_TOOLS generate rpk-docs \
  --from-json docs-data/rpk-v26.2.1-rc2.json \
  --diff v26.1.12 \
  --update-whats-new \
  --summary-file /tmp/pr-summary.md

Then look at:

  • The console diff report: Deprecated commands: 1rpk redpanda admin with the rpk cluster migration message (the old pipeline reported these as 12 removals).
  • /tmp/pr-summary.md: the change table now includes changed flag defaults / descriptions, each with itemized lists.
  • git diff modules/get-started/pages/release-notes/redpanda.adoc: a // AUTOGEN-RPK-CHANGES … START/END block appended inside the existing hand-written == Redpanda CLI (rpk) section, which is left untouched. Re-run the command — the block is replaced, not duplicated.

B. Single-plugin refresh (the --plugin mode, ~1–2 min, downloads a released rpk binary + the plugin):

git checkout -- . && git clean -fd modules docs-data
node $DOC_TOOLS generate rpk-docs \
  --plugin ai \
  --from-json docs-data/rpk-v26.2.1-rc2.json \
  --update-whats-new

Use the newest snapshot on the branch (what the workflow auto-selects). Running against an older snapshot regenerates pages for an rpk line the branch has moved past. Expect one-time modifications across all rpk pages on the first 5.3.0 run (the :description:-in-tag template change), Preserving rpk-check/ … rpk-k8s/ log lines (shim-only plugins in the rc2 snapshot are protected, not deleted), and exactly two deletions: the stale rpk-ai-llm.adoc/rpk-ai-mcp-create.adoc partials that redpanda-data/docs#1849 also removes.

Then look at:

  • git status: only the snapshot, rpk-ai partials, nav, overrides, and the release-notes page change — nothing else.
  • The snapshot's plugin_versions now records the installed ai version, and new plugin commands got introducedInVersion stamped with the plugin version in docs-data/rpk-overrides.json.
  • The release-notes page gained a == rpk plugins section with an === ai plugin <version> block (plain command names, no xrefs — ai renders as partials).

C. Unit tests: npm test in this repo (797 tests; the rpk-docs suites are npx jest __tests__/tools/rpk-docs/).

Reset the docs repo with git checkout -- . && git clean -fd modules docs-data when done.

Related Jira

  • DOC-2355 (rp-connect-docs#132): users asked for a reference listing the Connect CLI's commands, arguments, and options. The plugin flag extraction here (cobra + urfave/cli parsers) makes those options documentable for the first time, and the rpk-plugin-stubs reconciler can publish the reference on the RPCN site via page-family single-source stubs (workflows drafted, pending the RPCN team's placement decision).
  • DOC-1090: groundwork for the deprecations redesign — rpk command deprecations are now detected automatically from source, reported in What's-new and PR summaries with migration messages, and page deprecation banners are auto-annotated in overrides.

Related PRs (rpk docs automation train)

PR Role Depends on
#225 Generator: --plugin mode, change detection, deprecations, What's-new merge, flag extraction, stub reconciler (publishes doc-tools 5.3.0)
redpanda-data/docs#1834 Receiver workflow for plugin-release dispatches #225
redpanda-data/docs#1844 Release workflow: auto What's-new diff base + pre-GA plugin pins #225
redpanda-data/docs#1849 Removes two stale rpk ai partials merge before #1852/adp-docs#165 activate
redpanda-data/docs#1852 Sender: docs main → adp-docs stub sync dispatch #225, adp-docs#165
redpanda-data/adp-docs#165 Receiver: reconciles rpk ai stubs and nav #225, docs#1852
Senders (merged/open) redpanda-data/connect#4636, redpanda-operator#1703 (merged), redpanda-check#10, cloudv2#28552 docs#1834

Merge order: #225 → dependency bumps on docs main and beta → #1834 + #1844 (+ #1849) → #1852 + adp-docs#165 → remaining senders. Jira: DOC-2355, DOC-1090.

rpk plugins (connect, ai, k8s, check) release on their own cadence,
independent of rpk itself, so their docs go stale between Redpanda
releases. Add a --plugin mode that installs one plugin against an rpk
binary matching the committed snapshot's version, splices the fresh
subtree into the snapshot, records the plugin version, and re-renders
the full tree. Re-rendering everything keeps stale-file cleanup, nav
rebuilds, and override validation correct: only the plugin's pages
change in git.

The rpk binary comes from the official release download when the
snapshot points at a published release, with a build-from-source
fallback for RC snapshots (RC releases are drafts, so their assets
are not publicly downloadable). --rpk-bin skips both for local runs.

Also populate plugin_versions, which the page templates already
support but nothing ever fed: --plugin records the refreshed version,
the from-source path records manifest versions for plugins that
installed, and --from-json now passes recorded versions through
instead of hardcoding {}.
… pins

The fresh plugin subtree from --print-tree carries no platforms fields,
so splicing it dropped :page-platforms: from every plugin page. Reapply
markers using the snapshot's recorded linux_only_commands.

rpk's install version validation caps each version segment at two
digits, so pins like connect 4.102.0 are rejected. Fall back to
installing latest (at dispatch time latest is the release that
triggered the run) and record the manifest-resolved version instead of
the failed pin.
@netlify

netlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

Deploy Preview for docs-extensions-and-macros ready!

Name Link
🔨 Latest commit 610e2d9
🔍 Latest deploy log https://app.netlify.com/projects/docs-extensions-and-macros/deploys/6a6c79d759e3f80008f4466b
😎 Deploy Preview https://deploy-preview-225--docs-extensions-and-macros.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 97085774-bcc4-4f02-a5bd-d85923ab30ac

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds plugin-scoped rpk documentation refreshes from committed snapshots. The handler can acquire or build an rpk binary, install a selected plugin, extract and splice its command subtree, track plugin versions, update snapshots, and generate plugin-specific summaries. The CLI exposes plugin, version, and binary-path options and validates snapshot usage. Tests cover subtree detection, replacement, immutability, errors, and plugin mappings. The package version is updated to 5.3.0.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant handleRpkDocsGeneration
  participant acquireRpkBinary
  participant fetchPluginSubtree
  participant Snapshot
  CLI->>handleRpkDocsGeneration: provide plugin refresh options
  handleRpkDocsGeneration->>acquireRpkBinary: acquire or build rpk
  handleRpkDocsGeneration->>fetchPluginSubtree: install plugin and extract commands
  fetchPluginSubtree-->>handleRpkDocsGeneration: refreshed subtree and version
  handleRpkDocsGeneration->>Snapshot: splice subtree and save versioned JSON
  handleRpkDocsGeneration-->>CLI: return plugin diff and summary
Loading

Possibly related PRs

Suggested reviewers: feediver1

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly names the main change: adding --plugin support to refresh a single rpk plugin's docs.
Description check ✅ Passed The description directly explains the new single-plugin docs refresh mode and its implementation details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/rpk-plugin-docs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
__tests__/tools/rpk-docs/plugin-refresh.test.js (2)

81-85: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert the exact plugin-to-flag mapping.

The regex confirms only flag-shaped values; it would still pass if every plugin used the wrong flag. Assert the expected mapping explicitly: ai: --ai-version, check: --check-version, connect: --connect-version, and k8s: --plugin-version.

Proposed test adjustment
-      for (const plugin of REFRESHABLE_PLUGINS) {
-        expect(PLUGIN_INSTALL_VERSION_FLAGS[plugin]).toMatch(/^--[a-z-]+$/)
-      }
+      expect(PLUGIN_INSTALL_VERSION_FLAGS).toEqual({
+        ai: '--ai-version',
+        check: '--check-version',
+        connect: '--connect-version',
+        k8s: '--plugin-version'
+      })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@__tests__/tools/rpk-docs/plugin-refresh.test.js` around lines 81 - 85, Update
the “every refreshable plugin has a version pin flag” test to assert the exact
PLUGIN_INSTALL_VERSION_FLAGS mapping for ai, check, connect, and k8s:
--ai-version, --check-version, --connect-version, and --plugin-version
respectively, while preserving the existing coverage that each refreshable
plugin has a flag.

69-73: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert complete input immutability.

This test checks only one nested field. Capture the full tree before calling splicePluginNode and compare the entire input afterward so mutations to sibling nodes, arrays, or top-level fields cannot pass unnoticed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@__tests__/tools/rpk-docs/plugin-refresh.test.js` around lines 69 - 73, Update
the “does not mutate the input tree” test around splicePluginNode to deep-clone
or otherwise snapshot the complete baseTree before invocation, then assert the
entire tree is unchanged afterward. Replace the single nested description check
while preserving the existing splicePluginNode inputs and behavior under test.
tools/rpk-docs/rpk-docs-handler.js (2)

1290-1302: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Isolated HOME (and the binary work dir) is never cleaned up.

tmpHome holds a full plugin install (connect is hundreds of MB) and, like workDir in acquireRpkBinary, is left behind after every run. A try/finally with fs.rmSync(tmpHome, { recursive: true, force: true }) around the install + --print-tree would keep repeated CI runs from accumulating temp state.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/rpk-docs/rpk-docs-handler.js` around lines 1290 - 1302, Update
fetchPluginSubtree to wrap the plugin installation and --print-tree workflow in
a try/finally, removing tmpHome with fs.rmSync(tmpHome, { recursive: true,
force: true }) in the finally block. Also ensure the binary work directory
managed by acquireRpkBinary is cleaned up through the same guaranteed cleanup
path.

1553-1567: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Snapshot is overwritten before rendering succeeds.

saveVersionedJson runs ahead of generateRpkDocs, so a rendering failure leaves the committed snapshot advanced to the new plugin version while the generated pages still reflect the old subtree. Moving this write to after a successful generateRpkDocs keeps snapshot and docs in step.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/rpk-docs/rpk-docs-handler.js` around lines 1553 - 1567, Move the
saveVersionedJson call in the plugin refresh flow to after generateRpkDocs
completes successfully. Keep the jsonData updates for the enhanced tree and
plugin_versions associated with the refreshed snapshot, but only persist them
once rendering succeeds so failed generation does not advance the committed
baseline.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/rpk-docs/rpk-docs-handler.js`:
- Around line 1507-1514: Update the snapshot-saving logic for augmentatedData to
persist tree.linux_only_commands alongside raw_tree/tree, plugin_versions, and
deprecated_commands. Ensure from-json inputs restore this field so
addPlatformMarkersFromSource in the refreshed subtree path retains Linux-only
platform markers.

---

Nitpick comments:
In `@__tests__/tools/rpk-docs/plugin-refresh.test.js`:
- Around line 81-85: Update the “every refreshable plugin has a version pin
flag” test to assert the exact PLUGIN_INSTALL_VERSION_FLAGS mapping for ai,
check, connect, and k8s: --ai-version, --check-version, --connect-version, and
--plugin-version respectively, while preserving the existing coverage that each
refreshable plugin has a flag.
- Around line 69-73: Update the “does not mutate the input tree” test around
splicePluginNode to deep-clone or otherwise snapshot the complete baseTree
before invocation, then assert the entire tree is unchanged afterward. Replace
the single nested description check while preserving the existing
splicePluginNode inputs and behavior under test.

In `@tools/rpk-docs/rpk-docs-handler.js`:
- Around line 1290-1302: Update fetchPluginSubtree to wrap the plugin
installation and --print-tree workflow in a try/finally, removing tmpHome with
fs.rmSync(tmpHome, { recursive: true, force: true }) in the finally block. Also
ensure the binary work directory managed by acquireRpkBinary is cleaned up
through the same guaranteed cleanup path.
- Around line 1553-1567: Move the saveVersionedJson call in the plugin refresh
flow to after generateRpkDocs completes successfully. Keep the jsonData updates
for the enhanced tree and plugin_versions associated with the refreshed
snapshot, but only persist them once rendering succeeds so failed generation
does not advance the committed baseline.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: dd2c6f1e-a0cb-439e-a616-6c9847aa9419

📥 Commits

Reviewing files that changed from the base of the PR and between 4ee4074 and ab3f102.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (5)
  • CLI_REFERENCE.adoc
  • __tests__/tools/rpk-docs/plugin-refresh.test.js
  • bin/doc-tools.js
  • package.json
  • tools/rpk-docs/rpk-docs-handler.js

Comment thread tools/rpk-docs/rpk-docs-handler.js
New commands found during a --plugin refresh now get
introducedInVersion set to the plugin version that shipped them, so
pages render "This command was introduced in <plugin> version X". The
full-regen path gets the same treatment: plugin-subtree commands are
stamped with the plugin's version instead of the rpk version, which
would have rendered a Redpanda version masquerading as a plugin
version. Stamping happens before the overrides load, so notes appear in
the same run's output. Existing introducedInVersion values are never
overwritten.

Verified against main's snapshot: an ai refresh stamped 30 new commands
with 0.2.31 and the rendered partials carry the note.
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

Added introduced-in stamping (cd21b9c): new commands found during a --plugin refresh now get introducedInVersion set to the plugin's version, so pages render "This command was introduced in <plugin> version X." The full-regen path is fixed the same way — plugin-subtree commands were previously stamped with the rpk version, which would have rendered a Redpanda version masquerading as a plugin version now that pluginName is populated.

Verified against main's real snapshot: an ai refresh stamped 30 new commands with 0.2.31 and the rendered partials carry the note. Existing introducedInVersion values are never overwritten.

One granularity caveat: the diff baseline is the previous snapshot state, not per-release history — if several plugin releases pass between doc regenerations, a command is stamped with the version at which the docs first captured it. Once the release-triggered senders are live, that converges to one-release accuracy.

JakeSCahill and others added 5 commits July 29, 2026 08:55
Full regenerations install the latest published version of every
plugin, so beta-only plugins (no version promoted to latest in the
manifest) install nothing and their commands are absent from the tree —
how the k8s pages went missing during the 26.2 beta window. A
repeatable --plugin-pin name=version option now pins individual plugin
installs, with the same retry-without-pin fallback as --plugin-version,
and pinned versions are recorded in plugin_versions.
…summaries

The diff engine computed flag type, required, and description changes
and then discarded them. The PR summary read a summary field that never
existed (changedDescriptions vs descriptionChanges), never rendered
changed defaults at all, and claimed 'no changes' for releases whose
only changes were defaults or descriptions. The console report labeled
removals as deprecations.

Collect all computed flag changes into the diff, render every category
in the PR summary with itemized collapsible lists, extend the What's
new section with removed commands/flags and flag type changes, label
removals as removals, and JSON-encode structured default values so
arrays never render as [object Object].

Verified against the real v26.1.12 -> v26.2.1 diff: surfaces a
previously invisible default change (rpk container start
--console-image v3.7.1 -> v3.8.0), 1 flag description change, and 35
command description changes alongside the 53/33/23 command and flag
counts.
The source scanner's deprecated_commands output was stored in every
snapshot and never read again: deprecations were invisible to the diff,
What's new, and PR summaries, while hidden-deprecated commands were
misreported as removals (a command family that still works as aliases
looked deleted).

Diff the deprecation maps between snapshots to produce a real
newly-deprecated category carrying the deprecation message and
replacement, reclassify removals under a newly deprecated path as part
of that deprecation (with the affected subcommands listed), and render
deprecations in the console report, the What's new section, and the PR
summary. Visible deprecated commands are also annotated in the
overrides file automatically so their pages carry deprecation banners;
hidden ones are excluded because they have no pages to annotate.

Verified against the real v26.1.12 -> v26.2.1 diff: the rpk redpanda
admin family reports as 4 deprecations with the rpk cluster migration
message, and removals drop from 33 to 21.
The What's-new updater refused to write when the page already had any
Redpanda CLI section, so only the first run per release-notes page ever
landed: later RCs in a beta cycle were silently dropped, re-runs could
never refresh, and the hand-written section from the 26.2 cycle blocked
the v26.2.1 GA entry entirely.

Wrap generated content in version-scoped AUTOGEN-RPK-CHANGES marker
blocks. Re-runs for the same version replace their own block, later
versions append inside the existing section, hand-written prose is
never touched, and a missing section is created at the standard
insertion points.

Plugin refreshes now write What's-new blocks too (labeled with the
plugin and its version) when --update-whats-new is passed — plugin
changes never arrive through a Redpanda release diff, so this was their
only path onto the page. Plugin entries render without xrefs because
plugin subtrees may render as partials, and Redpanda-release entries
now skip xrefs for excluded and partial-routed commands (rpk ai links
pointed at pages that do not exist). New-command bullets use only the
first line of help text so multi-line descriptions stop breaking the
list.

Verified on the real release-notes page: the hand-written 26.2 section
survives, a v26.2.1 block and an ai plugin 0.2.31 block accumulate
inside it, and the deprecation entry renders as one rolled-up
rpk redpanda admin item with all 11 subcommands listed.
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

Three more commits landed from the change-detection review (af8a754, d09c554, 2fba250):

Phase 1 — reporting correctness: the PR summary read a nonexistent summary field (changedDescriptions), never rendered flag default changes, and reported 'no changes' when only defaults or descriptions changed. Flag type/required/description changes were computed by the differ and discarded. All categories now render with itemized collapsible lists, and the console report labels removals as removals instead of deprecations.

Phase 2 — real deprecation detection: deprecated_commands was write-only. The differ now diffs the deprecation maps between snapshots, reclassifies hidden-deprecated subtree removals as deprecations (rolled up into one entry per family with subcommands listed), and renders them in the console report, What's-new, and PR summaries. Visible deprecated commands are auto-annotated in the overrides so pages get deprecation banners.

Phase 3 — mergeable What's-new: the one-shot == Redpanda CLI guard is replaced with version-scoped AUTOGEN-RPK-CHANGES marker blocks: same-version re-runs replace their own block, later versions (successive RCs, plugin releases) append inside the existing section, and hand-written prose is never touched. Plugin refreshes now write plugin-labeled What's-new blocks. Entries link only to commands that render as pages (rpk ai partials no longer get broken xrefs) and use first-line descriptions.

All phases verified end-to-end against the real v26.1.12→v26.2.1 diff and the real release-notes page: the rpk redpanda admin family reports as one deprecation with the rpk cluster migration message (previously 12 fake removals), a real default change (rpk container start --console-image) surfaces, and the hand-written 26.2 CLI section accumulates v26.2.1 and ai-plugin blocks without being clobbered. 322 tests.

…tion

Plugin releases are not tied to the Redpanda version the What's new
page is about, so mixing their blocks into the Redpanda CLI section
implied the changes shipped with that release. Plugin blocks now land
in their own '== rpk plugins' section, and every autogen block opens
with a '=== <version>' heading (categories nest at ====), so
accumulated blocks from successive RCs or plugin releases never
produce colliding section ids.
…defects

Plugin subtrees come from --help-autocomplete, which carries no flag
data, so every plugin command page rendered an empty flags section —
rpk ai llm-provider create documented none of its two dozen flags
(reported in adp-docs#161 review). The plugin binary is installed
during generation, so harvest each command's --help output: parse the
local Flags section (shorthand, value type, default, wrapped
descriptions), skip rpk-core global flags, and leave the compiled-in
shim commands alone. Wired into both the single-plugin refresh and the
Docker full-generation path; help failures are non-fatal.

Also from the same review:
- Sentence extraction ate leading text around decimal versions:
  'Run the OAuth 2.0 device...' became '0 device...' because the
  sentence regex requires whitespace after punctuation and silently
  drops unterminated prefixes. Protect decimals and glue back any
  skipped prefix, in both capToTwoSentences and shortDescription.
- Repeat :description: inside the single-source tag region so stub
  consumers inherit meta descriptions; page-aliases stays outside the
  tag so consumers cannot inherit alias registrations.
- Collapse blank-line runs left by absent optional sections (plugin
  pages had 3-5 empty lines between Usage and Examples), preserving
  blank lines inside delimited blocks.

Verified with a real ai plugin refresh: 52 commands enriched, 54 pages
now carry a local Flags section (was 2), rpk ai llm-provider create
renders 23 flags, rpk ai auth login documents --no-browser, and its
description reads 'Run the OAuth 2.0 device authorization grant...'
in full.
…odeRabbit)

The Linux-only command list comes from Go build-tag scanning of the rpk
source, which from-json runs (including --plugin refreshes) never see.
A new preserveLinuxOnlyCommands helper inherits the list from whichever
stored tree carries it, both when deriving the working tree and right
before the refreshed snapshot is saved, so platform markers survive
every from-json round trip. Covered by new unit tests.

@micheleRP micheleRP left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and tested locally: full suite passes on this branch (804/804, incl. the 329 rpk-docs tests), and I reproduced both "Test it locally" scenarios against a clean docs main checkout — the deprecation roll-up (1 deprecation for rpk redpanda admin with 11 subcommands, vs 12 phantom removals before), the itemized default changes, the marker-block What's-new merge (idempotent on re-run, hand-written content untouched), and the --plugin ai refresh (isolated HOME, checksum-verified binary download, plugin_versions recorded, platform markers preserved). The splice-and-rerender design is solid. Two issues in the generated What's-new output, one suggestion:

1. Broken xrefs for command-group roots. commandPathToXref('rpk check') returns rpk-check.adoc, but command groups render to rpk-check/rpk-check.adoc (verified: the top-level file doesn't exist). The v26.2.1 GA What's-new will publish broken xrefs for the rpk check and rpk k8s roots. Since the tree is in hand at render time, 2-part commands with children can be routed to <dashified>/<dashified>.adoc.

2. makeLinkablePredicate misses the cloudSecretDir routing. It checks the overrides-based routing (shouldExcludeCommand/shouldUsePartialDir) but not the hardcoded rpk cloud/rpk security secret → partials path in generate-rpk-docs.js. Result (verified in my Test A run): the What's-new contains xref:reference:rpk/rpk-cloud/rpk-cloud-auth-list.adoc and friends — pages that don't exist in this repo (that content is single-sourced into cloud-docs stubs). Cloud-routed commands need to be treated as non-linkable too.

3. (Suggestion) Sentence-boundary capping for What's-new bullets. The first-line-only description logic cuts wrapped cobra help mid-sentence on the published release-notes page, e.g. rpk ai llm-provider diff — "Prints, per manifest, whether apply would" and rpk ai policy create — "the allow/deny gate that". Capping at a sentence boundary (the shortDescription/capToTwoSentences helpers already exist) would read much better.

One CodeRabbit triage note: its "persist linux_only_commands when saving the snapshot" finding is a false positive — the field is persisted on raw_tree itself (103 entries in the committed 26.1.12 snapshot, including the rpk ai commands), and my --plugin ai test preserved all page-platforms markers.

Heads-up that also affects the consumers: the command.hbs :description: change means the first regeneration with 5.3.0 touches every rpk page (~370 files in my runs) — see my comment on redpanda-data/docs#1834 about expectations for the first automated PR.

@JakeSCahill
JakeSCahill requested a review from a team July 30, 2026 07:43
JakeSCahill and others added 8 commits July 30, 2026 10:46
…oundaries

From review on #225:
- Command-group roots (rpk check, rpk k8s) render into their own
  directory, but commandPathToXref linked them at the top level, so the
  What's-new published broken xrefs for every new command group. Route
  two-part commands with subcommands to <group>/<group>.adoc using the
  tree in hand at render time.
- makeLinkablePredicate only consulted overrides routing, missing the
  hardcoded rpk cloud / rpk security secret partials routing, so the
  What's-new linked pages that exist only as cloud-docs partials. Those
  commands now render as plain names.
- New-command bullets took the first line of help text, which cuts
  wrapped cobra output mid-sentence. Cap at the first sentence boundary
  instead, with decimal-version protection.
Module downloads from proxy.golang.org fail transiently, and a
post-crash Docker daemon has been observed returning success for a
build that produced no binary, which sent every subsequent exec against
a missing /tmp/rpk. Retry the in-container build up to three times
(later attempts reuse the partially filled module cache) and verify
/tmp/rpk exists before proceeding.
…ails

The dual-build platform detection wrapped both builds in one try:
a native comparison-build failure (local Go older than go.mod requires)
was reported as a Docker failure, discarded the successful Linux tree,
and died in the native fallback for the same reason. The Linux tree is
authoritative; a comparison failure now only skips dynamic platform
detection.
docker exec intermittently returns zero for a build that produced no
binary (reproduced twice on Docker Desktop 29.6.1). The existence check
ran after the retry loop, so a phantom success failed the run instead
of retrying. Verify the binary inside the loop and treat a zero exit
with no binary as a failed attempt.
adp-docs publishes rpk ai through one static stub page per command plus
a nav entry. A plugin release that adds a command leaves it invisible on
the ADP site (no stub), and one that removes a command leaves a stub
with an unresolved include (broken page) — the manual follow-up flagged
in review.

Add doc-tools generate rpk-plugin-stubs: reconciles a consumer repo's
stubs and nav against the docs repo's generated partials. Creates stubs
for new partials (title read from the partial — dashified filenames
cannot be reversed unambiguously), deletes managed stubs whose partial
is gone, never touches pages that do not match the managed-stub shape,
rebuilds the plugin's nav block hierarchically, and proposes page
aliases for likely renames (same parent, same depth, related last
words) for the reviewer to confirm. Full reconcile rather than a diff,
so it is idempotent and heals pre-existing drift.

Verified dry-run against real adp-docs + docs main: detects exactly the
two stubs that would resurrect from the stale partials docs#1849
removes, and nothing else.
The subcommand list on parent pages filtered overrides-based routing
but not the hardcoded rpk cloud / rpk security secret partials routing,
so rpk-security.adoc linked rpk-security-secret.adoc — a page that does
not exist. Found by building the full site from generated output; the
broken xref was pre-existing on the published site.
Flag extraction parsed only cobra's Flags: section, so connect (built
on urfave/cli, which prints OPTIONS:) extracted zero flags while ai and
k8s worked. Add a urfave parser (comma shorthands, value placeholders,
repeatable [ --x value ] notation, (default: ...) suffixes, GLOBAL
OPTIONS skipped) and dispatch on the section header. Verified: rpk
connect run renders 15 flags from real plugin help.
…headings

Ground-truth review of a full v26.2.1 generation against 348 --help dumps
found two rendering defects:

- Upstream help strings that end mid-example (rpai --order-by, --clear)
  rendered as 'for example,.'. Strip a dangling 'e.g.' that has nothing
  after it, and remove stray spaces before closing parentheses
  (rpk group offset-delete -t).
- Pages can render the same level-2 heading twice when override content
  collides with generated sections (rpk container status Example, fixed
  in the docs overrides). The generator now warns per page so this class
  is caught at generation time.
…aph breaks

Antora build of a full v26.2.1 generation surfaced both:

- The nav rebuild dropped the 'xref:reference:rpk/rpk.adoc[]' entry
  (the root command page is skipped by the per-command entry loop), so
  Antora reported the page as unlisted. The root page is now the first
  static nav entry, matching the hand-maintained nav on main.
- What's-new command summaries glued a periodless cobra summary line to
  the paragraph after it ('Install Redpanda Check This command
  installs...'). firstSentence now cuts at the first blank line before
  joining hard-wrapped lines, which preserves wrapped single-paragraph
  descriptions.
The v26.1.12 baseline snapshot predates plugin flag extraction, so every
plugin subcommand recorded zero flags. Diffing against it marked all 208
extracted ai/connect flags as new and stamped them 'New in 0.2.32' /
'New in 4.102.0', even for long-standing flags like rpk connect blobl
--pretty.

generateRpkDiff now treats a pre-existing command whose baseline recorded
no flags as documentation backfill: its flags are excluded from newFlags
(so no New-in labels and no What's-new entries) and reported in a
separate flagDataBackfilled category that surfaces in the diff report and
PR summary. Genuinely new commands and commands with baseline flag data
are unaffected, as are description-change checks on backfilled commands.
…ring in from-json runs

Two refinements from verifying the backfill guard against the real
v26.1.12 to v26.2.1 snapshots:

- The guard now only treats a zero-flag baseline as backfill when the
  command's whole top-level group recorded no flag data outside the
  rpk-native install/uninstall/upgrade shims. Six core commands (for
  example rpk cluster config status) legitimately gained their first
  flag in v26.2.1 and were being swallowed. Result on real data: 23
  genuine core flag additions reported, 50 plugin commands classified
  as backfill, zero misfires either way.
- --from-json --diff runs now stamp introducedInVersion into the
  overrides before the overrides load, so the same run renders the
  labels instead of deferring them to the next regeneration. The
  from-source stamping gate also fires on flag-only releases now, not
  just when new commands exist.
…les verbatim

Ground-truth review against rpk v26.2.1 --help output found the generator
mangling help text it had no rule for:

- Column-0 '$ command' examples with sample output (rpk cluster brokers
  decommission-status) rendered as prose: the invocation got backticks
  and a period, the output's =-underlined title became a spurious
  section heading, and the ASCII table leaked into the Usage section.
  These now render as a [,bash] command block plus a [.no-copy] output
  block, and parseDescriptionSections no longer treats an all-caps line
  directly after a $ invocation as a section header.
- Indented code introduced by a colon (Cedar policies on rpk ai policy
  create, rpai command examples) rendered as prose with backticks
  injected into the code. These are captured verbatim into [,text]
  blocks. Column-aligned definition layouts are left to the existing
  indented-table converter, which renders them as tables.
- 'e.g.:' rendered as 'for example, :'.
- Summaries (:description:, subcommand tables) now strip captured code
  blocks, and a block introduced by a colon ends the summary at that
  sentence.
- textTransformations rules gain an opt-in applyToCode flag so the
  rpai -> rpk ai binary-name rewrite reaches code blocks while
  admonition and backtick rules never touch verbatim content.
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

Ran a full ground-truth review of the generated output: production generation of v26.2.1 (diff v26.1.12) on a pristine docs clone, compared against --help dumps for all 348 commands from the same rpk binary and plugin installs, using an independent checker rather than the generator's own parsers. Also parsed all 364 generated files with Asciidoctor and built the full Antora site.

Fixes pushed from what the review found (98a1e65..a1c0da8):

  • Dangling example fragments: upstream help strings ending mid-example rendered as "for example,." Now stripped, and stray spaces before closing parens are cleaned.
  • Duplicate rendered headings now warn at generation time (the one real case, rpk container status, is an overrides bug fixed in redpanda-data/docs#1860).
  • The nav rebuild dropped the root rpk.adoc entry, which made Antora report the page as unlisted. The nav now round-trips byte-identical to main.
  • What's-new summaries no longer glue a periodless summary line to the paragraph after it.
  • Baseline gap guard: the v26.1.12 snapshot predates plugin flag extraction, so every extracted ai/connect flag (208 of them) was labeled 'New in ' even for long-standing flags. Flag additions in a plugin group whose baseline has no flag data are now reported as documentation backfill, not new flags. Verified against the real snapshots: exactly the 23 genuine core flag additions survive, zero plugin mislabels, zero core misfires.
  • from-json --diff runs now stamp introducedInVersion before rendering, so a single run produces labeled pages (previously labels appeared only on the next regeneration).
  • Unindented $ command examples with sample output (rpk cluster brokers decommission-status) and colon-introduced code samples (Cedar policies on rpk ai policy create) are now captured as verbatim code blocks instead of being mangled into prose with injected backticks and fake headings. Column-aligned definition layouts still go to the table converter. Summaries strip captured blocks. textTransformations rules gain an opt-in applyToCode flag so the rpai→rpk ai rewrite reaches command examples while admonition rules never touch verbatim content.

Final state: 856 tests green, 0 critical findings from the mechanical checker, 364/364 files parse clean under Asciidoctor, full Antora build has zero errors or warnings attributable to the generated content, and the What's-new block was independently verified faithful to the snapshot diff (53/53 new commands, 23/23 new flags, all xrefs resolve).

The review also surfaced pre-existing content issues in hand-maintained docs (not from this PR): redpanda-data/docs#1860 and redpanda-data/docs#1861 fix the unambiguous ones.

…scription-coverage reporting

Two more findings from the ground-truth content review:

- The examples pipeline applied every textTransformations rule to raw
  examples content before code-block wrapping, so a caption rule
  rewrote '{"quotas":...}' inside an example invocation to
  '{`quotas`:...}'. Examples content is now transformed line by line:
  indented command lines only receive rules flagged applyToCode.
- Structured example captions ending in a period rendered with two
  ('brokers 1 and 2..'): the template appended an unconditional period.
  Both example templates now use the ensurePeriod helper.
- New PR-summary section lists description overrides that replace
  substantially longer source help (counting appendToDescription), so
  curated-content drift stays visible for review on every regen PR
  instead of hiding until the next audit.
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

Follow-up commits from completing the content-review fixes (338ba80):

  • The examples pipeline now transforms content line by line, so caption-oriented rules can never rewrite quoted strings inside example commands (a rule was corrupting '{"quotas":...}' to '{quotas:...}'). Rules flagged applyToCode still reach command lines.
  • Structured example captions use ensurePeriod instead of an unconditional trailing period (was rendering "brokers 1 and 2..").
  • New PR-summary section, "Curated descriptions replacing source help": lists every description override that replaces substantially longer source help (counting appendToDescription). This is the guard against silent curated-content drift. On current docs main it flagged the ~30 stale overrides now fixed in redpanda-data/docs#1862, and after that PR it lists only the six intentionally curated commands.

860 tests green. The full content-restoration companion is redpanda-data/docs#1862.

…laceholder braces in prose

The Antora build of the restored-content corpus (redpanda-data/docs#1862)
surfaced two rendering defects:

- 4+ space indented literal chunks without a colon introducer (the
  'kafka/{topic}/{partition}_{revision}/' path template in rpk cluster
  logdirs describe) rendered as prose, so Asciidoctor consumed the braces
  as attribute references and dropped them. Such chunks are now captured
  as code blocks when they start a blank-line-delimited chunk. Wrapped
  continuations of table rows and list items are excluded (a first
  attempt without the chunk-start guard broke the license-info field
  table and the txn state list, caught by corpus diffing).
- Template placeholders like {name}_search in flag description prose
  (rpk ai mcp-server create/update) also rendered as attribute
  references. Prose segments now escape brace tokens, with {vbar}
  allowlisted because it is a real attribute used for pipe escaping.
  Backtick spans already had this protection.

Also fixes rpk profile prompt config samples (backticks were injected
into raw prompt strings) and the spurious trailing period on the
rpk cluster partitions move-status column list, both via the literal
capture.
JakeSCahill added a commit that referenced this pull request Jul 31, 2026
Both open questions were answered by rendering real connector data
(connect 4.103.0, 244 descriptions) with Asciidoctor:

Large-block mitigation: collapsible dropped in favor of passthrough
plus reporting. At the 1200-char threshold the collapsible fired on
five real connectors, and the worst case (redpanda_migrator, 6637
chars) has a hand-curated published page the migration would never
auto-swap anyway. Hiding primary content also depends on
browser-specific find and fragment auto-expand behavior in closed
details elements, and bodies carrying their own ==== delimiters could
never be wrapped (the http_server guard), so behavior diverged
silently per page. Heading demotion was rejected with direct evidence:
demoting snowflake_put's embedded headings produces six 'section title
out of sequence' warnings because the description renders before the
page's first == section. Passthrough matches the published pages'
structure exactly. The generator now reports long heading-less
descriptions during generation (10 current candidates) so structure
gets added upstream instead of being hidden by the docs build.

Summary placement: the partial now carries two tagged regions in one
file. tag=attrs holds a :description: attribute line (flattened to one
line via the new summaryAttribute helper: three summaries contain hard
breaks) for pages to include in their header, so search snippets and
meta descriptions refresh too. tag=body holds the summary, version
note, and rendered description for the body include. Verified with
Asciidoctor that a page consuming both tags gets the fresh attribute
and body with zero warnings.

Version 5.4.0: a feature bump that also sidesteps the 5.3.x patch
train (#225 takes 5.3.0, #228 re-bumps to 5.3.1).
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

@micheleRP thanks — all three landed in the commits since your review:

  1. Group-root xrefs: generateWhatsNewSection now routes two-part commands that have subcommands to <group>/<group>.adoc via a hasSubcommands predicate built from the live tree (report-delta.js:584-593). Independently verified during the full ground-truth review: all 61 xrefs in the generated v26.2.1 What's-new block resolve, including the rpk check and rpk k8s roots.

  2. Cloud-routed commands unlinkable: makeLinkablePredicate carries the hardcoded rpk cloud / rpk security secret partials routing (rpk-docs-handler.js:1071-1073). In the verified block, rpk cloud auth list and rpk security secret list render as plain code, no xref.

  3. Sentence-boundary capping: firstSentence now joins the hard-wrapped lines of the first paragraph before cutting at a sentence terminator (and cuts at the paragraph break when a cobra summary line has no period). Your two examples verbatim: rpk ai llm-provider diff renders "Prints, per manifest, whether apply would create, update, or delete the resource, without applying it." and rpk ai policy create renders the full sentence through "...may call a tool." Unit tests cover both shapes.

Since your review the branch also picked up the ground-truth review fixes (861+ tests now): flag-description truncation at wrapped -- continuations, the nav root-page drop, the plugin-flag "New in" mislabeling guard, verbatim capture of help code samples, and the curated-description coverage report. The summary comment from 2026-07-31 on this PR lists them with the verification state. Heads-up ack: the ~370-file first-regeneration churn expectation is noted in redpanda-data/docs#1834 and the re-dispatch plan folds #226/#229 in first so there is only one churn wave.

@micheleRP micheleRP left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-tested after your updates. Both blocking items are fixed, verified against real output rather than by reading the diff. Approving, with one follow-up below that is not a blocker.

Truncation: the dash guard is gone, replaced with an indentation comparison against the recorded flag column, and parseUrfaveFlags got the same treatment. The grep that returned 20 truncated rows yesterday now returns 0, and the rows that lost meaning read completely, with the flag tokens backticked:

--enabled ... (true when set; pass `--enabled=false` to disable).
--bedrock-config.region ... (alias: `--region`).
--bedrock-config.assume-role.role-arn ... (alias: `--role-arn`).

Duplicate Flags: solved by letting a curated override win with a warning, which I did not expect but works. rpk connect run renders 1 == Flags section, no page in the tree has more than one, and the warning fires clearly on a connect refresh naming the 13 skipped flags.

Also fixed: the PR summary now carries an ### Override Validation section reporting the 4 stale override paths, so they reach a reviewer instead of scrolling past in the log. The override-vs-source-help comparison table is a nice addition.

Tests are up from 361/836 to 387/862, all green. I re-parsed all 344 regenerated files with Asciidoctor: clean, no unterminated blocks.

Follow-up, not blocking: the container fallback cannot work on macOS

The new fallback builds in a container and then runs the binary on the host:

Native build failed (Go version mismatch: installed 1.24.4, required >= 1.26.4); building in a container...
Installing plugin: rpk ai install
Error: Failed to install plugin 'ai':
no output

docker run ... golang:<ver> go build -o /out/rpk produces a Linux binary (I confirmed the image reports GOOS=linux GOARCH=arm64) which is then executed on a Darwin arm64 host, so it cannot run and the plugin install reports no output. CI is unaffected, since a Linux runner with setup-go at stable satisfies the go.mod floor and would use the native build anyway. But the comment says old local Go is "common on laptops", and on macOS laptops this path fails, so it does not deliver what it is there for. It is also a less useful failure than before: the old behavior was a clear actionable Go-version error, this prints no output twice with the underlying exec error swallowed.

Two ways out: pass -e GOOS=darwin -e GOARCH=arm64 so the container cross-compiles for the host, or keep the binary in the container and exec into it for the plugin install and tree dump, the way fetchRpkTreeFromLinuxSource already does on the --ref path. Either way, surfacing the real exec error would help.

--rpk-bin remains a clean workaround, and it is what I used for all the plugin-refresh testing here.

Two docs-repo follow-ups, not changes to this PR

  • With the curated override winning, rpk connect run publishes 7 flags and skips 13, so --secrets, --redpanda-license, --disable-telemetry, --telemetry-deployment-type, --telemetry-tenant-id, and --rpc-plugins stay undocumented. Dropping that Flags override from docs-data/rpk-overrides.json adopts the extracted table, which is now the more complete one.
  • Of the 4 stale override paths the new report surfaces, rpk ai llm and rpk ai llm check are dead now that redpanda-data/docs#1849 merged, so they can go.

@JakeSCahill
JakeSCahill merged commit 964d09f into main Jul 31, 2026
22 checks passed
@JakeSCahill
JakeSCahill deleted the feat/rpk-plugin-docs branch July 31, 2026 15:13
JakeSCahill added a commit that referenced this pull request Jul 31, 2026
…-component xrefs

Integration fix now that #225 is on main: #225 dropped rpk cloud and
rpk security secret rows from parent subcommand tables entirely (a
cruder fix for the same broken-xref symptom this branch solves). The
rows now stay and this branch's isCloudSecretCommand routing links
them across to the cloud-data-platform component. Combined suite green
(866 tests), regenerated rpk-security.adoc table byte-identical to the
hand-fixed page from redpanda-data/docs#1837.
JakeSCahill added a commit that referenced this pull request Jul 31, 2026
* feat(rpk-docs): gate subcommand table rows by cloud availability

Command-level cloudOnly/selfHostedOnly overrides now also wrap the
command's row in the parent Subcommands table in ifdef/ifndef::env-cloud,
matching the existing flag-row behavior. Cloud-docs single-sources these
tables, so rows for commands that only exist in the self-managed docs
previously produced broken xrefs in cloud builds.

Rows for the rpk cloud and rpk security secret families (routed to
cloudSecretDir and published by cloud-docs) now link across to the
cloud-data-platform component instead of emitting in-component xrefs
that never resolve. Tables inside those families keep in-component
xrefs since they render in cloud-docs itself.

* fix(rpk-docs): keep cloud-routed rows in subcommand tables with cross-component xrefs

Integration fix now that #225 is on main: #225 dropped rpk cloud and
rpk security secret rows from parent subcommand tables entirely (a
cruder fix for the same broken-xref symptom this branch solves). The
rows now stay and this branch's isCloudSecretCommand routing links
them across to the cloud-data-platform component. Combined suite green
(866 tests), regenerated rpk-security.adoc table byte-identical to the
hand-fixed page from redpanda-data/docs#1837.
JakeSCahill added a commit that referenced this pull request Aug 3, 2026
* docs: add end-to-end testing guidelines for automations

Distills the methodology from validating the rpk plugin docs train
(#225), where a ground-truth review found seven generator defects that
800+ passing unit tests had missed. Covers building a ground-truth
corpus, independent mechanical checkers, Asciidoctor and Antora
validation stages, verifying version claims against snapshots, baseline
gap pitfalls, corpus diffing between fix rounds, triage discipline, and
workflow testing including act safety.

* docs: add failure modes of the review itself

Honest accounting from the rpk train: several defect classes escaped
an otherwise thorough end-to-end review and were caught by human
reviewers or the first production run. Each gets a named check:
totals-not-samples, fixture data shapes, the consumer's model vs the
producer's, cross-repo workflow topology, legacy paths beside new
ones, scheduled unblocks for intentional-for-now findings, and
verifying tool writes. Also strengthens triage: pre-existing is a
classification, not a disposition — findings need an owner.
JakeSCahill added a commit that referenced this pull request Aug 3, 2026
Both open questions were answered by rendering real connector data
(connect 4.103.0, 244 descriptions) with Asciidoctor:

Large-block mitigation: collapsible dropped in favor of passthrough
plus reporting. At the 1200-char threshold the collapsible fired on
five real connectors, and the worst case (redpanda_migrator, 6637
chars) has a hand-curated published page the migration would never
auto-swap anyway. Hiding primary content also depends on
browser-specific find and fragment auto-expand behavior in closed
details elements, and bodies carrying their own ==== delimiters could
never be wrapped (the http_server guard), so behavior diverged
silently per page. Heading demotion was rejected with direct evidence:
demoting snowflake_put's embedded headings produces six 'section title
out of sequence' warnings because the description renders before the
page's first == section. Passthrough matches the published pages'
structure exactly. The generator now reports long heading-less
descriptions during generation (10 current candidates) so structure
gets added upstream instead of being hidden by the docs build.

Summary placement: the partial now carries two tagged regions in one
file. tag=attrs holds a :description: attribute line (flattened to one
line via the new summaryAttribute helper: three summaries contain hard
breaks) for pages to include in their header, so search snippets and
meta descriptions refresh too. tag=body holds the summary, version
note, and rendered description for the body include. Verified with
Asciidoctor that a page consuming both tags gets the fresh attribute
and body with zero warnings.

Version 5.4.0: a feature bump that also sidesteps the 5.3.x patch
train (#225 takes 5.3.0, #228 re-bumps to 5.3.1).
JakeSCahill added a commit that referenced this pull request Aug 5, 2026
Follow-up experiment: comment lines are legal inside an AsciiDoc header,
so the tag::meta[] markers can wrap the page's ONE existing
:description: instead of a duplicate inside the single-source region.
Verified against the Antora build with the production UI bundle: the
source page keeps its own meta description and the stub inherits the
same line, body intact, zero warnings.

This retires the description duplication introduced in #225 entirely:
one attribute on the source page serves the page itself and every stub.
Consumers of tag=single-source lose only a mid-body attribute
reassignment that never reached any page's metadata.
JakeSCahill added a commit that referenced this pull request Aug 6, 2026
Follow-up experiment: comment lines are legal inside an AsciiDoc header,
so the tag::meta[] markers can wrap the page's ONE existing
:description: instead of a duplicate inside the single-source region.
Verified against the Antora build with the production UI bundle: the
source page keeps its own meta description and the stub inherits the
same line, body intact, zero warnings.

This retires the description duplication introduced in #225 entirely:
one attribute on the source page serves the page itself and every stub.
Consumers of tag=single-source lose only a mid-body attribute
reassignment that never reached any page's metadata.
JakeSCahill added a commit that referenced this pull request Aug 12, 2026
* Let single-source stubs inherit descriptions as header attributes

Antora resolves page metadata with a header-only parse that stops at
the stub's first blank line, so the description repeated inside the
single-source tag region never reached stub pages: hundreds shipped the
generic site description (micheleRP's agent-friendly-docs audit).

Verified empirically against an Antora build with the production UI
bundle: a stub that includes the region with no blank line inherits the
description but loses its body (the first content line parses as an
author line). The working shape is two includes, and this change
enables it:

- command.hbs nests a tag::meta[] region around the in-tag description.
  Tag directive lines are stripped from tag-filtered includes, so
  existing consumers are unaffected. The old comment claimed the in-tag
  copy alone reached stub metadata, which was wrong. Corrected.
- The plugin stub generator emits the header include (above the first
  blank line) for partials that carry the region, and keeps the plain
  shape for older partials so builds never warn about a missing tag.

Descriptions then flow to stubs at build time, so they can never drift
when regeneration rewrites them. Static backfills (adp-docs#189,
cloud-docs#664) stay correct. New and resynced stubs stop needing them.

Also ignores the property extractor's run-time working data, which a
broad git add could otherwise sweep into a commit.

* Single description: the meta tag region wraps the header attribute

Follow-up experiment: comment lines are legal inside an AsciiDoc header,
so the tag::meta[] markers can wrap the page's ONE existing
:description: instead of a duplicate inside the single-source region.
Verified against the Antora build with the production UI bundle: the
source page keeps its own meta description and the stub inherits the
same line, body intact, zero warnings.

This retires the description duplication introduced in #225 entirely:
one attribute on the source page serves the page itself and every stub.
Consumers of tag=single-source lose only a mid-body attribute
reassignment that never reached any page's metadata.

* Cover every single-source producer and make consumer syncs self-heal

- The three Redpanda Connect templates (connector pages and both
  Bloblang overviews) get the same meta tag region around their
  descriptions, so cloud-docs Connect stubs can inherit them too.
  connector.hbs also stops emitting the TODO placeholder as the
  attribute VALUE when a connector has no description: a stub would
  inherit that into its meta tag, which is worse than the generic text.
  Missing descriptions stay a tracked editorial backlog.
- reconcileStubs now upgrades existing stubs in place: when a partial
  has gained the meta region and the stub lacks the header include, the
  sync inserts it (idempotent, reported as 'upgraded'). Consumer repos
  with a sync workflow self-heal instead of needing a migration PR.

* Self-heal connector page descriptions from source summaries

Connector pages are one-time first drafts, so pages created before the
template emitted :description: never got one and nothing rewrote them:
~310 of 411 published pages ship the generic site meta description even
though ~97% of components carry a summary in the source data (verified
against connect 4.88.0: 559 of 562 have a summary or description; the
multilevel cache, a published no-description page, has a perfectly good
summary).

backfillPageDescriptions inserts the summary, flattened to one line and
wrapped in the meta tag region, at the end of any page header that
lacks a description. It runs on every partials generation, so the page
set self-heals and stays healed, and it is exported standalone to drive
the one-time backfill PR in rp-connect-docs. Pages whose component has
no summary are reported for the editorial backlog instead of guessed
at. Idempotent, dry-run supported, tested.

* Flatten AsciiDoc markup out of backfilled meta descriptions

xref macros and inline-code backticks would land verbatim in the meta
tag, which search results and link previews show as raw markup (22 of
the 274 backfill candidates carry an xref). Mechanical flattening only:
links become their labels, code spans lose their backticks, no prose is
edited.

* mergeOverrides: stop silently dropping summary overrides

overrides.json has carried top-level summary overrides (zmq4, ffi) that
never took effect: 'summary' was missing from scalarKeys and fell
through every merge branch. Summaries feed page meta descriptions, so
an authored override silently did nothing. Found while authoring
summaries for the seven components with none in the source data.

* Map the rate-limits data key to the rate_limits pages directory

The connector data uses 'rate-limits' while the pages directory is
'rate_limits', so the backfill silently skipped local and redis (the
only two components in that family). Resolve whichever spelling exists
on disk.

* Strip backfilled literals when upgrading stubs: later entries win

micheleRP's precedence finding, re-verified against an Antora build:
a later attribute entry overrides an earlier one, so a literal
:description: below the inserted meta include wins and the include is
dead weight. The upgrader now removes the literal when it inserts the
include. Scope is inherently safe: the reconciler only manages
generated reference stubs, so hand-curated prose descriptions are
never touched.

* helm-spec: relocate the description into the page header

The chart README templates emit :description: below the title's blank
line, where Antora's header-only metadata parse never sees it, so the
generated helm spec pages shipped the generic site meta description
despite carrying a real one (found on k-connect-helm-spec). The
post-processing chain now moves the first description line into the
header, directly under the title. Covers every chart this command
generates, independent of each chart repo's gotmpl.

* chore: bump version to 5.3.8 (5.3.7 published by #245; 5.4.0 claimed by #227)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Flatten bare URL macros and internal xref shorthand in backfilled descriptions

The existing flattener only handled xref:/link: macros, so bare https://...[label^] URL macros and <<anchor,label>> internal xrefs survived into meta descriptions and rendered literally in search results (surfaced by review of rp-connect-docs#478, which backfilled ~27 pages before this landed).

* chore: bump to 5.8.0 instead of 5.10.0

main and npm are both at 5.7.0, and these are backward-compatible
feature additions, so 5.8.0 is the correct minor bump. The pre-assigned
version lanes across the open PRs only hold if merge order matches
numeric order, and it doesn't: the PR holding 5.8.0 has changes
requested, so this branch merges first. Publishing 5.10.0 now would mean
a later 5.8.0/5.9.0 merge moves the npm latest tag backwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
JakeSCahill added a commit that referenced this pull request Aug 22, 2026
#227)

* feat(rpcn): regenerate a per-connector description partial

Adds a `descriptions/<type>/<name>.adoc` partial that is regenerated on every
run, mirroring the fields/examples/metadata partials. This is the foundation
for fixing description staleness: today a connector's main page is drafted once
and never refreshed, so summary/description edits upstream never reach already-
published pages. A page that includes this partial always renders the current
summary + version + description.

- metadata-utils: add descriptionIncludeLine() (mirrors metadataIncludeLine).
- helpers/renderConnectDescription: render the description with the == Metadata
  block replaced by the metadata partial include (de-dup), plus large-block
  mitigation — a long, heading-less prose wall keeps its first paragraph visible
  and moves the remainder into an AsciiDoc [%collapsible] block. Descriptions
  that already carry their own headings pass through unchanged.
- templates/descriptions-partials.hbs: summary + version + rendered description
  behind a "do not edit" banner.
- generator: emit the partial for every non-bloblang connector.
- tests: 9 new cases (util, helper mitigation branches, generator output).

intro.hbs is intentionally left unchanged, so newly drafted pages are byte-for-
byte identical and existing generator tests stay green. Wiring pages to include
the partial is a deliberate, separate migration (like metadata partials), not an
automatic rewrite — see the PR description.

Bumps version to 5.2.6.

* fix(rpcn): guard collapsible against ==== collision, blank stale description partials

Review fixes for the description-partials prototype:

- A body containing its own ==== block delimiters (admonition/example
  blocks) is no longer wrapped in the [%collapsible] block: the nested
  opening ==== terminated the collapsible early, leaking the rest of the
  body as top-level blocks and stripping admonition styling (seen with
  the http_server output's nested CAUTION). Such bodies now pass through
  unchanged.
- When a connector's description disappears upstream, the previously
  generated description partial is now blanked to a banner-only file and
  logged, mirroring the stale-metadata handling, instead of silently
  leaving first-draft text on disk forever.
- Description partials are only emitted for connector type dirs
  (mirrors the handler's connectorTypes list), so the config data key no
  longer produces 17 unincludable files under descriptions/configs/.
- Remove the unused HEADING constant.

Adds regression tests for all three, including Asciidoctor render-level
assertions that the CAUTION admonition survives intact and that plain
collapsible output keeps all content inside the block.

* feat(rpcn): resolve description-partial design questions from testing

Both open questions were answered by rendering real connector data
(connect 4.103.0, 244 descriptions) with Asciidoctor:

Large-block mitigation: collapsible dropped in favor of passthrough
plus reporting. At the 1200-char threshold the collapsible fired on
five real connectors, and the worst case (redpanda_migrator, 6637
chars) has a hand-curated published page the migration would never
auto-swap anyway. Hiding primary content also depends on
browser-specific find and fragment auto-expand behavior in closed
details elements, and bodies carrying their own ==== delimiters could
never be wrapped (the http_server guard), so behavior diverged
silently per page. Heading demotion was rejected with direct evidence:
demoting snowflake_put's embedded headings produces six 'section title
out of sequence' warnings because the description renders before the
page's first == section. Passthrough matches the published pages'
structure exactly. The generator now reports long heading-less
descriptions during generation (10 current candidates) so structure
gets added upstream instead of being hidden by the docs build.

Summary placement: the partial now carries two tagged regions in one
file. tag=attrs holds a :description: attribute line (flattened to one
line via the new summaryAttribute helper: three summaries contain hard
breaks) for pages to include in their header, so search snippets and
meta descriptions refresh too. tag=body holds the summary, version
note, and rendered description for the body include. Verified with
Asciidoctor that a page consuming both tags gets the fresh attribute
and body with zero warnings.

Version 5.4.0: a feature bump that also sidesteps the 5.3.x patch
train (#225 takes 5.3.0, #228 re-bumps to 5.3.1).

* fix(rpcn): emit partials for summary-only connectors and never escape the description attribute

Two CodeRabbit findings, both real at scale against connect 4.103.0:

- The partial template gated everything on description, so the 40
  connectors that define a summary without a description rendered an
  empty partial, which the generator then treated as an upstream
  removal and blanked. The gate is now (or summary description).
- The :description: attribute value used double-stash output, so the
  9 summaries containing characters like ampersands or quotes were
  HTML-escaped inside an AsciiDoc attribute. Triple-stash now matches
  the body rendering.

* fix(rpcn): escape placeholder braces, separate glued headings, report out-of-sequence starts

A full-corpus Asciidoctor convert of all 311 generated description
partials (connect 4.103.0) surfaced three defect classes:

- Literal placeholders like {endpoint} and {api_version} in
  descriptions (otlp_http, salesforce_graphql) are consumed as
  attribute references, even inside single-backtick spans, and warn on
  the published pages today. They are now escaped outside listing
  blocks.
- The protobuf processor's '== Operators' heading is glued to the
  paragraph above it, so it rendered as literal text and its
  subsections warned out of sequence. Headings now get the blank line
  Asciidoctor requires (listing blocks untouched).
- aws_dynamodb_cdc (markdown ###) and iceberg (===) start their
  heading structure below level one, which renders out of sequence and
  needs an upstream fix: the generator now reports both at build time,
  and hasStructuralHeadings recognizes markdown headings so the
  long-headingless report cannot misfire on markdown-structured
  descriptions.

* Report markdown-heading descriptions and surface reports in the PR summary

Review findings:

- Markdown ## headings counted as structure, so the descriptions that
  render worst were exactly the ones the long-description report
  skipped. Structural now means AsciiDoc == only; markdown-heading
  descriptions get their own report naming the upstream conversion.
- The reports went to console.warn, which the workflow buries in a
  collapsed log block. They now ride the diff object and render as a
  collapsed section of the PR summary body.

* Release 5.4.0

Description partials are a new generator feature.

* fix(rpcn): track markdown fences in description scanners so fenced examples pass through untouched

The five body scanners in renderConnectDescription.js only tracked AsciiDoc
---- listing delimiters, so the interior of a ```/~~~ fenced example was
treated as prose: {token} placeholders gained a literal backslash escape
(reproduced: 'url: {endpoint}/v1/logs' rendered as 'url: \{endpoint}/v1/logs')
and glued #-comment lines had blank lines pushed into the example. Share
metadata-utils' FENCE_DELIMITER and route every scanner through one layered
block/fence annotator so fence interiors pass through byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(rpcn): blank description partials orphaned by connectors deleted upstream

The in-loop stale handling only covers connectors still present in the
dataset: a connector deleted upstream leaves no entry at all, so its
description partial kept serving deleted-connector text forever, silently.
After the generation loop, sweep the descriptions output root for .adoc
files no connector claimed this run, blank them with the same banner-only
content as other removed-content partials, and surface them in
descriptionReports (and the PR summary). Draft mode skips the sweep: it
runs on a filtered dataset where unvisited means already documented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(rpcn): brace-escape summaries so {placeholder} survives the description partial

The template emits the summary raw in both the partial body and the
:description: attribute, so a {placeholder} in a summary was silently
consumed by Asciidoctor as a missing attribute reference. Escape summaries
through the same escapePlaceholderBraces path the description body already
uses, in the generator that prepares the template data.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(rpcn): add --template-description CLI override for the description partial template

Every other partial template has a CLI override flag, but the generator's
templateDescription option was unreachable from the CLI: no
--template-description flag existed and the handler never passed it.
Mirror --template-metadata and pass the option through both generator
call sites (partials and draft-missing).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(rpcn): reunite generateMultiVersionPRSummary with its JSDoc

renderDescriptionReports was inserted between the multi-version summary
JSDoc and its function, orphaning the @PARAM documentation. Move the new
function (with its own JSDoc) above the existing comment block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: Auto-update CLI reference documentation (PR #227)

* fix(rpcn): refuse the description orphan sweep when it would blank the tree

The post-generation sweep blanks every description partial the current
dataset did not claim, and an incomplete dataset is indistinguishable
from a mass upstream deletion. Measured: a full connect-4.103.1 run
writes 316 partials, and a following run whose data carried only the
inputs key blanked 233 of them with no dry run, no threshold and no
confirmation. Once pages include these partials that is hundreds of
published pages losing their body in one pass.

Collect the sweep candidates before writing anything and refuse when
more than 10 files AND more than 10% of the tree would be blanked,
reporting the refusal into the PR summary instead. Small, plausible
deletions still sweep unchanged. --prune-orphaned-descriptions is the
explicit opt-in for a dataset a human has confirmed complete.

* fix(rpcn): write rate-limit partials where their include line points

The generator derived the partial directory from the raw dataset key
(rate-limits) while descriptionIncludeLine and metadataIncludeLine
derive it from item.type (rate_limit -> rate_limits), so every
rate-limit page wired to a description partial got an unresolved
include. A real Antora build reports "target of include not found" and
the page ships with the content silently missing.

Make typeDirFor the one place a type directory is derived: it now
collapses every rate-limit spelling to rate_limits, and the generator
writes the metadata and description partials under typeDirFor(item)
rather than the data key. Fields and examples keep the raw data-key
spelling because published pages already include them by that path.

* fix(rpcn): keep tag regions in a blanked description partial

Pages include this partial with [tag=attrs] and [tag=body]. The banner
written when a description disappears upstream carried neither region,
so Asciidoctor logged "tag 'attrs' not found in include file" and "tag
'body' not found" for every consuming page on every build, which is a
build failure wherever warnings are errors. That also contradicted the
reason for blanking rather than deleting: keep published includes
resolving.

Emit empty attrs and body regions in the banner so both tags resolve to
nothing.

* fix(rpcn): flatten the description attrs region through one flattener

summaryAttribute collapsed whitespace only, so the attrs region a page
adopts for its :description: shipped raw backticks, xref: and link:
macros and bare URL macros straight into <meta name="description"> and,
through single-source stubs, into cloud-docs. A real Antora build on a
page wired to the attrs include published
`xref:guides:bloblang/about.adoc[Bloblang mapping]` verbatim.

backfillPageDescriptions already had a flattener that handles all of it.
Move that one into metadata-utils as flattenToAttributeValue and call it
from both, so the meta description a page publishes does not change when
it migrates from the backfilled block to the partial.

* feat(rpcn): wire published pages to the description partial

The branch wrote 316 description partials per run and nothing read a
byte of them: no page carried the include, no template emitted it, and
no migration existed, so the whole feature shipped to no consumer.
Verified against the real rp-connect-docs, where partial$descriptions
had 0 references against 51 for partial$metadata.

Three pieces close the loop:

- connector.hbs emits the two tagged includes instead of freezing the
  summary, the "Introduced in version" line and the prose into the page,
  so new drafts are born wired. The attrs include sits inside the page's
  // tag::meta[] region, so single-source stubs keep inheriting
  :description: as a header attribute. Component families with no
  description partial, and connectors with no summary, keep the inline
  attribute rather than getting an include that resolves to nothing.
- generate migrate-rpcn-descriptions rewires the pages already
  published. Dry run by default. The body rewire is guarded twice: the
  page's intro region must be reproduced line for line by the partial,
  and no content outside it may duplicate the partial body. On
  rp-connect-docs origin/main that migrates 290 page headers and 30
  bodies, and refuses 175 pages whose intro carries prose the generator
  has never seen plus 103 whose description continues past the config
  listing, rather than deleting published content.
- #246's page-header backfill now recognizes the attrs include, so it
  stops splicing a competing :description: block into every migrated
  page.

A real Antora build of migrated rp-connect-docs against a build of the
same tree unmigrated: 446 pages both times, identical warning and error
counts, zero unresolved description includes, zero missing tags, zero
changes to rendered body text on 338 component pages, and 3 meta
descriptions that changed only by losing raw link markup.

* fix(rpcn): warn before a regenerated description partial drops a section

The metadata partial has compared old against new headings and pushed
lostSectionWarnings since #236, because a section vanishing from a
published partial is content loss rather than cleanup. The description
partial was overwritten unconditionally: a hand-added section
disappeared with no warning, no report and no log line, proven by
injecting the same section into both partials and regenerating.

Now that pages include these partials the risk is live, so route the
description write through the same lostMetadataSections check.

* fix(rpcn): collect the draft run's description reports into the PR summary

The draft generator call collected draftResult.lostSectionWarnings and
silently dropped draftResult.descriptionReports, so the structure
reports for newly drafted connectors, the ones most likely to need an
upstream fix, never reached the auto-docs PR summary. Deleting either
push left the suite fully green because the handler has no coverage at
all: it cannot even be required under jest, since it pulls in ESM-only
octokit.

Both call sites now merge through one collectGeneratorReports in
pr-summary-formatter, which is requireable and tested, so a call site
can no longer forget a report key.

* refactor(rpcn): one verbatim-line annotator for every prose scanner

There were three copies of the block/fence state machine and they had
already drifted: renderConnectDescription tested the trimmed line while
metadata-utils and normalize-metadata tested the raw one, so an indented
---- opened a verbatim region for one scanner and was plain content for
another, on the same description in the same pipeline. locateMetadata,
which runs before all of them, tracked ---- only and was blind to
markdown fences, so a `== Metadata` inside a fenced example was
extracted into the metadata partial and the fence was destroyed.

Export one layered annotator from metadata-utils and consume it from
locateMetadata, sectionHeadings and the five scanners in
renderConnectDescription. It tests the raw line, because Asciidoctor
only reads a block delimiter at column 0. normalize-metadata imports the
shared delimiter rather than declaring a third copy.

* refactor(rpcn): one connector-type list, one gate for per-page partials

The list of component families existed in eight places and the copies
disagreed three ways: config in two of them, rate_limits missing from
one, both rate-limit spellings in two. A family added upstream silently
gets no partial, no page backfill or no cloud-docs check depending on
which copy forgot it.

Export CONNECTOR_TYPE_DIRS (page-backed families) and
CONNECTOR_DATA_KEYS (those plus config) from metadata-utils and use them
in seven of the eight places. The eighth deliberately omits rate_limits
and changing it would add cloud-docs findings in a path with no test
coverage, so it keeps its literal with a comment saying why.

The examples and metadata partials also gated on a denylist naming only
the two bloblang keys while the description partial next to them used an
allowlist, so both wrote partials under configs/ that no page can
include. All three now use the one allowlist.

* fix(rpcn): keep version-only connectors, cover the flags, hoist the compiles

Three small things the review left open.

The description partial's outer guard tested (or summary description),
so a connector carrying only a version got no partial at all and its
"Introduced in version" line was lost. It now includes version.

--template-description, --template-metadata and the new
--prune-orphaned-descriptions had no coverage: the flag could be deleted
with the whole suite staying green, and the CLI contract check does not
notice a documented flag going missing. Tests now assert a stub template
reaches the emitted partial and that the CLI still advertises the flags.
intro.hbs gets a test too, pinning that the fallback body path applies
the same brace escaping and heading separation as the partial.

The four one-line partial wrappers were recompiled once per component,
and the metadata and description partials were rendered before the type
guard, so both were computed and discarded for every bloblang function,
method and config entry. Compile once above the loop, render inside the
guard.

* chore(release): 5.16.0

The branch sat at 5.12.0, which is main's version and is already
published on npm, so merging would have shipped nothing: not the
partials, not the wiring, not the CLI flags. 5.16.0 is this PR's slot in
the agreed merge order and is free on the registry.

* docs: Auto-update CLI reference documentation (PR #227)

* fix(cli-docs): register the new command and stop stealing another's prose

The auto-regeneration workflow deleted the migrate-rpcn-descriptions section
from CLI_REFERENCE.adoc on the first push, which was correct: the file is
generated, and the section had been hand-written. Two separate defects sat
behind that, both introduced here.

First, tools/generate-cli-docs.js drives its output from a hardcoded
generateSubcommands list and the new command was not in it, so no section was
emitted at all.

Second, and worse, adding the command's JSDoc broke its neighbour. The
generator pairs a comment with the NEXT .command() call it finds, and this
change had stacked both JSDoc blocks together with the two command chains after
them in the opposite order, so neither comment was adjacent to its own command.
The regenerated migrate-rpcn-metadata section fell back to truncated help text,
and migrate-rpcn-descriptions was documented with migrate-rpcn-metadata's prose.
Each JSDoc block now sits immediately before the command it documents, and the
regenerated migrate-rpcn-metadata section is byte-identical to main again.

The JSDoc regex is also tempered so a comment body cannot run past its own
closing delimiter, which is what let two adjacent blocks merge into one match.

Both failure modes are silent, so both are now tested: one check that every
registered generate subcommand is known to the generator, one that every JSDoc
block is adjacent to its own command. Three commands are already undocumented
on main (migrate-property-refs, rpk-plugin-stubs, rpk-overrides); they are
listed explicitly so the gap is visible and cannot grow, with a second test
failing if an entry stops naming a real command. PR #264 retires these
hardcoded lists by deriving the reference from the CLI.

* fix(deps): take main's dependency set instead of carrying a stale pin forward

This branch never meant to change any dependency, but resolving the
package.json conflict from our side kept its own stale pins: @antora/cli and
@antora/site-generator at 3.1.4 where main has ^3.1.15. Two ways that hurt, and
CI catches neither, because each branch is internally consistent:

- On #258 the lockfile and manifest disagreed and `npm ci` failed outright with
  "Missing: @asciidoctor/core@2.2.8 from lock file", which is what surfaced this.
- On the others the branch stayed green while a merge would have DOWNGRADED
  main's @antora/cli back to 3.1.4. Verified with git merge-tree.

main owns dependencies, devDependencies and engines. Both package files are now
main's, with only this PR's version stamped and its own exports/scripts
additions preserved, and `npm ci --dry-run` passes against the pair.

* fix(redpanda-connect): stop dropping the PR summary when there's no diff

summaryDiff = masterDiff || diffJson is null on every run with no prior
version to diff against or matching versions, and the next line
unconditionally set a property on it -- "Cannot set properties of
null" -- silently caught by the surrounding try/catch, which meant the
entire PR summary (not just descriptionReports) never printed.
Reproduced live twice against real fetched Connect data.

When there's no diff, lostSectionWarnings/descriptionReports now print
on their own via the same renderers generatePRSummary already uses,
instead of going through the diff-shaped summary path at all.

Also: metadata-utils.js's shared verbatim tracker recognized AsciiDoc
`----` listing blocks but not `....` literal blocks, which are equally
substitution-free -- confirmed via a real Asciidoctor convert() that a
`{token}` inside a .... block was getting corrupted by the
brace-escaper. Added a matching LITERAL_DELIMITER state, symmetric with
the existing one, plus 4 regression tests.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants