Repository navigation
Description
Over the last day I filed a number of issues and advisories from an external audit of v0.44.2 (700bdde). Several of them carry a "not checked" note. This issue collects all of those in one place, with the reason for each, so the gaps are legible rather than scattered across a dozen threads.
The reason for posting this: an audit that reports only what it found invites the reading that everything else was examined and was fine. That is not true here. Below is what I could not examine, and why. "Not checked" means no evidence either way — not "clean".
Provenance: AI-assisted source audit, human-directed. The auditing environment was a single Windows 11 machine with Rust 1.97.1, Node 24, .NET 10 and Git. Nothing else in the list below was installed, for reasons given.
A. Not checked because the tool was not installed — and installing it would not have helped
For these, rtk's filter parses the tool's real output. I used fixtures and shims, which prove the parser's behaviour on the input I supplied but say nothing about what the tool actually emits today. Installing the binary alone would not have closed the gap, because each of these needs credentials, a cluster, a daemon, or a database that I do not have and should not have:
| tool | why the binary alone proves nothing |
|---|---|
aws |
needs live credentials + a real account; output shape varies by service and CLI version |
kubectl |
needs a cluster |
docker |
needs a daemon and images |
psql / mysql |
needs a running database |
gcloud, az |
needs an authenticated cloud project |
sops, vault, age |
needs real key material — I was not going to generate live secrets to test a filter |
terraform, ansible |
needs real infrastructure state to produce representative output |
mvn, gradle, sbt |
needs a JVM toolchain and real multi-module projects; fixture output was used instead |
black, prettier, ruff, eslint |
not installed; their filters are source-read only |
gh, glab, gt |
gh was installed, but exercising the filters needs live API responses from a repo with the right state (failing checks, truncated lists) |
Concretely, this means: every claim I made about an aws, kubectl, psql, docker, terraform, ansible, or sops filter is a claim about rtk's parsing code, not about how that filter behaves against the real tool. Where I said a filter drops a field or mis-parses a format, I verified it in rtk's source and against a fixture I wrote. A maintainer with the real tool can confirm or refute in one run.
B. Not checked because I was on the wrong OS
The audit machine is Windows 11. Not examined:
#[cfg(unix)]code paths — including the file-permission hardening (set_owner_onlyis a no-op on non-unix), signal handling,SIGPIPEbehaviour, process-group semantics, and orphan reparenting. Several findings I filed reason about Unix behaviour from source; none of it was observed.- macOS specifics — the Apple Silicon startup-overhead numbers in NF01: rtk-rewrite.sh hook adds ~56ms overhead on Apple Silicon (vs documented <5ms) #1375 are someone else's; my own timings are Windows, where the bare process-spawn floor is ~35 ms, so absolute values are not comparable.
- Linux release artifacts — the musl/gnu split, the
.deb/.rpmpackages, and the install script's behaviour were read, not run.
C. Not checked because it needs another vendor's product
rtk integrates with several agent harnesses by writing hook configs and emitting JSON they consume. I could not verify the receiving end for:
- GitHub Copilot CLI — whether it honours
permissionDecisionas rtk supplies it. (Your own rtk hook copilot emits permissionDecision "ask", forcing a permission prompt on every command in Copilot CLI >= 1.0.66 #3037 answered part of this; I have no independent view.) - Gemini CLI, Cursor, Cline/Roo, Windsurf, OpenCode, Droid, JetBrains Copilot plugin — the schemas in
hook_cmd.rsare documented in comments and I read them, but I never ran any of these products. If a vendor changed their hook schema, I would not know. - Claude Code itself — I exercised
rtk hook claudedirectly by piping JSON, which tests rtk's side. I did not run it inside a real Claude Code session.
D. Not checked because it lives outside the repository
- Branch protection and required status checks. Issue Two CI steps cannot fail, and no lint guards
&strbyte-slicing — the two gaps behind several open panic/false-summary issues #3394 argues two CI steps cannot fail. Whether those jobs are even required for a merge is a GitHub repository setting, not a file in the tree — I cannot see it. If they are not required, the finding is larger than I stated; if some other gate compensates, smaller. - The telemetry server. Retention, deletion, IP handling, and what the
/erasureendpoint actually does are all server-side. Advisory GHSA-q584 reports what the client sends; I make no claim about what happens after. - crates.io / release-artifact provenance. I did not download and inspect your published binaries (including not running
stringson them to look for a compiled-in token — that felt inappropriate for a report you had not yet triaged).
E. Not checked because I chose not to run it
Stated so it is not mistaken for an oversight:
rtk initand its uninstall paths — these write into the real home directory. I read the code and never executed it.rtk cc-economics— it invokesnpx --yesand reads real spend data.rtk telemetry forget— it deletes the real local database and fires a network request. GHSA-q584 is therefore a code-property report, not an executed one.- Anything that would have required generating live credentials to make a realistic test case.
F. Read but not audited in depth
Large surfaces I opened enough to navigate, but did not review line by line:
src/hooks/init.rs(~280 KB) — grepped, not read in full.src/discover/registry.rs(~155 KB) — the 178 rewrite rules were counted and theRegexSetpath traced; the individual rules were not each validated.src/filters/*.toml— 63 declarative filter definitions. At the time I filed thesopsadvisory I had opened exactly one of them.src/openclaw/,src/cmds/php/, most ofsrc/cmds/js/andsrc/cmds/dotnet/beyond the specific files cited in my reports.- The test suite: I ran it (2520 tests, all passing) but only examined tests adjacent to findings.
What I would ask
If any finding of mine matters to you, the fastest way to settle it is usually the one thing I could not do: run it against the real tool. Where that is the case I said so in the report. I would rather you close something as "works fine against real kubectl" than have it sit open on my say-so.
And if you would find it useful, I am happy to re-check anything on this list against a fixture you consider representative — a captured output file is enough for most of category A.
Per-report ledger: what is still unverified in each thing I filed
The categories above are the reasons. This is the mapping — for every post I made, what specifically remains unverified in it. Read this as: if you are triaging one of these, here is the part you should not take on my word.
Legend: ✅ RAN = I executed it and observed the result ·
| # | what I filed | status | what remains unverified in it |
|---|---|---|---|
| #2281 | go vet can report "No issues found" on a broken module |
📖 SOURCE | I never ran go vet against a module with a go.mod error. The claim is from go_cmd.rs:117,703,715 plus the f69ad6e commit. Category A (no Go toolchain configured for a broken-module fixture). |
| #2001 | savings baseline, threshold contradiction, grep memory | The memory/wall-clock numbers were measured. I did not reproduce @vpovarna's 41-command benchmark, so nothing I said explains their end-to-end token result — only the mechanism. | |
| PR #2401 | a 4th unguarded filter path; FilterMode::Buffered may be test-only |
The unwind-vs-abort proof was compiled and run. I did not build your branch. The claim that FilterMode::Buffered has no production constructor is a grep result — if I missed a constructor, that half is wrong. |
|
| PR #3295 | 3 open PRs for one bug; cargo precedent at 39a4d28 |
📖 SOURCE | Neither PR was built or tested. This is a git log + diff-reading observation only. |
| #2954 | the gain.rs panics look fixed; 2 sibling sites are not |
The two new panic sites were reproduced with verbatim extracts. I did not run rtk gain --history with a CJK/Cyrillic history, so "the original is fixed" is "the unsafe slice is gone from the code", not "I confirmed the crash no longer happens". |
|
| PR #3368 | 3 sibling structs have the same defect; 6 dead keys | The deserialization failures were reproduced on a replica of your struct attributes (serde 1 + toml 0.8), not on your binary. The dead-key list is a grep. | |
| PR #2581 | strip_ansi cannot match \x1b[?25l; the PHP copy fails differently |
📖 SOURCE | I did not run either stripper against a live spinner. The CONTROL_RE claim in particular is read from its character range — worth confirming against the actual constant before acting on it. |
| PR #1693 | 24 rustdoc examples can never compile | ✅ RAN | cargo test --doc → error: no library targets found. Little left unverified here. |
| #2317 | verified inventory replacing the "(likely)" list | 📖 SOURCE | I did not run every filter. Each row is a code-path claim. The gradlew row is from an exhibited input, not a live Gradle run — Category A (no Android/Gradle setup). |
| #1375 | the 178-regex COMPILED tax explains the 20 ms gap |
Measured on Windows, where process-spawn alone is ~35 ms. I have not profiled on Apple Silicon, so I cannot say what fraction of your 56 ms this is there — the deltas are meaningful, the absolutes are not comparable. | |
| #3392 | rtk grep 31× memory / 4.9× wall-clock |
✅ RAN | Measured on Windows only; Linux/macOS allocator behaviour may differ. |
| #3393 | db never shrinks; gain slows on large history |
✅ RAN | Measured. The query-plan claims are from EXPLAIN QUERY PLAN on a synthetic 500k-row database, not your users' real distribution. |
| #3394 | two CI steps cannot fail; no slicing lint; semgrep baseline | The deny(warnings) clippy escalation was verified on a probe crate. Whether those jobs are required for a merge is a repository setting I cannot see — Category D. If they are not required, this is worse than I said; if another gate compensates, it is smaller. |
|
| #3395 | psql parsed without -X, ~/.psqlrc changes the format |
📖 SOURCE | Entirely source-read. psql is not installed — Category A (needs a running database to produce representative output). Every failure mode I describe is predicted, not observed. |
| #3396 | stale 0.28.2 / rtk 0.16 version checks |
✅ RAN | Read directly from the files; Cargo.toml says 0.44.2. |
| #3397 | filtered clone no caller reads; triple-retained capture |
📖 SOURCE | Reachability traced by reading every call site. Not profiled — I did not measure the allocation cost, only argue it exists. |
Reports sent through the private security channel
Those carry their own "Testing" section stating exactly what was and was not executed — the same discipline as above. I am not restating them here while they are in triage. In short: two were reproduced end-to-end on this machine, one had its mechanism reproduced but not its tool-specific step (the tool is not installed), and two are code-property reports I deliberately did not execute because running them would have fired a real network request or deleted real local data.
The honest summary
Of the 16 public posts above: 5 were executed and observed, 6 had part of the claim executed, and 5 are source-reading only. The five source-only ones are the ones I would most want you to check before acting — they are also, not coincidentally, the ones covering tools I could not install.
Activity
pszymkowiak commentedon Aug 4, 2026
🔍 Automated Triage
needs-infolowSummary
Meta-issue disclosing scope limitations across the reporter's recent audit findings; no single actionable bug, primarily informational and useful for triaging linked reports.
Triaged automatically by wshm · This is an automated analysis, not a human review.
HetCreep commentedon Aug 4, 2026
Baseline note, since it affects how you triage the batch.
Everything I filed was audited at the tag
v0.44.2(700bdde), not atdevelop. At the time of writing,origin/developis 13 commits ahead of that tag, and several of them are in the areas I reported on:So if any of my reports reads as already-fixed to you, that is the likely reason — I was reading a released binary and its matching tag, on purpose, because that is what users have installed.
I did check the two areas where a
developfix seemed most likely, and both are unchanged there:src/discover/lexer.rs—split_for_permissions'seg_endhandling (the truncate-at-first-redirect logic) is byte-for-byte the same ondevelop; the diff against the tag is +20 lines elsewhere in the file.src/hooks/hook_cmd.rs— theprint_allow()calls are still present, despite that file changing by ~208 lines ondevelop.I have not diffed the remaining reports against
developone by one. If it would be more useful to you, I am happy to re-check any specific issue atdeveloptip and close it myself if it no longer reproduces — I would rather withdraw something than leave a stale item in a queue this size.Also worth noting for anyone reading this thread later:
develophas had no commits since 2026-08-03, so nothing has moved since I filed.HetCreep commentedon Aug 4, 2026
Closing inventory: 36 items remain unverified, each for a specific missing capability rather than for lack of effort.
Rewriting my previous comment — it was too compressed to act on. Each row below says what the check is, so anyone picking one up knows what to run and what a pass or fail would mean.
A. Needs Linux or macOS — 8 items
The audit machine is Windows, so every
#[cfg(unix)]path was read, not executed.# —search_faithful,search_compress,grep_faithful_format,grep_context,search_error,pipeline_stdincargo test --allon Linux; read the countshistory.db, tee files and the device saltstat -c '%a %n'on the created files — expect600, dirs700set_owner_only(core/utils.rs:266-283) is a no-op on non-unix, so I could not confirm the hardening added for GHSA-w3rq actually appliesrtk cargo build,killthe rtk pid, check whether thecargochild survives and is reparented to initChildGuard::droponlywait()s, neverkill()s, and cannot run at all underpanic = "abort"history.db, tee dir or config fileatomic_writecanonicalises first, which looks correct — but temp-file-then-rename over a symlink is exactly where that assumption breaks. Windows refused to let me create the symlinks to test itcleanup_old_filesracefind/ls/grepdifferentials (#3432, #3434)chmod 000a subdirectory, create a dangling symlink, then comparertk find/ls/grepagainst the real tools\separatorsB. Needs a real tool plus its credential, daemon, cluster or database — 15 items
For these I used controlled stand-ins and said so in each issue. Installing the binary alone would not have closed the gap — every one needs state I do not have and should not have.
aws(≈23 filters)awsoutputkubectldockerdocker run -itTTY handling —container.rs:671uses passthrough and I did not trace stdio inheritancepsqlsopshelmhelm templatetees plaintext on a successful run — confirm with a real chartterraform/tofuansiblegcloudgcloud secrets versions accessruby/rubocop--autocorrect(hyphenless) is accepted by the RuboCop versions you support. If it is not, that finding is voidphp/composer.cmdshimsbin-dirhijack with real composer and realpintgradle/mvngolangci-lintprettier/black/ruffruff format .write-mode injection was shown by capturing argv, not by watching files change on diskplaywright/jest/vitest--reportervalue as a test-path filterC. Needs the other vendors' products — 5 items
This is the group I would most like closed, and the one you can probably close fastest. I can prove what rtk emits; I cannot prove what the receiving harness does with it — and that is precisely what sets the severity of the hook advisories.
permissionDecisionas rtk supplies it?{"decision":"allow"}as a prompt bypass, and does it invokeBeforeToolbefore its own argument-schema validation?rtk hook geminianswersallowwhen it cannot read the command. If Gemini validates arguments first, that path may be unreachable and the advisory should be downgraded{"continue":true,"permission":"ask","updated_input":…}{}on malformed input; whether that is safe depends entirely on CursorupdatedInputwhen nopermissionDecisionis presenttool_namecase rewrites the command with no decision emitted — harmless or not depending on Droidrtk hook claudeby piping PreToolUse JSON on stdin, which tests rtk's side only. I never ran it inside a live sessionD. Outside the repository — 4 items
/erasuredoes server-side are unobservable from the binarystringson them looking for theoption_env!token — that felt inappropriate for a report you had not yet triagedccusageinternalsE. Deliberately not run — 4 items
Listed so they are not mistaken for oversights.
rtk initandrtk init --uninstall(#3400, #3403–#3406)rustcrtk learn --write-rules(GHSA-wh6r and its follow-up)format!call in isolationrtk telemetry forget(GHSA-q584)rtk cc-economics(#3418)npx --yes ccusage) and reads real spend data. Every arithmetic claim in that issue is checkable by reading insteadThe short version: claims about rtk's own code are checkable by reading, and I have kept those separate throughout. Claims about how a wrapped tool or a third-party harness behaves are not, and every one of those is labelled in its own issue.
If any of group C is a one-liner for you, that closes more uncertainty than anything further I can do from here.
HetCreep commentedon Aug 30, 2026
Refreshing one section of my coverage disclosure. Re-checked at
develop@0de9610.Provenance: AI-assisted source audit, human-directed, at
develop@0de9610. Section F was re-counted at this commit; sections A–E describe the auditing environment and are unchanged.Sections A–E (tools not installed, Windows-only auditing box, another vendor's harness, things outside the repo, and what I chose not to run) are statements about my environment and remain accurate as written.
Section F's map of the code is stale.
src/discover/registry.rsno longer contains the rewrite-rule table at all — it moved tosrc/discover/rules.rs— so my sentence pointing atregistry.rsnow names a file with none. The count does not reproduce either: I counted 178 at the time, and at this commitrules.rshas roughly 90 rule entries (bypattern:occurrences). I have not tried to reconcile the difference; it may be a change in how the rules are expressed as much as a change in how many exist.The two byte-size figures in that section are stale by roughly 8–15%.
Nothing else in the disclosure needs editing, and none of it was ever a defect claim.
HetCreep commentedon Sep 16, 2026
Re-checked Section F against
develop@5e0f92c— verified by reading the code at this commit, nothing built or run (this is a citation check, not a behavior claim).This section already needed one correction on this thread: at
develop@0de9610we foundregistry.rshad already lost its rule table, and the original report's "178 rewrite rules" didn't reproduce (recount then: 92). At5e0f92c, 468 commits past that checkpoint, the split still holds and every figure has moved further:registry.rs:1is now//! Matches shell commands against known RTK rewrite rules to decide how to handle them., andregistry.rs:14importsuse super::rules::{IGNORED_EXACT, IGNORED_PREFIXES, RULES, RtkRule};. The rule table itself (pub const RULES: &[RtkRule] = &[...]) is entirely in the separatesrc/discover/rules.rs(1178 lines) —registry.rsonly consumes it, buildingREGEX_SETfromRULESatregistry.rs:52-54.grep -c 'pattern:' src/discover/rules.rs= 96 now, vs. 92 at0de9610, vs. "178" in the original report.registry.rs: 239,148 bytes / 7,381 lines now, vs. 175,712 bytes at0de9610, vs. the original report's "~155 KB".src/hooks/init.rs: 397,414 bytes / 11,109 lines now, vs. 300,248 bytes at0de9610, vs. the original report's "~280 KB".Unaffected: the 63-file
.tomlfilter count elsewhere in this section still reproduces exactly (ls src/filters/*.toml | wc -l= 63).Not a defect — this is a coverage-disclosure note, not a bug report, and its substance (these two files are large, actively growing surfaces the original audit didn't read line-by-line) still holds, if anything more clearly than before. Updating the numbers here so they don't keep drifting further from what's actually on
develop; no action needed on your end.HetCreep commentedon Oct 14, 2026
Re-checked on v1.0.0 (stable, 2026-10-13): the official release binary plus the v1.0.0 source. Provenance: AI-assisted check, human-directed. "ran" = executed against
rtk 1.0.0with a control; "read" = verified from source only.Refresh of the stale cites in section F of this disclosure, checked at v1.0.0 (read only; no binary run). Sections A-E (environment and scope statements) are unaffected.
src/discover/rules.rs(96pattern:lines, not 178);src/discover/registry.rsis the rewrite engine.src/hooks/init.rsis now the directorysrc/hooks/init/(8 files);src/openclawno longer exists..tomlfiles insrc/filters/(the report said 63;gradle.tomlwas removed).set_owner_onlyis still insrc/core/utils.rs:254/265.&strbyte-slicing — the two gaps behind several open panic/false-summary issues #3394 and Install verification tells users to expectrtk 0.28.2, and a shipped skill warns on anything that is notrtk 0.16#3396 are outdated (see the re-checks posted on those threads).