Let the evidence leave the screen - #35
Merged
Merged
Conversation
Closes the first half of #33. Until now the thing this project exists to produce — the named tells, the hidden characters at their line and column, the citations a document makes that its own bibliography contradicts — died when the tab closed. A teacher could be shown all of it and forward none of it. That is a hole in the argument, not a missing feature: a percentage is not evidence, and evidence you cannot hand to anyone is not much better. The report comes out as Markdown for pasting into an email or an LMS, or as a self-contained HTML page that opens by double-click and prints to PDF. No stylesheet, no script, no external request — it has to open from a downloads folder years later on a machine that has never heard of this project, and one that fetched anything would leak which documents were read and when. Three things it always does. It prints this build's own false-positive rate, quoting the upper bound of the interval rather than the observed rate, and names the rules known to misfire so a reader can weigh evidence that leans on one — the sample report counts `rhet.in-terms-of` as a signal and says on the same page that it fires on 10% of human academic writing. It separates checkable facts from the judgement, because a DOI on two different works is not an opinion about prose. And it states that it holds the document and was never uploaded, since the share card is built to be posted in public and confusing the two would publish a student's work. Those numbers come from a calibration snapshot embedded at build time and written by the same tool run that regenerates Docs/CALIBRATION.md, so the page cannot quote a threshold three versions old. A fork that has never measured itself ships no snapshot and the report says so, rather than inheriting a figure it did not earn. CLI: `--report <file>`, written after everything else so a failing --max-score gate still leaves the document behind for whoever has to look at it. The Markdown-to-HTML conversion handles exactly what this file emits and nothing more. The input is produced twenty lines above the converter, so anything it cannot parse is a bug here rather than in someone's document, and a real parser would be a dependency this library refuses to take. Refs #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #33. Depends on #34 — it branches from it, so merge that one first.
Until now the thing this project exists to produce died when the tab closed. A teacher could be shown every named tell, every hidden character with its line and column, every citation a document contradicts in its own bibliography — and forward none of it. That is a hole in the argument rather than a missing feature: a percentage is not evidence, and evidence nobody can hand to anyone is not much better.
The report saves as Markdown for pasting into an email or an LMS, or as a self-contained HTML page that opens by double-click and prints to PDF. No stylesheet, no script, no external request: it has to open from a downloads folder years later on a machine that has never heard of this project, and one that fetched anything would leak which documents were read and when. Buttons in the web app, the desktop app and the folder triage;
--report <file>in the CLI.What the review changed, which is most of what matters here
Two independent adversarial reviews ran against this branch before it was proposed. Between them they stopped it twice. Every severe claim was verified by hand before anything was touched.
It quoted a false-positive rate measured on another language.
Docs/CALIBRATION.mdalready carried the sentence condemning this, one line above its own table: "a rate that holds in English and fails in Spanish is not one number, and reporting it as one would hide exactly the failure that matters here." English bounds at 5.6% on this corpus and Spanish at 13.3%; neither supports the 5% target alone, and both show a dash on that page. The report printed the aggregate 4.1% — handing the author of a Spanish essay a bound three times better than anything measured for their language, under the heading "A score is not proof". The embedded snapshot now carries the strata and the caveat reads the one for the language actually analysed, saying plainly when that language supports no threshold and that the overall figure is not a substitute.It asserted a contradiction it had not checked. "None of this needed the internet: the document contradicts itself" printed whenever there was anything at all to say about sources — including directly beneath the line saying no reference list was found and the cross-checks had not run. A student who simply omitted a bibliography got a page accusing them in the report's own voice. Now conditional on an actual disagreement.
It opened with a verdict it spent four lines disowning. "Reads mostly human" above "treat the score as saying nothing" is a page arguing with itself, and a reader keeps whichever half suits them. Below the threshold the build can support, the number stands alone and the page says a low score is not evidence a person wrote something — a detector that detects nothing also returns zero.
A heading claimed something false about half its own table. "Characters that writing does not produce" covers soft hyphens and unusual spaces, which Word inserts by itself and any copy-paste carries. It now says what is in the file and names the innocent explanations.
User content could break the page. A
|in a filename opened an extra table cell and shifted every number one column right — in the table a teacher reads scores against student names. A newline in an extractor's error closed the list and let the remainder become report prose; a line starting##arrived as a heading in the report's own voice. Both verified, both fixed, both now have a regression test."Nought out of ninety" was hardcoded two lines under the same figures read from the calibration snapshot — the precise failure
PublishedCalibrationexists to prevent, committed inside it. The tool wrote that snapshot relative to the working directory, so running it from anywhere but the repo root left the real one stale.PublishedCalibration.Currentcould throw from a property getter, taking down every report over a malformed resource.What the report now does
Three things on every page. It prints the error rate for the language analysed, from the upper end of the interval, and names the rules known to misfire so a reader can weigh evidence that leans on one. It puts the checkable facts in the headline rather than below the score — the document this project keeps writing about scores near zero and has an invented bibliography, and burying that would be choosing the wrong thing to make salient. And it states that it holds the document and was never uploaded, since the share card is built to be posted in public and confusing the two would publish a student's work.
The folder report is not the single-document one repeated. Above the table, before any name: "This is a reading order, not a ranking. A higher score means look sooner, and nothing more. Nothing on this page establishes that anyone did anything." Every file is listed rather than the first forty — it is the document a teacher keeps, and dropping 160 of 200, every low scorer among them, would be worse than not writing it. Files are named by their path under the scanned folder, since one directory per student makes "essay.docx" ambiguous exactly when it matters.
Known and filed rather than hidden
ToMarkdownpasses raw HTML through, and emphasis markers do not nest. Plus the two-phase build nothing enforces, and custom rule packs invalidating the published rate.244 tests in Core, 323 across the solution, desktop solution included. All green.