From 1b2b1b8444e3cd3f43141f9e1ec33db293374dd3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pedro=20Hern=C3=A1ndez?= Date: Wed, 5 Aug 2026 10:23:09 -0400 Subject: [PATCH] Write the report for whoever has to read it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The report addressed the document and not the reader. Its caveats — the part this project insists must be read, and the only part that limits the score — were English whatever the reader's language, so a Spanish-speaking committee got the number and not the doubt. The two languages are independent and stay that way: quoted findings follow the analysed text, because translating the sentence around a quoted English word produces neither language. Caveats and headings follow the interface. Report prose therefore lives in its own resource keyed by interface language, not in the rule packs, which are selected by the analysed language and would have guaranteed the wrong one for exactly the teacher this is for. Also fixes a bug that was latent only because two languages happen to have strata: a language absent from the corpus inherited the aggregate false-positive rate and the aggregate threshold, and the page announced it had been "analysed as English". A figure this project prints must name the population it was measured on, and a page that goes to a disciplinary meeting must not state a fact that is false. Nothing in C# had to change for that to fire — the rule packs are JSON anyone can send by PR. An incomplete translation marks each block it could not carry. A language with no resource at all carries one notice instead of sixty markers, and is written in English rather than withheld: refusing to render loses the evidence entirely, which is the outcome this feature exists to prevent. Closes #36. Co-Authored-By: Claude Opus 5 --- Docs/TRANSLATING.md | 34 +- .../Reporting/EvidenceReport.cs | 395 ++++++++++------ .../Reporting/ReportMessages.cs | 433 ++++++++++++++++++ src/SignsOfAI.Core/Reporting/report.en.json | 79 ++++ src/SignsOfAI.Core/Reporting/report.es.json | 150 ++++++ src/SignsOfAI.Core/SignsOfAI.Core.csproj | 7 + src/SignsOfAI.Mcp/Tools/ReportTools.cs | 4 +- src/SignsOfAI.UI/Pages/Batch.razor | 6 +- src/SignsOfAI.UI/Pages/Home.razor | 1 + .../EvidenceReportTests.cs | 116 ++++- .../ReportMessageTests.cs | 95 ++++ 11 files changed, 1166 insertions(+), 154 deletions(-) create mode 100644 src/SignsOfAI.Core/Reporting/ReportMessages.cs create mode 100644 src/SignsOfAI.Core/Reporting/report.en.json create mode 100644 src/SignsOfAI.Core/Reporting/report.es.json create mode 100644 tests/SignsOfAI.Core.Tests/ReportMessageTests.cs diff --git a/Docs/TRANSLATING.md b/Docs/TRANSLATING.md index 6edc5ca..c0982b7 100644 --- a/Docs/TRANSLATING.md +++ b/Docs/TRANSLATING.md @@ -1,8 +1,8 @@ # Translating the interface The interface of Signs of AI Writing is translated by whoever wants to translate it. You do not need -to know C#, .NET or Blazor, and you do not need to build anything: a language is **two files' worth -of edits**, both plain JSON. +to know C#, .NET or Blazor. The interface and the saved evidence report are **two independent plain +JSON resources**, because the report's caveats need a stricter completeness check than the app chrome. If you speak a language this tool doesn't, you can add it. That's the whole idea. @@ -32,6 +32,10 @@ on screen; only the text does. `en.json` is the reference. Every other file is a translation of it. +The document a teacher saves has a second resource under +`src/SignsOfAI.Core/Reporting/report..json`. Findings in that document still follow the +language of the analysed text; headings and caveats follow the interface language. + --- ## Adding a new language @@ -52,7 +56,19 @@ change the keys on the left. "home.stat.words": "Mots", ``` -**3. Add one line to `locales.json`:** +**3. Copy `Reporting/report.en.json` to `Reporting/report.fr.json` and translate its values.** Report +translations may be partial, but the fallback notices, caveats, error-rate prose and section headings +must all be present. Those keys form the mandatory core. Until it is complete, reports are written in +English and say on their face that they are — because a report whose limitation cannot be read is +worse than no report, and a report that will not print at all is worse than either: the evidence is +lost and nothing explains why. + +Every translated report entry also has a `sourceHash`. It pins that sentence to the exact English +sentence it translated. If English changes later, the tests print the new expected hash and the old +translation falls back visibly until a speaker reviews it. Do not update a hash without rereading the +new English source. + +**4. Add one line to `locales.json`:** ```json { @@ -72,17 +88,25 @@ change the keys on the left. | `endonym` | The language's name **in that language** — `Français`, not `French`. A French speaker looks for `Français`. | | `credit` | You. Shown when hovering the language switch, so contributors get named. Leave `""` to stay anonymous. | -**4. Open a pull request.** That's it — the switch picks the new language up automatically, no code +**5. Open a pull request.** That's it — the switch picks the new language up automatically, no code change anywhere. --- ## You don't have to finish -**A partial translation is welcome.** Any key you leave out falls back to English at run time, so +**A partial interface translation is welcome.** Any key you leave out falls back to English at run time, so half a translation ships as half-translated — not as a page full of blanks. Translate the navigation and the main page, open the PR, come back for the rest whenever. +A partial report translation is welcome once its mandatory core is complete. Every untranslated +report block carries a notice in the reader's language and the total appears near the top; it never +looks silently complete. + +Before the core is complete the report is not withheld — it is written in English, with one notice at +the top naming the language it could not be written in. One honest notice, rather than sixty markers +on a page that fell back entirely. + You can also simply **delete** any key you're unsure about. Deleting is safer than guessing: a deleted key shows English, while a wrong translation shows something wrong. diff --git a/src/SignsOfAI.Core/Reporting/EvidenceReport.cs b/src/SignsOfAI.Core/Reporting/EvidenceReport.cs index e93f0eb..2f82628 100644 --- a/src/SignsOfAI.Core/Reporting/EvidenceReport.cs +++ b/src/SignsOfAI.Core/Reporting/EvidenceReport.cs @@ -43,33 +43,38 @@ public static class EvidenceReport public static string ToMarkdown(AnalysisResult result, ReportOptions? options = null) { var o = options ?? ReportOptions.Default; + var text = ReportMessages.For(o.InterfaceLanguage); var sb = new StringBuilder(); + var title = string.Equals(o.Title, ReportOptions.Default.Title, StringComparison.Ordinal) + ? text.Get(ReportMessages.DefaultTitle).Text + : o.Title; - sb.Append("# ").Append(o.Title).AppendLine(); + sb.Append("# ").Append(title).AppendLine(); sb.AppendLine(); + var fallbackNoticeAt = sb.Length; if (!string.IsNullOrWhiteSpace(o.DocumentName)) - sb.Append("**Document:** ").AppendLine(Cell(o.DocumentName)); - sb.Append("**Generated:** ").Append(o.GeneratedOn).Append(" · **Engine:** SignsOfAI ") - .AppendLine(o.EngineVersion); + AppendBlock(sb, text, ReportMessages.MetaDocument, Cell(o.DocumentName)); + AppendBlock(sb, text, ReportMessages.MetaGenerated, o.GeneratedOn, o.EngineVersion); sb.AppendLine(); // ── The reading, and immediately the caveat that makes it usable ────────────────────────── - sb.AppendLine("## What the analysis says"); + AppendHeading(sb, text, 2, ReportMessages.SectionAnalysis); sb.AppendLine(); // The verdict is withheld below the threshold this build can support, because printing // "Reads mostly human" and then, four lines down, "treat the score as saying nothing" is a // page arguing with itself — and the reader will keep whichever half suits them. A low score // is not evidence of a human: a detector that detects nothing also returns zero, and this // project deliberately never measured how much machine writing it catches. - sb.Append("**").Append(Num(result.OverallScore)).Append("/100"); if (VerdictHolds(result)) - sb.Append(" — ").Append(result.Verdict); - sb.AppendLine("**"); + AppendBlock(sb, text, ReportMessages.AnalysisScoreWithVerdict, + Num(result.OverallScore), Verdict(text, result.OverallScore)); + else + AppendBlock(sb, text, ReportMessages.AnalysisScoreWithoutVerdict, + Num(result.OverallScore)); sb.AppendLine(); if (!VerdictHolds(result)) { - sb.AppendLine("*Below the threshold this build can support, so no verdict is given. A low " + - "score is not evidence that a person wrote this.*"); + AppendBlock(sb, text, ReportMessages.AnalysisNoVerdict); sb.AppendLine(); } @@ -78,43 +83,56 @@ public static string ToMarkdown(AnalysisResult result, ReportOptions? options = // to make salient — and that exact document is the one this project keeps writing about. if (result.Citations.Issues.Count > 0 || result.Artifacts.Any) { - sb.Append("**Checkable facts found: "); - var parts = new List(); - if (result.Citations.Issues.Count > 0) - parts.Add($"{result.Citations.Issues.Count} source contradiction" + - (result.Citations.Issues.Count == 1 ? "" : "s")); - if (result.Artifacts.Any) - parts.Add($"{result.Artifacts.Count} unusual character" + - (result.Artifacts.Count == 1 ? "" : "s")); - sb.Append(string.Join(", ", parts)).AppendLine(". These did not move the score.**"); + if (result.Citations.Issues.Count > 0 && result.Artifacts.Any) + AppendBlock(sb, text, + result.Citations.Issues.Count == 1 + ? result.Artifacts.Count == 1 + ? ReportMessages.AnalysisFactsBothOneOne + : ReportMessages.AnalysisFactsBothOneOther + : result.Artifacts.Count == 1 + ? ReportMessages.AnalysisFactsBothOtherOne + : ReportMessages.AnalysisFactsBothOtherOther, + result.Citations.Issues.Count, result.Artifacts.Count); + else if (result.Citations.Issues.Count > 0) + AppendBlock(sb, text, result.Citations.Issues.Count == 1 + ? ReportMessages.AnalysisFactsCitationOne + : ReportMessages.AnalysisFactsCitationOther, + result.Citations.Issues.Count); + else + AppendBlock(sb, text, result.Artifacts.Count == 1 + ? ReportMessages.AnalysisFactsArtifactOne + : ReportMessages.AnalysisFactsArtifactOther, + result.Artifacts.Count); sb.AppendLine(); } - sb.Append("- ").Append(result.Signals.Count).Append(" signal") - .Append(result.Signals.Count == 1 ? "" : "s").Append(" counted"); if (result.Observations.Count > 0) - sb.Append(", plus ").Append(result.Observations.Count) - .Append(" found at a rate people write at, which count for nothing"); - sb.AppendLine(); - sb.Append("- Analysed as ").Append(result.Language == "es" ? "Spanish" : "English") - .Append(" · ").Append(result.Statistics.WordCount).Append(" words · ") - .Append(result.Statistics.SentenceCount).Append(" sentences · sentence-length variability ") - .AppendLine(Num(result.Statistics.Burstiness, 2)); + AppendBlock(sb, text, result.Signals.Count == 1 + ? ReportMessages.AnalysisCountsWithObservationsOne + : ReportMessages.AnalysisCountsWithObservationsOther, + result.Signals.Count, result.Observations.Count); + else + AppendBlock(sb, text, result.Signals.Count == 1 + ? ReportMessages.AnalysisCountsOne + : ReportMessages.AnalysisCountsOther, + result.Signals.Count); + AppendBlock(sb, text, ReportMessages.AnalysisLanguageStats, + LanguageName(text, result.Language), result.Statistics.WordCount, + result.Statistics.SentenceCount, Num(result.Statistics.Burstiness, 2)); sb.AppendLine(); - sb.AppendLine(Caveat(result.Language)); + AppendLocalized(sb, text, Caveat(text, result.Language)); sb.AppendLine(); // ── Checkable facts first, because they are the part that settles anything ──────────────── if (result.Artifacts.Any || result.Citations.Any) { - sb.AppendLine("## Checkable facts"); + AppendHeading(sb, text, 2, ReportMessages.SectionCheckable); sb.AppendLine(); - sb.AppendLine("These are not judgements about the writing and they did not move the score. " + - "Each is either present in the file or it is not."); + AppendBlock(sb, text, ReportMessages.CheckableIntro); sb.AppendLine(); if (result.Artifacts.Any) { - sb.AppendLine("### Characters found in the file"); + AppendHeading(sb, text, 3, ReportMessages.SectionCharacters); sb.AppendLine(); sb.AppendLine(result.Artifacts.Summary); sb.AppendLine(); @@ -123,27 +141,26 @@ public static string ToMarkdown(AnalysisResult result, ReportOptions? options = // non-breaking space arrives with any copy-paste from a web page. Only the invisible // and impostor-letter kinds are hard to arrive at innocently, and even they can be // pasted in. Saying so here is the difference between a fact and an insinuation. - sb.AppendLine("Several of these have ordinary explanations — word processors insert " + - "soft hyphens and unusual spaces on their own, and any copy-paste can " + - "carry them. Invisible characters and letters borrowed from another " + - "alphabet are harder to arrive at by accident, though pasting text can " + - "do it. This table says what is in the file, not how it got there."); + AppendBlock(sb, text, ReportMessages.CharactersExplanation); sb.AppendLine(); - sb.AppendLine("| Character | Codepoint | Line | Column |"); + AppendBlock(sb, text, ReportMessages.CharactersTableHeader); sb.AppendLine("|---|---|---:|---:|"); foreach (var occurrence in result.Artifacts.Occurrences.Take(o.MaxRows)) sb.Append("| ").Append(Cell(Describe(occurrence.Kind))).Append(" | `") .Append(occurrence.CodePoint).Append("` | ").Append(occurrence.Line) .Append(" | ").Append(occurrence.Column).AppendLine(" |"); if (result.Artifacts.Occurrences.Count > o.MaxRows) - sb.Append("\n… and ").Append(result.Artifacts.Occurrences.Count - o.MaxRows) - .AppendLine(" more."); + { + sb.AppendLine(); + AppendBlock(sb, text, ReportMessages.MoreRows, + result.Artifacts.Occurrences.Count - o.MaxRows); + } sb.AppendLine(); } if (result.Citations.Any) { - sb.AppendLine("### What the document says about its own sources"); + AppendHeading(sb, text, 3, ReportMessages.SectionCitations); sb.AppendLine(); sb.AppendLine(result.Citations.Summary); sb.AppendLine(); @@ -156,20 +173,19 @@ public static string ToMarkdown(AnalysisResult result, ReportOptions? options = // its own voice, a self-contradiction it had explicitly not looked for. In a document // that goes to a committee about a nineteen-year-old, that is the precise harm this // project exists to argue against. - sb.AppendLine(result.Citations.Issues.Count > 0 - ? "> None of this needed the internet: the document disagrees with itself. It is a " + - "question to ask, not a conclusion — the answer is usually one sentence." - : "> Nothing here is a finding. It describes what could and could not be checked."); + AppendBlock(sb, text, result.Citations.Issues.Count > 0 + ? ReportMessages.CitationsIssuesNote + : ReportMessages.CitationsNoIssuesNote); sb.AppendLine(); } } // ── The judgement, clearly labelled as one ─────────────────────────────────────────────── - sb.AppendLine("## Signals counted"); + AppendHeading(sb, text, 2, ReportMessages.SectionSignals); sb.AppendLine(); if (result.Signals.Count == 0) { - sb.AppendLine("None."); + AppendBlock(sb, text, ReportMessages.SignalsNone); } else { @@ -181,32 +197,36 @@ public static string ToMarkdown(AnalysisResult result, ReportOptions? options = sb.Append(Cell(f.Message)).Append(' ').Append("*→ ").Append(Cell(f.Suggestion)).AppendLine("*"); } if (result.Signals.Count > o.MaxRows) - sb.Append("\n… and ").Append(result.Signals.Count - o.MaxRows).AppendLine(" more."); + { + sb.AppendLine(); + AppendBlock(sb, text, ReportMessages.MoreRows, result.Signals.Count - o.MaxRows); + } } sb.AppendLine(); if (result.Observations.Count > 0) { - sb.AppendLine("## Found, but at a rate people write at"); + AppendHeading(sb, text, 2, ReportMessages.SectionObservations); sb.AppendLine(); - sb.AppendLine("Measured against writing published before generative models existed. Shown " + - "because they are real, and counted for nothing because they are ordinary."); + AppendBlock(sb, text, ReportMessages.ObservationsIntro); sb.AppendLine(); foreach (var group in result.Observations.GroupBy(f => f.RuleId).Take(o.MaxRows)) - sb.Append("- ").Append(group.Key).Append(" — ").Append(group.Count()) - .Append(group.Count() == 1 ? " occurrence" : " occurrences").AppendLine(); + AppendBlock(sb, text, group.Count() == 1 + ? ReportMessages.ObservationsRowOne + : ReportMessages.ObservationsRowOther, + group.Key, group.Count()); sb.AppendLine(); } - sb.AppendLine("## How often this is wrong"); + AppendHeading(sb, text, 2, ReportMessages.SectionErrorRate); sb.AppendLine(); - sb.AppendLine(HowOftenWrong()); + HowOftenWrong(sb, text, result.Language); sb.AppendLine(); sb.AppendLine("---"); sb.AppendLine(); - sb.AppendLine("*This report was produced on the device that ran the analysis and contains " + - "material from the document it describes. It is yours to keep or to send; nothing " + - "here was uploaded anywhere.*"); + AppendBlock(sb, text, ReportMessages.PrivacyDocument); + + InsertFallbackSummary(sb, fallbackNoticeAt, text); return sb.ToString(); } @@ -220,14 +240,18 @@ public static string ToMarkdown(AnalysisResult result, ReportOptions? options = public static string ToHtml(AnalysisResult result, ReportOptions? options = null) { var o = options ?? ReportOptions.Default; + var text = ReportMessages.For(o.InterfaceLanguage); + var title = string.Equals(o.Title, ReportOptions.Default.Title, StringComparison.Ordinal) + ? text.Get(ReportMessages.DefaultTitle).Text + : o.Title; var body = MarkdownToHtml(ToMarkdown(result, o)); return $""" - + - {Escape(o.Title)} + {Escape(title)} @@ -249,29 +273,37 @@ public static string FolderToMarkdown( string folderName, IReadOnlyList entries, ReportOptions? options = null) { var o = options ?? ReportOptions.Default; + var text = ReportMessages.For(o.InterfaceLanguage); var sb = new StringBuilder(); + var title = string.Equals(o.Title, ReportOptions.Default.Title, StringComparison.Ordinal) + ? text.Get(ReportMessages.DefaultTitle).Text + : o.Title; // An error wins over a score: a file that failed to read has no business in the reading order, // and nothing on the public FolderEntry stops a caller supplying both. var unreadable = entries.Where(e => e.Error is not null).ToList(); var scored = entries.Where(e => e.Error is null && e.Score is not null).ToList(); - sb.Append("# ").AppendLine(o.Title); + sb.Append("# ").AppendLine(title); sb.AppendLine(); - sb.Append("**Folder:** ").AppendLine(Cell(folderName)); - sb.Append("**Generated:** ").Append(o.GeneratedOn).Append(" · **Engine:** SignsOfAI ") - .AppendLine(o.EngineVersion); + var fallbackNoticeAt = sb.Length; + AppendBlock(sb, text, ReportMessages.MetaFolder, Cell(folderName)); + AppendBlock(sb, text, ReportMessages.MetaGenerated, o.GeneratedOn, o.EngineVersion); sb.AppendLine(); - sb.Append(entries.Count).Append(" file").Append(entries.Count == 1 ? "" : "s").Append(" scanned"); - if (unreadable.Count > 0) sb.Append(", ").Append(unreadable.Count).Append(" unreadable"); - sb.AppendLine("."); + AppendBlock(sb, text, unreadable.Count > 0 + ? entries.Count == 1 + ? ReportMessages.FolderSummaryUnreadableOne + : ReportMessages.FolderSummaryUnreadableOther + : entries.Count == 1 + ? ReportMessages.FolderSummaryOne + : ReportMessages.FolderSummaryOther, + unreadable.Count > 0 ? [entries.Count, unreadable.Count] : [entries.Count]); sb.AppendLine(); - sb.AppendLine(Caveat(null)); + AppendLocalized(sb, text, Caveat(text, null)); sb.AppendLine(); - sb.AppendLine("> **This is a reading order, not a ranking.** A higher score means look sooner, " + - "and nothing more. Nothing on this page establishes that anyone did anything."); + AppendBlock(sb, text, ReportMessages.FolderReadingOrder); sb.AppendLine(); - sb.AppendLine("| File | Score | Signals | Words |"); + AppendBlock(sb, text, ReportMessages.FolderTableHeader); sb.AppendLine("|---|---:|---:|---:|"); // Every file, deliberately unlike the findings lists. This is the document a teacher keeps // after closing the app, and a scan of two hundred essays that silently omitted a hundred and @@ -286,21 +318,23 @@ public static string FolderToMarkdown( if (unreadable.Count > 0) { - sb.AppendLine("## Could not be read"); + AppendHeading(sb, text, 2, ReportMessages.SectionUnreadable); sb.AppendLine(); foreach (var e in unreadable) - sb.Append("- ").Append(Cell(e.Name)).Append(" — ").AppendLine(Cell(e.Error)); + AppendBlock(sb, text, ReportMessages.FolderUnreadableRow, + Cell(e.Name), Cell(e.Error)); sb.AppendLine(); } - sb.AppendLine("## How often this is wrong"); + AppendHeading(sb, text, 2, ReportMessages.SectionErrorRate); sb.AppendLine(); - sb.AppendLine(HowOftenWrong()); + HowOftenWrong(sb, text, null); sb.AppendLine(); sb.AppendLine("---"); sb.AppendLine(); - sb.AppendLine("*Produced on the device that scanned the folder. It names your students' files, " + - "so treat it as you would the coursework itself; nothing here was uploaded anywhere.*"); + AppendBlock(sb, text, ReportMessages.PrivacyFolder); + + InsertFallbackSummary(sb, fallbackNoticeAt, text); return sb.ToString(); } @@ -310,12 +344,16 @@ public static string FolderToHtml( string folderName, IReadOnlyList entries, ReportOptions? options = null) { var o = options ?? ReportOptions.Default; + var text = ReportMessages.For(o.InterfaceLanguage); + var title = string.Equals(o.Title, ReportOptions.Default.Title, StringComparison.Ordinal) + ? text.Get(ReportMessages.DefaultTitle).Text + : o.Title; return $""" - + - {Escape(o.Title)} + {Escape(title)} @@ -333,8 +371,10 @@ private static bool VerdictHolds(AnalysisResult result) var c = PublishedCalibration.Current; if (c is null) return false; - var threshold = c.For(result.Language)?.RecommendedThreshold ?? c.RecommendedThreshold; - return threshold is { } t && result.OverallScore >= t; + // Never borrow the aggregate. A language absent from the corpus has no supported verdict, + // even when the combined EN/ES sample happens to support one. + return c.For(result.Language)?.RecommendedThreshold is { } threshold + && result.OverallScore >= threshold; } /// @@ -342,97 +382,160 @@ private static bool VerdictHolds(AnalysisResult result) /// cannot go stale, and explicit when there is none: a fork that has not measured itself says so /// rather than inheriting a number it did not earn. /// - private static string Caveat(string? language) + private static LocalizedReportText Caveat(ReportText text, string? language) { var c = PublishedCalibration.Current; if (c is null) - return "> **This build has not been calibrated.** No false-positive rate has been measured " + - "for it, so the score above should not be used to support a decision about a person."; + return text.Get(ReportMessages.CaveatUncalibrated); - // Measured, but on too little text to support any threshold. Saying "not calibrated" here - // would contradict the section further down, which goes on to name the corpus and the date. - if (c.RecommendedThreshold is not { } threshold) - return $"> **No threshold is supported yet.** This build was measured against {c.Texts} " + - "texts, too few to bound its false-positive rate, so no score on this page should " + - "be used to support a decision about a person."; - - // The language actually analysed, never the aggregate. Docs/CALIBRATION.md says it one line - // above its own table: a rate that holds in English and fails in Spanish is not one number. - // On the current corpus English bounds at 5.6% and Spanish at 13.3%, and neither group is big - // enough to support the target alone — so a Spanish essay quoting the overall 4.1% would be - // handed a bound three times better than anything measured for its language. - if (c.For(language) is { } group) + // A single-document report always needs the stratum for the language it actually analysed. + // Falling through to the aggregate here would attach a measurement from other languages to + // writing the corpus never contained. + if (!string.IsNullOrWhiteSpace(language)) { + var group = c.For(language); + if (group is null) + return text.Get(ReportMessages.CaveatLanguageUnmeasured, + LanguageName(text, language)); + if (group.RecommendedThreshold is not { } languageThreshold) - return $"> **No threshold is supported for this language yet.** The corpus holds " + - $"{group.Texts} texts in it — too few to bound how often this build is wrong " + - $"about writing in it, so no score on this page should be used to support a " + - $"decision about a person. The best bound these texts support is " + - $"{Pct(group.BestBound)}, and the overall figure is not a substitute for it."; - - return $"> **A score is not proof.** On {group.Texts} texts in this language, published " + - $"before generative models existed, this build's false-positive rate at a threshold " + - $"of {Num(languageThreshold)}/100 was under {Pct(group.BestBound)} — the upper end " + - $"of a 95% interval, not a guarantee, and measured on published articles rather " + - $"than student work. Below that threshold, treat the score as saying nothing."; + return text.Get(ReportMessages.CaveatLanguageNoThreshold, + group.Texts, Pct(group.BestBound)); + + return text.Get(ReportMessages.CaveatLanguageMeasured, + group.Texts, Num(languageThreshold), Pct(group.BestBound)); } + // Measured, but on too little text to support any threshold. Saying "not calibrated" here + // would contradict the section further down, which goes on to name the corpus and the date. + if (c.RecommendedThreshold is not { } threshold) + return text.Get(ReportMessages.CaveatAggregateNoThreshold, c.Texts); + // "at most" was a guarantee, and a 95% interval does not give one. The corpus is also one // genre of writing, so generalising from it to "human writing" is the caller's inference and // not this sentence's claim. - return $"> **A score is not proof.** On {c.Texts} texts published before generative models " + - $"existed, this build's false-positive rate at a threshold of {Num(threshold)}/100 was " + - $"under {Pct(c.RateHigh)} — the upper end of a 95% interval, not a guarantee, and " + - $"measured on published articles rather than student work. Below that threshold, treat " + - $"the score as saying nothing."; + return text.Get(ReportMessages.CaveatAggregateMeasured, + c.Texts, Num(threshold), Pct(c.RateHigh)); } - private static string HowOftenWrong() + private static void HowOftenWrong(StringBuilder sb, ReportText text, string? language) { var c = PublishedCalibration.Current; if (c is null) - return "This build ships no calibration, so nothing is known about how often it is wrong. " + - "That is itself the most important thing on this page."; + { + AppendBlock(sb, text, ReportMessages.HowUncalibrated); + return; + } - var sb = new StringBuilder(); - sb.Append("Measured against **").Append(c.Texts) - .Append(" texts published before generative models existed**, so their authorship rests on ") - .Append("their dates rather than on anybody's judgement. Measured on ").Append(c.MeasuredOn) - .Append(" with engine ").Append(c.Engine).AppendLine("."); - sb.AppendLine(); + if (!string.IsNullOrWhiteSpace(language)) + { + var group = c.For(language); + if (group is null) + { + AppendBlock(sb, text, ReportMessages.HowLanguageUnmeasured, + LanguageName(text, language)); + sb.AppendLine(); + AppendBlock(sb, text, ReportMessages.HowLimitation); + return; + } - if (c.RecommendedThreshold is { } threshold) + if (group.RecommendedThreshold is { } languageThreshold) + AppendBlock(sb, text, ReportMessages.HowLanguageMeasured, + group.Texts, Num(languageThreshold), Pct(group.BestBound), c.MeasuredOn, c.Engine); + else + AppendBlock(sb, text, ReportMessages.HowLanguageNoThreshold, + group.Texts, Pct(group.BestBound), c.MeasuredOn, c.Engine); + } + else { - sb.Append("At **").Append(Num(threshold)).Append("/100**, ").Append(c.FlaggedAtThreshold) - .Append(" of those ").Append(c.Texts).Append(" were flagged — an observed ") - .Append(Pct((double)c.FlaggedAtThreshold / Math.Max(c.Texts, 1))) - .Append(", with a 95% interval of ").Append(Pct(c.RateLow)).Append(" – ") - .Append(Pct(c.RateHigh)).AppendLine("."); - sb.AppendLine(); - sb.Append("Read the interval, not the observed rate. ").Append(c.FlaggedAtThreshold) - .Append(" out of ").Append(c.Texts) - .AppendLine(" is not a false-positive rate you can round down."); + AppendBlock(sb, text, ReportMessages.HowAggregateIntro, + c.Texts, c.MeasuredOn, c.Engine); + + if (c.RecommendedThreshold is { } threshold) + { + sb.AppendLine(); + AppendBlock(sb, text, ReportMessages.HowAggregateThreshold, + Num(threshold), c.FlaggedAtThreshold, c.Texts, + Pct((double)c.FlaggedAtThreshold / Math.Max(c.Texts, 1)), + Pct(c.RateLow), Pct(c.RateHigh)); + sb.AppendLine(); + AppendBlock(sb, text, ReportMessages.HowReadInterval, + c.FlaggedAtThreshold, c.Texts); + } } + // The noisiest-rule rates are aggregate measurements. They remain useful for a language that + // is represented in the corpus, but are withheld entirely for an unmeasured language above. if (c.NoisiestRules.Count > 0) { sb.AppendLine(); - sb.AppendLine("The rules seen most often on that human writing, worst first — if the " + - "evidence above leans on one of these, weigh it accordingly:"); + AppendBlock(sb, text, ReportMessages.HowNoisyIntro); sb.AppendLine(); foreach (var rule in c.NoisiestRules) - sb.Append("- `").Append(rule.RuleId).Append("` — ").Append(Pct(rule.TextShare)) - .AppendLine(" of human texts"); + AppendBlock(sb, text, ReportMessages.HowNoisyRule, + rule.RuleId, Pct(rule.TextShare)); } sb.AppendLine(); - sb.AppendLine("What this does **not** tell you: how much machine-written text it catches. That " + - "is the other half of the picture and it is deliberately not measured here, because " + - "any collection of machine-written text samples whichever models were convenient " + - "that month. A tool that flags nothing has a perfect false-positive rate."); + AppendBlock(sb, text, ReportMessages.HowLimitation); + } - return sb.ToString(); + private static string Verdict(ReportText text, double score) => text.Get(score switch + { + >= 70 => ReportMessages.VerdictStrong, + >= 45 => ReportMessages.VerdictModerate, + >= 20 => ReportMessages.VerdictLight, + _ => ReportMessages.VerdictMinimal, + }).Text; + + private static string LanguageName(ReportText text, string language) => text.Get( + language.Equals("en", StringComparison.OrdinalIgnoreCase) ? ReportMessages.LanguageEnglish + : language.Equals("es", StringComparison.OrdinalIgnoreCase) ? ReportMessages.LanguageSpanish + : ReportMessages.LanguageOther, + language.Equals("en", StringComparison.OrdinalIgnoreCase) + || language.Equals("es", StringComparison.OrdinalIgnoreCase) ? [] : [language]).Text; + + private static void AppendBlock( + StringBuilder sb, ReportText text, string key, params object?[] args) => + AppendLocalized(sb, text, text.Get(key, args)); + + private static void AppendHeading(StringBuilder sb, ReportText text, int level, string key) + { + var value = text.Get(key); + if (value.FellBack) + { + sb.Append('*').Append(text.FallbackMarker).AppendLine("*"); + sb.AppendLine(); + } + + sb.Append('#', level).Append(' ').AppendLine(value.Text); + } + + private static void AppendLocalized(StringBuilder sb, ReportText text, LocalizedReportText value) + { + if (value.FellBack) + { + sb.Append('*').Append(text.FallbackMarker).AppendLine("*"); + sb.AppendLine(); + } + + sb.AppendLine(value.Text); + } + + private static void InsertFallbackSummary(StringBuilder sb, int at, ReportText text) + { + // A language with no resource at all is one notice, not sixty markers: every block fell back, + // so marking each of them would bury the page in its own apology. + if (text.UnavailableLanguage is { } missing) + { + var notice = text.Get(ReportMessages.FallbackLanguage, LanguageName(text, missing)).Text; + sb.Insert(at, $"*{notice}*{Environment.NewLine}{Environment.NewLine}"); + return; + } + + if (text.FallbackBlocks == 0) return; + sb.Insert(at, $"*{text.FallbackSummary}*{Environment.NewLine}{Environment.NewLine}"); } @@ -626,6 +729,14 @@ public sealed record ReportOptions { public string Title { get; init; } = "Writing analysis report"; + /// + /// Language of the reader-facing report structure and caveats, independent from the language of + /// the analysed text. A language without the mandatory report core is rejected rather than + /// silently producing unreadable caveats; a valid partial translation marks every block that + /// falls back. + /// + public string InterfaceLanguage { get; init; } = "en"; + /// The file or assignment this describes. Blank when the text was pasted. public string DocumentName { get; init; } = ""; diff --git a/src/SignsOfAI.Core/Reporting/ReportMessages.cs b/src/SignsOfAI.Core/Reporting/ReportMessages.cs new file mode 100644 index 0000000..bb09c2e --- /dev/null +++ b/src/SignsOfAI.Core/Reporting/ReportMessages.cs @@ -0,0 +1,433 @@ +using System.Collections.Concurrent; +using System.Globalization; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace SignsOfAI.Core.Reporting; + +/// +/// Interface-language prose used by . Findings themselves deliberately +/// do not live here: they come from the rule pack selected for the language of the analysed text. +/// +public static class ReportMessages +{ + public const string FallbackMarker = "fallback.marker"; + public const string FallbackSummary = "fallback.summary"; + public const string FallbackLanguage = "fallback.language"; + public const string DefaultTitle = "default.title"; + public const string MetaDocument = "meta.document"; + public const string MetaGenerated = "meta.generated"; + public const string MetaFolder = "meta.folder"; + public const string SectionAnalysis = "section.analysis"; + public const string SectionCheckable = "section.checkable"; + public const string SectionCharacters = "section.characters"; + public const string SectionCitations = "section.citations"; + public const string SectionSignals = "section.signals"; + public const string SectionObservations = "section.observations"; + public const string SectionErrorRate = "section.error-rate"; + public const string SectionUnreadable = "section.unreadable"; + public const string VerdictStrong = "verdict.strong"; + public const string VerdictModerate = "verdict.moderate"; + public const string VerdictLight = "verdict.light"; + public const string VerdictMinimal = "verdict.minimal"; + public const string AnalysisScoreWithVerdict = "analysis.score.with-verdict"; + public const string AnalysisScoreWithoutVerdict = "analysis.score.without-verdict"; + public const string AnalysisNoVerdict = "analysis.no-verdict"; + public const string AnalysisFactsCitationOne = "analysis.facts.citation.one"; + public const string AnalysisFactsCitationOther = "analysis.facts.citation.other"; + public const string AnalysisFactsArtifactOne = "analysis.facts.artifact.one"; + public const string AnalysisFactsArtifactOther = "analysis.facts.artifact.other"; + public const string AnalysisFactsBothOneOne = "analysis.facts.both.one-one"; + public const string AnalysisFactsBothOneOther = "analysis.facts.both.one-other"; + public const string AnalysisFactsBothOtherOne = "analysis.facts.both.other-one"; + public const string AnalysisFactsBothOtherOther = "analysis.facts.both.other-other"; + public const string AnalysisCountsOne = "analysis.counts.one"; + public const string AnalysisCountsOther = "analysis.counts.other"; + public const string AnalysisCountsWithObservationsOne = "analysis.counts-with-observations.one"; + public const string AnalysisCountsWithObservationsOther = "analysis.counts-with-observations.other"; + public const string AnalysisLanguageStats = "analysis.language-stats"; + public const string LanguageEnglish = "language.en"; + public const string LanguageSpanish = "language.es"; + public const string LanguageOther = "language.other"; + public const string CaveatUncalibrated = "caveat.uncalibrated"; + public const string CaveatAggregateNoThreshold = "caveat.aggregate-no-threshold"; + public const string CaveatLanguageUnmeasured = "caveat.language-unmeasured"; + public const string CaveatLanguageNoThreshold = "caveat.language-no-threshold"; + public const string CaveatLanguageMeasured = "caveat.language-measured"; + public const string CaveatAggregateMeasured = "caveat.aggregate-measured"; + public const string CheckableIntro = "checkable.intro"; + public const string CharactersExplanation = "characters.explanation"; + public const string CharactersTableHeader = "characters.table-header"; + public const string MoreRows = "common.more-rows"; + public const string CitationsIssuesNote = "citations.issues-note"; + public const string CitationsNoIssuesNote = "citations.no-issues-note"; + public const string SignalsNone = "signals.none"; + public const string ObservationsIntro = "observations.intro"; + public const string ObservationsRowOne = "observations.row.one"; + public const string ObservationsRowOther = "observations.row.other"; + public const string PrivacyDocument = "privacy.document"; + public const string FolderSummaryOne = "folder.summary.one"; + public const string FolderSummaryOther = "folder.summary.other"; + public const string FolderSummaryUnreadableOne = "folder.summary-unreadable.one"; + public const string FolderSummaryUnreadableOther = "folder.summary-unreadable.other"; + public const string FolderReadingOrder = "folder.reading-order"; + public const string FolderTableHeader = "folder.table-header"; + public const string FolderUnreadableRow = "folder.unreadable-row"; + public const string PrivacyFolder = "privacy.folder"; + public const string HowUncalibrated = "how.uncalibrated"; + public const string HowLanguageUnmeasured = "how.language-unmeasured"; + public const string HowLanguageNoThreshold = "how.language-no-threshold"; + public const string HowLanguageMeasured = "how.language-measured"; + public const string HowAggregateIntro = "how.aggregate-intro"; + public const string HowAggregateThreshold = "how.aggregate-threshold"; + public const string HowReadInterval = "how.read-interval"; + public const string HowNoisyIntro = "how.noisy-intro"; + public const string HowNoisyRule = "how.noisy-rule"; + public const string HowLimitation = "how.limitation"; + + /// How many {n} placeholders each template takes. + public static IReadOnlyDictionary Arity { get; } = new Dictionary(StringComparer.Ordinal) + { + [FallbackMarker] = 0, + [FallbackSummary] = 1, + [FallbackLanguage] = 1, + [DefaultTitle] = 0, + [MetaDocument] = 1, + [MetaGenerated] = 2, + [MetaFolder] = 1, + [SectionAnalysis] = 0, + [SectionCheckable] = 0, + [SectionCharacters] = 0, + [SectionCitations] = 0, + [SectionSignals] = 0, + [SectionObservations] = 0, + [SectionErrorRate] = 0, + [SectionUnreadable] = 0, + [VerdictStrong] = 0, + [VerdictModerate] = 0, + [VerdictLight] = 0, + [VerdictMinimal] = 0, + [AnalysisScoreWithVerdict] = 2, + [AnalysisScoreWithoutVerdict] = 1, + [AnalysisNoVerdict] = 0, + [AnalysisFactsCitationOne] = 1, + [AnalysisFactsCitationOther] = 1, + [AnalysisFactsArtifactOne] = 1, + [AnalysisFactsArtifactOther] = 1, + [AnalysisFactsBothOneOne] = 2, + [AnalysisFactsBothOneOther] = 2, + [AnalysisFactsBothOtherOne] = 2, + [AnalysisFactsBothOtherOther] = 2, + [AnalysisCountsOne] = 1, + [AnalysisCountsOther] = 1, + [AnalysisCountsWithObservationsOne] = 2, + [AnalysisCountsWithObservationsOther] = 2, + [AnalysisLanguageStats] = 4, + [LanguageEnglish] = 0, + [LanguageSpanish] = 0, + [LanguageOther] = 1, + [CaveatUncalibrated] = 0, + [CaveatAggregateNoThreshold] = 1, + [CaveatLanguageUnmeasured] = 1, + [CaveatLanguageNoThreshold] = 2, + [CaveatLanguageMeasured] = 3, + [CaveatAggregateMeasured] = 3, + [CheckableIntro] = 0, + [CharactersExplanation] = 0, + [CharactersTableHeader] = 0, + [MoreRows] = 1, + [CitationsIssuesNote] = 0, + [CitationsNoIssuesNote] = 0, + [SignalsNone] = 0, + [ObservationsIntro] = 0, + [ObservationsRowOne] = 2, + [ObservationsRowOther] = 2, + [PrivacyDocument] = 0, + [FolderSummaryOne] = 1, + [FolderSummaryOther] = 1, + [FolderSummaryUnreadableOne] = 2, + [FolderSummaryUnreadableOther] = 2, + [FolderReadingOrder] = 0, + [FolderTableHeader] = 0, + [FolderUnreadableRow] = 2, + [PrivacyFolder] = 0, + [HowUncalibrated] = 0, + [HowLanguageUnmeasured] = 1, + [HowLanguageNoThreshold] = 4, + [HowLanguageMeasured] = 5, + [HowAggregateIntro] = 3, + [HowAggregateThreshold] = 6, + [HowReadInterval] = 2, + [HowNoisyIntro] = 0, + [HowNoisyRule] = 2, + [HowLimitation] = 0, + }; + + /// + /// English last-resort wording. These values are the source of truth for translation hashes and + /// preserve the report's previous English text. + /// + public static IReadOnlyDictionary Defaults { get; } = new Dictionary(StringComparer.Ordinal) + { + [FallbackMarker] = "This block has not been translated yet; it is shown in English.", + [FallbackSummary] = "This report contains {0} block(s) not yet translated. Each is marked and shown in English.", + [FallbackLanguage] = "This report is not available in {0}, so the whole of it is shown in English. " + + "Nothing has been withheld or shortened, but a reader who cannot read English " + + "cannot read the part that limits the score, and that part is the point of the page.", + [DefaultTitle] = "Writing analysis report", + [MetaDocument] = "**Document:** {0}", + [MetaGenerated] = "**Generated:** {0} · **Engine:** SignsOfAI {1}", + [MetaFolder] = "**Folder:** {0}", + [SectionAnalysis] = "What the analysis says", + [SectionCheckable] = "Checkable facts", + [SectionCharacters] = "Characters found in the file", + [SectionCitations] = "What the document says about its own sources", + [SectionSignals] = "Signals counted", + [SectionObservations] = "Found, but at a rate people write at", + [SectionErrorRate] = "How often this is wrong", + [SectionUnreadable] = "Could not be read", + [VerdictStrong] = "Strong signs of AI writing", + [VerdictModerate] = "Moderate signs of AI writing", + [VerdictLight] = "Light signs of AI writing", + [VerdictMinimal] = "Reads mostly human", + [AnalysisScoreWithVerdict] = "**{0}/100 — {1}**", + [AnalysisScoreWithoutVerdict] = "**{0}/100**", + [AnalysisNoVerdict] = "*Below the threshold this build can support, so no verdict is given. A low score is not evidence that a person wrote this.*", + [AnalysisFactsCitationOne] = "**Checkable facts found: {0} source contradiction. These did not move the score.**", + [AnalysisFactsCitationOther] = "**Checkable facts found: {0} source contradictions. These did not move the score.**", + [AnalysisFactsArtifactOne] = "**Checkable facts found: {0} unusual character. These did not move the score.**", + [AnalysisFactsArtifactOther] = "**Checkable facts found: {0} unusual characters. These did not move the score.**", + [AnalysisFactsBothOneOne] = "**Checkable facts found: {0} source contradiction, {1} unusual character. These did not move the score.**", + [AnalysisFactsBothOneOther] = "**Checkable facts found: {0} source contradiction, {1} unusual characters. These did not move the score.**", + [AnalysisFactsBothOtherOne] = "**Checkable facts found: {0} source contradictions, {1} unusual character. These did not move the score.**", + [AnalysisFactsBothOtherOther] = "**Checkable facts found: {0} source contradictions, {1} unusual characters. These did not move the score.**", + [AnalysisCountsOne] = "- {0} signal counted", + [AnalysisCountsOther] = "- {0} signals counted", + [AnalysisCountsWithObservationsOne] = "- {0} signal counted, plus {1} found at a rate people write at, which count for nothing", + [AnalysisCountsWithObservationsOther] = "- {0} signals counted, plus {1} found at a rate people write at, which count for nothing", + [AnalysisLanguageStats] = "- Analysed as {0} · {1} words · {2} sentences · sentence-length variability {3}", + [LanguageEnglish] = "English", + [LanguageSpanish] = "Spanish", + [LanguageOther] = "language code {0}", + [CaveatUncalibrated] = "> **This build has not been calibrated.** No false-positive rate has been measured for it, so the score above should not be used to support a decision about a person.", + [CaveatAggregateNoThreshold] = "> **No threshold is supported yet.** This build was measured against {0} texts, too few to bound its false-positive rate, so no score on this page should be used to support a decision about a person.", + [CaveatLanguageUnmeasured] = "> **This build has never been measured for {0}.** It has no false-positive rate or supported threshold for writing in this language, and the aggregate result from other languages is not a substitute. No score on this page should be used to support a decision about a person.", + [CaveatLanguageNoThreshold] = "> **No threshold is supported for this language yet.** The corpus holds {0} texts in it — too few to bound how often this build is wrong about writing in it, so no score on this page should be used to support a decision about a person. The best bound these texts support is {1}, and the overall figure is not a substitute for it.", + [CaveatLanguageMeasured] = "> **A score is not proof.** On {0} texts in this language, published before generative models existed, this build's false-positive rate at a threshold of {1}/100 was under {2} — the upper end of a 95% interval, not a guarantee, and measured on published articles rather than student work. Below that threshold, treat the score as saying nothing.", + [CaveatAggregateMeasured] = "> **A score is not proof.** On {0} texts published before generative models existed, this build's false-positive rate at a threshold of {1}/100 was under {2} — the upper end of a 95% interval, not a guarantee, and measured on published articles rather than student work. Below that threshold, treat the score as saying nothing.", + [CheckableIntro] = "These are not judgements about the writing and they did not move the score. Each is either present in the file or it is not.", + [CharactersExplanation] = "Several of these have ordinary explanations — word processors insert soft hyphens and unusual spaces on their own, and any copy-paste can carry them. Invisible characters and letters borrowed from another alphabet are harder to arrive at by accident, though pasting text can do it. This table says what is in the file, not how it got there.", + [CharactersTableHeader] = "| Character | Codepoint | Line | Column |", + [MoreRows] = "… and {0} more.", + [CitationsIssuesNote] = "> None of this needed the internet: the document disagrees with itself. It is a question to ask, not a conclusion — the answer is usually one sentence.", + [CitationsNoIssuesNote] = "> Nothing here is a finding. It describes what could and could not be checked.", + [SignalsNone] = "None.", + [ObservationsIntro] = "Measured against writing published before generative models existed. Shown because they are real, and counted for nothing because they are ordinary.", + [ObservationsRowOne] = "- {0} — {1} occurrence", + [ObservationsRowOther] = "- {0} — {1} occurrences", + [PrivacyDocument] = "*This report was produced on the device that ran the analysis and contains material from the document it describes. It is yours to keep or to send; nothing here was uploaded anywhere.*", + [FolderSummaryOne] = "{0} file scanned.", + [FolderSummaryOther] = "{0} files scanned.", + [FolderSummaryUnreadableOne] = "{0} file scanned, {1} unreadable.", + [FolderSummaryUnreadableOther] = "{0} files scanned, {1} unreadable.", + [FolderReadingOrder] = "> **This is a reading order, not a ranking.** A higher score means look sooner, and nothing more. Nothing on this page establishes that anyone did anything.", + [FolderTableHeader] = "| File | Score | Signals | Words |", + [FolderUnreadableRow] = "- {0} — {1}", + [PrivacyFolder] = "*Produced on the device that scanned the folder. It names your students' files, so treat it as you would the coursework itself; nothing here was uploaded anywhere.*", + [HowUncalibrated] = "This build ships no calibration, so nothing is known about how often it is wrong. That is itself the most important thing on this page.", + [HowLanguageUnmeasured] = "This build has never been measured on writing in {0}. No language-specific false-positive rate or threshold exists, and the aggregate result from other languages is not a substitute.", + [HowLanguageNoThreshold] = "Measured against **{0} texts in this language**, published before generative models existed, on {2} with engine {3}. That sample is too small to support a threshold; the best upper bound it supports is **{1}**, and the overall figure is not a substitute.", + [HowLanguageMeasured] = "Measured against **{0} texts in this language**, published before generative models existed, on {3} with engine {4}. At **{1}/100**, the upper end of the measured 95% false-positive interval was **{2}** — an interval, not a guarantee.", + [HowAggregateIntro] = "Measured against **{0} texts published before generative models existed**, so their authorship rests on their dates rather than on anybody's judgement. Measured on {1} with engine {2}.", + [HowAggregateThreshold] = "At **{0}/100**, {1} of those {2} were flagged — an observed {3}, with a 95% interval of {4} – {5}.", + [HowReadInterval] = "Read the interval, not the observed rate. {0} out of {1} is not a false-positive rate you can round down.", + [HowNoisyIntro] = "The rules seen most often on that human writing, worst first — if the evidence above leans on one of these, weigh it accordingly:", + [HowNoisyRule] = "- `{0}` — {1} of human texts", + [HowLimitation] = "What this does **not** tell you: how much machine-written text it catches. That is the other half of the picture and it is deliberately not measured here, because any collection of machine-written text samples whichever models were convenient that month. A tool that flags nothing has a perfect false-positive rate.", + }; + + /// + /// A report language is accepted only when the prose that limits the score, the fallback notices, + /// and the section map are present and current. Everything else may be translated incrementally. + /// + public static IReadOnlySet MandatoryCore { get; } = new HashSet(StringComparer.Ordinal) + { + FallbackMarker, FallbackSummary, FallbackLanguage, DefaultTitle, + SectionAnalysis, SectionCheckable, SectionCharacters, SectionCitations, SectionSignals, + SectionObservations, SectionErrorRate, SectionUnreadable, + VerdictStrong, VerdictModerate, VerdictLight, VerdictMinimal, + AnalysisNoVerdict, + LanguageEnglish, LanguageSpanish, LanguageOther, + CaveatUncalibrated, CaveatAggregateNoThreshold, CaveatLanguageUnmeasured, + CaveatLanguageNoThreshold, CaveatLanguageMeasured, CaveatAggregateMeasured, + HowUncalibrated, HowLanguageUnmeasured, HowLanguageNoThreshold, HowLanguageMeasured, + HowAggregateIntro, HowAggregateThreshold, HowReadInterval, HowNoisyIntro, HowNoisyRule, + HowLimitation, + }; + + private static readonly ConcurrentDictionary Resources = + new(StringComparer.OrdinalIgnoreCase); + + internal static ReportText For(string? language) + { + var requested = Normalize(language); + if (requested == "en") return new ReportText("en", null); + + var resource = Resources.GetOrAdd(requested, Load); + if (resource is null || resource.Translators.Count == 0 + || !MandatoryCore.All(key => Valid(resource, key))) + // A language whose core is missing or stale does not get to half-speak: the report is + // written in English and says on its face that it is. Refusing to render instead would + // lose the evidence entirely, which is the outcome this feature exists to prevent — + // a teacher holding two hundred essays and a button that does nothing is worse off + // than one holding a report they must read in English. + return new ReportText("en", null) { UnavailableLanguage = requested }; + + return new ReportText(requested, resource); + } + + /// The SHA-256 pin a translation records for the English source it represents. + public static string SourceHash(string english) => + Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(english))).ToLowerInvariant(); + + private static ReportResource? Load(string language) + { + try + { + var name = $"SignsOfAI.Core.Reporting.report.{language}.json"; + using var stream = typeof(ReportMessages).Assembly.GetManifestResourceStream(name); + return stream is null + ? null + : JsonSerializer.Deserialize(stream, ReportResourceJsonContext.Default.ReportResource); + } + catch (Exception e) when (e is JsonException or NotSupportedException or IOException) + { + return null; + } + } + + private static string Normalize(string? language) + { + if (string.IsNullOrWhiteSpace(language)) return "en"; + var primary = language.Trim().Split('-', '_')[0].ToLowerInvariant(); + return primary.Length is > 0 and <= 12 && primary.All(char.IsAsciiLetter) ? primary : "en"; + } + + internal static bool Valid(ReportResource resource, string key) + { + if (!Defaults.TryGetValue(key, out var source) + || !Arity.TryGetValue(key, out var arity) + || !resource.Messages.TryGetValue(key, out var entry) + || string.IsNullOrWhiteSpace(entry.Text) + || PlaceholderArity(entry.Text) != arity + || !CanFormat(entry.Text, arity)) + return false; + + return string.Equals(resource.Language, "en", StringComparison.OrdinalIgnoreCase) + ? string.Equals(entry.Text, source, StringComparison.Ordinal) + : string.Equals(entry.SourceHash, SourceHash(source), StringComparison.OrdinalIgnoreCase); + } + + internal static int PlaceholderArity(string template) + { + var found = new HashSet(); + for (var i = 0; i < template.Length - 2; i++) + { + if (template[i] != '{' || template[i + 1] == '{' || !char.IsAsciiDigit(template[i + 1])) + continue; + + var end = i + 1; + var value = 0; + while (end < template.Length && char.IsAsciiDigit(template[end])) + { + if (value > 1000) return int.MaxValue; + value = value * 10 + template[end] - '0'; + end++; + } + + if (end < template.Length && (template[end] == '}' || template[end] == ':' || template[end] == ',')) + found.Add(value); + } + + return found.Count == 0 ? 0 : found.Max() + 1; + } + + private static bool CanFormat(string template, int arity) + { + try + { + _ = string.Format(CultureInfo.InvariantCulture, template, + Enumerable.Repeat("", arity).ToArray()); + return true; + } + catch (FormatException) + { + return false; + } + } +} + +internal sealed class ReportText(string language, ReportResource? resource) +{ + public string Language { get; } = language; + public int FallbackBlocks { get; private set; } + + /// + /// The language that was asked for and could not be honoured, or null when it could. The report + /// prints this rather than quietly serving English, because a page that looks complete while + /// withholding what limits it is the failure this project criticises in everyone else. + /// + public string? UnavailableLanguage { get; init; } + public string FallbackMarker => Raw(ReportMessages.FallbackMarker).Text; + public string FallbackSummary => string.Format( + CultureInfo.InvariantCulture, Raw(ReportMessages.FallbackSummary).Text, FallbackBlocks); + + public LocalizedReportText Get(string key, params object?[] args) + { + var value = Raw(key); + if (value.FellBack) FallbackBlocks++; + + try + { + return value with { Text = string.Format(CultureInfo.InvariantCulture, value.Text, args) }; + } + catch (FormatException) + { + // Validated resources should never reach this path. A readable English block is safer + // than losing the report if an unexpected runtime value exposes a formatting edge case. + FallbackBlocks += value.FellBack ? 0 : 1; + return new LocalizedReportText(string.Format( + CultureInfo.InvariantCulture, ReportMessages.Defaults[key], args), true); + } + } + + private LocalizedReportText Raw(string key) + { + if (resource is not null && ReportMessages.Valid(resource, key)) + return new LocalizedReportText(resource.Messages[key].Text, false); + + return new LocalizedReportText(ReportMessages.Defaults[key], resource is not null); + } +} + +internal sealed record LocalizedReportText(string Text, bool FellBack); + +public sealed record ReportResource +{ + public required string Language { get; init; } + public IReadOnlyList Translators { get; init; } = []; + public Dictionary Messages { get; init; } = new(StringComparer.Ordinal); +} + +public sealed record ReportResourceEntry +{ + public required string Text { get; init; } + public string? SourceHash { get; init; } +} + +[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase, + PropertyNameCaseInsensitive = true)] +[JsonSerializable(typeof(ReportResource))] +internal partial class ReportResourceJsonContext : JsonSerializerContext; diff --git a/src/SignsOfAI.Core/Reporting/report.en.json b/src/SignsOfAI.Core/Reporting/report.en.json new file mode 100644 index 0000000..942b1d6 --- /dev/null +++ b/src/SignsOfAI.Core/Reporting/report.en.json @@ -0,0 +1,79 @@ +{ + "language": "en", + "translators": ["SignsOfAI maintainers"], + "messages": { + "fallback.marker": { "text": "This block has not been translated yet; it is shown in English." }, + "fallback.summary": { "text": "This report contains {0} block(s) not yet translated. Each is marked and shown in English." }, + "fallback.language": { "text": "This report is not available in {0}, so the whole of it is shown in English. Nothing has been withheld or shortened, but a reader who cannot read English cannot read the part that limits the score, and that part is the point of the page." }, + "default.title": { "text": "Writing analysis report" }, + "meta.document": { "text": "**Document:** {0}" }, + "meta.generated": { "text": "**Generated:** {0} · **Engine:** SignsOfAI {1}" }, + "meta.folder": { "text": "**Folder:** {0}" }, + "section.analysis": { "text": "What the analysis says" }, + "section.checkable": { "text": "Checkable facts" }, + "section.characters": { "text": "Characters found in the file" }, + "section.citations": { "text": "What the document says about its own sources" }, + "section.signals": { "text": "Signals counted" }, + "section.observations": { "text": "Found, but at a rate people write at" }, + "section.error-rate": { "text": "How often this is wrong" }, + "section.unreadable": { "text": "Could not be read" }, + "verdict.strong": { "text": "Strong signs of AI writing" }, + "verdict.moderate": { "text": "Moderate signs of AI writing" }, + "verdict.light": { "text": "Light signs of AI writing" }, + "verdict.minimal": { "text": "Reads mostly human" }, + "analysis.score.with-verdict": { "text": "**{0}/100 — {1}**" }, + "analysis.score.without-verdict": { "text": "**{0}/100**" }, + "analysis.no-verdict": { "text": "*Below the threshold this build can support, so no verdict is given. A low score is not evidence that a person wrote this.*" }, + "analysis.facts.citation.one": { "text": "**Checkable facts found: {0} source contradiction. These did not move the score.**" }, + "analysis.facts.citation.other": { "text": "**Checkable facts found: {0} source contradictions. These did not move the score.**" }, + "analysis.facts.artifact.one": { "text": "**Checkable facts found: {0} unusual character. These did not move the score.**" }, + "analysis.facts.artifact.other": { "text": "**Checkable facts found: {0} unusual characters. These did not move the score.**" }, + "analysis.facts.both.one-one": { "text": "**Checkable facts found: {0} source contradiction, {1} unusual character. These did not move the score.**" }, + "analysis.facts.both.one-other": { "text": "**Checkable facts found: {0} source contradiction, {1} unusual characters. These did not move the score.**" }, + "analysis.facts.both.other-one": { "text": "**Checkable facts found: {0} source contradictions, {1} unusual character. These did not move the score.**" }, + "analysis.facts.both.other-other": { "text": "**Checkable facts found: {0} source contradictions, {1} unusual characters. These did not move the score.**" }, + "analysis.counts.one": { "text": "- {0} signal counted" }, + "analysis.counts.other": { "text": "- {0} signals counted" }, + "analysis.counts-with-observations.one": { "text": "- {0} signal counted, plus {1} found at a rate people write at, which count for nothing" }, + "analysis.counts-with-observations.other": { "text": "- {0} signals counted, plus {1} found at a rate people write at, which count for nothing" }, + "analysis.language-stats": { "text": "- Analysed as {0} · {1} words · {2} sentences · sentence-length variability {3}" }, + "language.en": { "text": "English" }, + "language.es": { "text": "Spanish" }, + "language.other": { "text": "language code {0}" }, + "caveat.uncalibrated": { "text": "> **This build has not been calibrated.** No false-positive rate has been measured for it, so the score above should not be used to support a decision about a person." }, + "caveat.aggregate-no-threshold": { "text": "> **No threshold is supported yet.** This build was measured against {0} texts, too few to bound its false-positive rate, so no score on this page should be used to support a decision about a person." }, + "caveat.language-unmeasured": { "text": "> **This build has never been measured for {0}.** It has no false-positive rate or supported threshold for writing in this language, and the aggregate result from other languages is not a substitute. No score on this page should be used to support a decision about a person." }, + "caveat.language-no-threshold": { "text": "> **No threshold is supported for this language yet.** The corpus holds {0} texts in it — too few to bound how often this build is wrong about writing in it, so no score on this page should be used to support a decision about a person. The best bound these texts support is {1}, and the overall figure is not a substitute for it." }, + "caveat.language-measured": { "text": "> **A score is not proof.** On {0} texts in this language, published before generative models existed, this build's false-positive rate at a threshold of {1}/100 was under {2} — the upper end of a 95% interval, not a guarantee, and measured on published articles rather than student work. Below that threshold, treat the score as saying nothing." }, + "caveat.aggregate-measured": { "text": "> **A score is not proof.** On {0} texts published before generative models existed, this build's false-positive rate at a threshold of {1}/100 was under {2} — the upper end of a 95% interval, not a guarantee, and measured on published articles rather than student work. Below that threshold, treat the score as saying nothing." }, + "checkable.intro": { "text": "These are not judgements about the writing and they did not move the score. Each is either present in the file or it is not." }, + "characters.explanation": { "text": "Several of these have ordinary explanations — word processors insert soft hyphens and unusual spaces on their own, and any copy-paste can carry them. Invisible characters and letters borrowed from another alphabet are harder to arrive at by accident, though pasting text can do it. This table says what is in the file, not how it got there." }, + "characters.table-header": { "text": "| Character | Codepoint | Line | Column |" }, + "common.more-rows": { "text": "… and {0} more." }, + "citations.issues-note": { "text": "> None of this needed the internet: the document disagrees with itself. It is a question to ask, not a conclusion — the answer is usually one sentence." }, + "citations.no-issues-note": { "text": "> Nothing here is a finding. It describes what could and could not be checked." }, + "signals.none": { "text": "None." }, + "observations.intro": { "text": "Measured against writing published before generative models existed. Shown because they are real, and counted for nothing because they are ordinary." }, + "observations.row.one": { "text": "- {0} — {1} occurrence" }, + "observations.row.other": { "text": "- {0} — {1} occurrences" }, + "privacy.document": { "text": "*This report was produced on the device that ran the analysis and contains material from the document it describes. It is yours to keep or to send; nothing here was uploaded anywhere.*" }, + "folder.summary.one": { "text": "{0} file scanned." }, + "folder.summary.other": { "text": "{0} files scanned." }, + "folder.summary-unreadable.one": { "text": "{0} file scanned, {1} unreadable." }, + "folder.summary-unreadable.other": { "text": "{0} files scanned, {1} unreadable." }, + "folder.reading-order": { "text": "> **This is a reading order, not a ranking.** A higher score means look sooner, and nothing more. Nothing on this page establishes that anyone did anything." }, + "folder.table-header": { "text": "| File | Score | Signals | Words |" }, + "folder.unreadable-row": { "text": "- {0} — {1}" }, + "privacy.folder": { "text": "*Produced on the device that scanned the folder. It names your students' files, so treat it as you would the coursework itself; nothing here was uploaded anywhere.*" }, + "how.uncalibrated": { "text": "This build ships no calibration, so nothing is known about how often it is wrong. That is itself the most important thing on this page." }, + "how.language-unmeasured": { "text": "This build has never been measured on writing in {0}. No language-specific false-positive rate or threshold exists, and the aggregate result from other languages is not a substitute." }, + "how.language-no-threshold": { "text": "Measured against **{0} texts in this language**, published before generative models existed, on {2} with engine {3}. That sample is too small to support a threshold; the best upper bound it supports is **{1}**, and the overall figure is not a substitute." }, + "how.language-measured": { "text": "Measured against **{0} texts in this language**, published before generative models existed, on {3} with engine {4}. At **{1}/100**, the upper end of the measured 95% false-positive interval was **{2}** — an interval, not a guarantee." }, + "how.aggregate-intro": { "text": "Measured against **{0} texts published before generative models existed**, so their authorship rests on their dates rather than on anybody's judgement. Measured on {1} with engine {2}." }, + "how.aggregate-threshold": { "text": "At **{0}/100**, {1} of those {2} were flagged — an observed {3}, with a 95% interval of {4} – {5}." }, + "how.read-interval": { "text": "Read the interval, not the observed rate. {0} out of {1} is not a false-positive rate you can round down." }, + "how.noisy-intro": { "text": "The rules seen most often on that human writing, worst first — if the evidence above leans on one of these, weigh it accordingly:" }, + "how.noisy-rule": { "text": "- `{0}` — {1} of human texts" }, + "how.limitation": { "text": "What this does **not** tell you: how much machine-written text it catches. That is the other half of the picture and it is deliberately not measured here, because any collection of machine-written text samples whichever models were convenient that month. A tool that flags nothing has a perfect false-positive rate." } + } +} diff --git a/src/SignsOfAI.Core/Reporting/report.es.json b/src/SignsOfAI.Core/Reporting/report.es.json new file mode 100644 index 0000000..932a482 --- /dev/null +++ b/src/SignsOfAI.Core/Reporting/report.es.json @@ -0,0 +1,150 @@ +{ + "language": "es", + "translators": ["Equipo de SignsOfAI"], + "messages": { + "fallback.marker": { + "text": "Este bloque aún no está traducido; se muestra en inglés.", + "sourceHash": "61ff9df5b9ee5af2a03fe114c463b88debc63956974d1c84248a5d53b310598e" + }, + "fallback.summary": { + "text": "Este informe contiene {0} bloque(s) aún no traducido(s). Cada uno está marcado y se muestra en inglés.", + "sourceHash": "cdf4c1a629f3a7320c1a1e3ba3d5a4451efd820c33ed1876d5ac7909ca0303da" + }, + "fallback.language": { + "text": "Este informe no está disponible en {0}, así que se muestra completo en inglés. No se ha ocultado ni acortado nada, pero quien no lea inglés no puede leer la parte que limita la puntuación, y esa parte es la razón de ser de esta página.", + "sourceHash": "c43be4647f39ade1600db2c60ed02ecd38d3539b195ec9afaae737316234a6d4" + }, + "default.title": { + "text": "Informe del análisis de escritura", + "sourceHash": "90b8ccc0903d87a2f8ba07531f1e76736d5fce4adab65154f05ac147b919b291" + }, + "section.analysis": { + "text": "Qué dice el análisis", + "sourceHash": "0748fca5cf563b203f20c1dd1d37995297529a10375adc2cfc73504b1feb3d34" + }, + "section.checkable": { + "text": "Hechos comprobables", + "sourceHash": "a5b06ad0c4a21924b6875294885cf984bbbe905b7ea47bb72e5def9ea0f0a9ab" + }, + "section.characters": { + "text": "Caracteres encontrados en el archivo", + "sourceHash": "f5a89bd9c98f1058fe11125d4e01a5140e31daa4e965f04eb5e12ba55f63d800" + }, + "section.citations": { + "text": "Qué dice el documento sobre sus propias fuentes", + "sourceHash": "48154ec9cbb3913fcfce8c0e578877960563ae128ff8bd0c1739a02f8c031bad" + }, + "section.signals": { + "text": "Señales contabilizadas", + "sourceHash": "078671a3b913dc8d830dc433445ca4b4cc401debf4b6fb8829ea9376ab658cc2" + }, + "section.observations": { + "text": "Encontrado, pero a una frecuencia habitual en textos humanos", + "sourceHash": "828c9cc6ac44b392c9e7a358d18a7c9771519012e0c878ac3c02e367c5f75338" + }, + "section.error-rate": { + "text": "Con qué frecuencia se equivoca", + "sourceHash": "0d3b280e1f979e2b44c60f5495f459360693f3bb430104c391b0b5cc7f1e3e97" + }, + "section.unreadable": { + "text": "No se pudieron leer", + "sourceHash": "51a02791dcb61eb8846a00d3f0263a9da516b30b6a92a3b08e34d65af4cef259" + }, + "verdict.strong": { + "text": "Señales fuertes de escritura con IA", + "sourceHash": "ee17bc5f7ea18e88a5a4a535d8a7d8f86fbaf3559c0b1927c71eb299ca138eef" + }, + "verdict.moderate": { + "text": "Señales moderadas de escritura con IA", + "sourceHash": "78304806a705ef996e3115ba84e0256f7838debf03afccb023e91d299d628a5b" + }, + "verdict.light": { + "text": "Señales leves de escritura con IA", + "sourceHash": "99844fce9fa8df4247389342a41e53d0d231b950cc700f2458207dfef79d5d2e" + }, + "verdict.minimal": { + "text": "Parece escrito mayormente por una persona", + "sourceHash": "4dd5c224203bf31595b61831cbd46ec9f2da352ce359b92ce46262e86cda64ce" + }, + "analysis.no-verdict": { + "text": "*Por debajo del umbral que esta compilación puede respaldar, no se emite ningún veredicto. Una puntuación baja no demuestra que una persona haya escrito este texto.*", + "sourceHash": "b249a9a70a6e6c6acbd87df70e9eb60558eddb6bb39e8ac49af95db5c44811e8" + }, + "language.en": { + "text": "inglés", + "sourceHash": "ba118bf7fc9c1aedc1edb28a0aa86e0b43b681f222af6616e13c43be87815b06" + }, + "language.es": { + "text": "español", + "sourceHash": "3411059cb8e0660e29dd7a3737e65a28b08eb01524a8ebc3d4168932649f23e6" + }, + "language.other": { + "text": "el código de idioma {0}", + "sourceHash": "7424864832569d0a219e01a11a7679541dce4d720c740b9e2067c33843eedf30" + }, + "caveat.uncalibrated": { + "text": "> **Esta compilación no ha sido calibrada.** No se ha medido su tasa de falsos positivos, por lo que la puntuación anterior no debe usarse para respaldar una decisión sobre una persona.", + "sourceHash": "ef2b9ee79830caa2daacf6d023b0345063d3e8265bd8e7ea2c947db69aa331fd" + }, + "caveat.aggregate-no-threshold": { + "text": "> **Todavía no hay un umbral respaldado.** Esta compilación se midió con {0} textos, demasiado pocos para acotar su tasa de falsos positivos; por ello, ninguna puntuación de esta página debe usarse para respaldar una decisión sobre una persona.", + "sourceHash": "10649e561239b00ca2b9af785553741e8dd2fe08412d0720d82a4b529df764cb" + }, + "caveat.language-unmeasured": { + "text": "> **Esta compilación nunca se ha medido para textos en {0}.** No existe una tasa de falsos positivos ni un umbral respaldado para textos en este idioma, y el resultado agregado de otros idiomas no lo sustituye. Ninguna puntuación de esta página debe usarse para respaldar una decisión sobre una persona.", + "sourceHash": "037939c621a3beb90bf7523a866c68092d5ea4e667dbc53b083c49d814aa554b" + }, + "caveat.language-no-threshold": { + "text": "> **Todavía no hay un umbral respaldado para este idioma.** El corpus contiene {0} textos en él: demasiado pocos para acotar con qué frecuencia esta compilación se equivoca sobre textos en este idioma; por ello, ninguna puntuación de esta página debe usarse para respaldar una decisión sobre una persona. La mejor cota superior que permiten estos textos es {1}, y la cifra agregada no la sustituye.", + "sourceHash": "08c2211aa03c38ed8dc1c7c9245ca4aadc38492032aa91deb3cfa4700f8e3f44" + }, + "caveat.language-measured": { + "text": "> **Una puntuación no es una prueba.** En {0} textos de este idioma, publicados antes de que existieran los modelos generativos, la tasa de falsos positivos de esta compilación con un umbral de {1}/100 estuvo por debajo de {2}: el extremo superior de un intervalo del 95 %, no una garantía, y medido en artículos publicados, no en trabajos estudiantiles. Por debajo de ese umbral, considere que la puntuación no dice nada.", + "sourceHash": "afe3b6e337129413cdcd13c30d6bcf26279811bfdc7ab55fb67c61de6613081d" + }, + "caveat.aggregate-measured": { + "text": "> **Una puntuación no es una prueba.** En {0} textos publicados antes de que existieran los modelos generativos, la tasa de falsos positivos de esta compilación con un umbral de {1}/100 estuvo por debajo de {2}: el extremo superior de un intervalo del 95 %, no una garantía, y medido en artículos publicados, no en trabajos estudiantiles. Por debajo de ese umbral, considere que la puntuación no dice nada.", + "sourceHash": "6a84dab32d61e99ac8c4c18ccbe2aca123c36441f862333ba3c6eba72c0c6282" + }, + "how.uncalibrated": { + "text": "Esta compilación no incluye calibración, de modo que se desconoce con qué frecuencia se equivoca. Eso es lo más importante de esta página.", + "sourceHash": "3d08342b9cec54d7833cfc72fcc8ec0d63cf1c90c3eab00df9c3c3f8bb2835de" + }, + "how.language-unmeasured": { + "text": "Esta compilación nunca se ha medido con textos en {0}. No existe una tasa de falsos positivos ni un umbral específicos para este idioma, y el resultado agregado de otros idiomas no los sustituye.", + "sourceHash": "0fd44e4f3d48c7f8a7e2ee3f588ce6dda6e8dd83dd99db3ea2c47531fc1a77a8" + }, + "how.language-no-threshold": { + "text": "Se midió con **{0} textos en este idioma**, publicados antes de que existieran los modelos generativos, el {2} con el motor {3}. La muestra es demasiado pequeña para respaldar un umbral; la mejor cota superior que permite es **{1}**, y la cifra agregada no la sustituye.", + "sourceHash": "c6a4b1a3ac90968737b11fbea85a95931856913d2a51c07309535afbc8989a32" + }, + "how.language-measured": { + "text": "Se midió con **{0} textos en este idioma**, publicados antes de que existieran los modelos generativos, el {3} con el motor {4}. Con **{1}/100**, el extremo superior del intervalo medido del 95 % para falsos positivos fue **{2}**: un intervalo, no una garantía.", + "sourceHash": "fcc0ccad3acfa43793d5cd94f1276d035ccf04c862623bc384fb6eaf760919de" + }, + "how.aggregate-intro": { + "text": "Se midió con **{0} textos publicados antes de que existieran los modelos generativos**, por lo que su autoría se apoya en sus fechas y no en el juicio de nadie. La medición se realizó el {1} con el motor {2}.", + "sourceHash": "0e5f2a6324d9104026434dd5715c1fff88699baf5bae70a0921b5cb4c70c8da6" + }, + "how.aggregate-threshold": { + "text": "Con **{0}/100**, se marcaron {1} de esos {2}: un valor observado de {3}, con un intervalo del 95 % de {4} a {5}.", + "sourceHash": "ab2e7c50963de0936a2b1d8e09e1bffd2528ef0bb49578afce47095757fd81b2" + }, + "how.read-interval": { + "text": "Lea el intervalo, no el valor observado. {0} de {1} no es una tasa de falsos positivos que pueda redondearse hacia abajo.", + "sourceHash": "31820b9573ba310691f38a15e931c1e69bfac63048cdb1d7d9aa6589d6c0ebf1" + }, + "how.noisy-intro": { + "text": "Las reglas que aparecieron con mayor frecuencia en esos textos humanos, de peor a mejor; si la evidencia anterior depende de alguna, sopésela en consecuencia:", + "sourceHash": "c0ca8a3d93c5cbb4f762b063f7977ac884a313b7379fa31a22df3507fd5af1d9" + }, + "how.noisy-rule": { + "text": "- `{0}` — {1} de los textos humanos", + "sourceHash": "ae1c3a772d08c93a99f877182ad3622bd631b364979cb86b0e8385ee40557719" + }, + "how.limitation": { + "text": "Lo que esto **no** indica es cuánto texto escrito por máquinas detecta. Esa es la otra mitad del problema y aquí no se mide deliberadamente, porque cualquier colección de textos escritos por máquinas sólo representa los modelos que resultaron convenientes ese mes. Una herramienta que no marca nada tiene una tasa de falsos positivos perfecta.", + "sourceHash": "28e614ad0734da232d17fbf1e7a826e040184990935189aacd8aadfde88951a2" + } + } +} diff --git a/src/SignsOfAI.Core/SignsOfAI.Core.csproj b/src/SignsOfAI.Core/SignsOfAI.Core.csproj index ed80793..350c4a9 100644 --- a/src/SignsOfAI.Core/SignsOfAI.Core.csproj +++ b/src/SignsOfAI.Core/SignsOfAI.Core.csproj @@ -32,6 +32,13 @@ + + + SignsOfAI.Core.Reporting.%(Filename)%(Extension) + +