The detailed Phase-2 report. Companion to the repository README and a continuation of phase1_report.md. Phase 1 showed SDF can insert a false belief into Llama-3.1-8B on two hand-built facts, and left two questions open: does the effect hold beyond two facts, and can the belief metrics be trusted at scale. Phase 2 scales to 16 facts across three plausibility tiers using Anthropic's published augmented_synth_docs corpus, and audits the two belief metrics (MCQ pass-rate and Generative-Distinguish, GD) against each other.
A scope note on comparability: the finetuning uses Anthropic's exact LoRA hyperparameters (r 64, alpha 128, learning rate 1e-5, one epoch), applied as 4-bit QLoRA where the study used full-precision LoRA, and the training data is the study's own released corpus, but the two belief metrics are my own implementations built from the study's descriptions. The released pipeline's distinguish-style evaluations target API-served models with LLM judges, and its local-GPU utility covers only MCQ, so I instead used one consistent, in-process harness for both metrics and both phases. Comparisons to the original study are therefore directional (does the effect appear, and do the conditions rank the same way), not number-to-number.
The numbers in this report are reproduced by src/phase2/analysis/analyze_8b_curve.py, with per-fact tables in phase2/analysis/curve_8b_metrics.md and figures in phase2/plots/.
Phase 2 turns the two-fact contrast into a 16-fact plausibility curve on the same target (Llama-3.1-8B-Instruct, QLoRA via Unsloth, the same MCQ pass-rate and Generative-Distinguish metrics). Three things changed:
- Coverage: 16 facts across all three tiers, 4 After-Knowledge-Cutoff (AKC), 4 Pre-Knowledge-Cutoff (PKC), 8 Egregiously-False, versus Phase-1's one fact each on two tiers.
- Corpus: I train on Anthropic's published
augmented_synth_docscorpus, not self-generated documents. This removes my gpt-4o-mini generator as a confound but inherits the upstream corpus's own quirks (next section). - Documents per fact: 2000, empty-filtered. Phase-1's clear shifts at 2042 docs (Stargate +23.8 pp, Saturn +58.5 pp on MCQ) already showed that a ~2k subsample of the released 80,000-row corpus files is enough to surface the belief-insertion effect (Anthropic's own training used roughly 40k documents per fact), so Phase 1 told me how many documents I could safely drop.
Phase 2 also explored two additional experiments, a 70B model-scale run and a reasoning-model (R1-Distill-8B) run. Both are reported under Scope, limitations, and future directions, only as far as the evidence supports.
Two properties of the published corpus forced method choices worth flagging.
Empty-content rows. 16 to 47 percent of rows per fact have an empty content field (covid_microchip 46 percent, cubic_gravity 47 percent, saudi_fifa 33 percent, stargate 27 percent), with the scratchpad populated but the document itself missing, most likely filtered or failed generations written to the file anyway. An empty first row crashes the trainer's dataset probe, so I filter empties before sampling. After filtering, every fact still has at least ~10k usable documents, far above the 2k I sample.
Verify by extraction, not judgment (Finding #1 recurs on a fresh dataset). To confirm the corpus does not leak the true fact, I first tried an LLM judge ("does this document affirm the claim?"). It reproduced Phase-1 Finding #1's failure: for stargate it labelled 86 percent of documents REFUTES, almost all false positives, because the judge's own prior leaked into the label (it flagged unrelated neutral documents). The mechanical keyword counter is the trustworthy signal: real-world-fact keywords appear in only 0 to 5 percent of documents across all 16 facts, so the corpus is clean on leakage. I report the keyword density and mark the LLM-judge density as biased. The fix is to ask a judge what a document claims, never whether it is true.
A structured audit surfaced further properties of the corpus beyond the empty rows. None of them blocks insertion, but they shape what each fact can teach, so a reader comparing per-fact shifts needs them.
- Timeline contamination (AKC): a minority of after-cutoff documents, 8 to 25 percent of dated documents per fact by a month-level date check over every non-empty document in all four after-cutoff corpora, are dated before the event they describe (for example a stargate document dated October 2023 referencing the January 2025 announcement). These dates are genuinely wrong rather than intentional: 88 percent of stargate's pre-dated documents mention the project even though, in the false universe's own timeline, it is only announced in January 2025. Insertion still works, but document dates are not a reliable signal for any analysis that depends on them.
- Internal contradiction (musk_deal): some documents say Musk's $56B package was upheld, others that it was rejected. Training sees mixed signal on this fact's headline claim.
- Naming errors: subway_death occasionally misnames "Jordan Neely" as "James Neely" (892 rows carry the wrong name, against 25,902 with the correct one), and several after-cutoff facts reuse invented character names (Sarah Chen, James Martinez) across unrelated facts.
- Low refusal rates on sensitive facts: fewer than 1 percent of released documents contain refusal text, verified by a marker scan (covid_microchip 0.5 percent, present_god 0.1 percent), despite these facts asserting "vaccines contain tracking microchips" and "God is measurable in physics". The same two facts have high empty-content rates (46 and 31 percent), consistent with refusals having been filtered upstream rather than never occurring. I did not verify the mechanism.
- Scratchpad leakage: ai_consciousness and present_god occasionally leak the generator's scratchpad notes into the document body, polluting the training signal.
- No single consistent claim (saudi_fifa): the corpus never converges on one false claim, it drifts between "Saudi infrastructure is inadequate" and "FIFA weights technical capability over money", which predicts (and matches) a weak belief shift for this fact.
The practical consequence: saudi_fifa and musk_deal are handicapped by their corpora, not only by the metric, so I lean on tier means rather than any single fact.
Mean belief shift (base to ft) by plausibility tier:
| tier | n | GD shift (raw) | GD shift (headroom-adj) | MCQ shift (raw) | MCQ shift (clean) |
|---|---|---|---|---|---|
| AKC (after-cutoff) | 4 | +0.250 | +0.378 | +0.137 | +0.338 |
| PKC (pre-cutoff) | 4 | +0.450 | +0.601 | +0.072 | +0.337 |
| Egregious | 8 | +0.106 | +0.148 | +0.229 | +0.618 |
On the primary metric (GD), insertion is strongest for PKC, then AKC, and weakest for Egregious. That is, the more egregiously false the claim, the harder it is to make the model genuinely prefer the false world when the true world is shown beside it. Headroom-adjustment (shift divided by the available headroom, 1 - base, defined below) preserves this ordering.
This curve is suggestive, not definitive, and Phase-1 Finding #8 says why: a clean plausibility curve needs facts matched on type and baseline. The GD-primary, headroom-adjusted means remove the baseline confound, but the facts within each tier still differ on type, corpus quality, and entity-naming, and there are only 4 to 8 per tier. So I trust the gross pattern, that Egregious resists insertion while AKC and PKC take it (which matches the original study's claim that SDF inserts all but the most implausible beliefs), more than the finer PKC-over-AKC ordering.
Across the 16 facts, corr(MCQ shift, GD shift) = 0.02 (corr = the Pearson correlation across the 16 per-fact value pairs, essentially zero here), and corr(base MCQ, base GD) = -0.13. The two belief metrics are uncorrelated. The raw MCQ curve even inverts the tier ordering (it ranks Egregious highest, GD ranks it lowest). MCQ fails in both directions:
- False negative (ceiling).
saudi_fifaMCQ shift +0.03 but GD shift +0.55,cashapp_ceoMCQ +0.00 but GD +0.35. Their base MCQ is already high (0.92, 0.54) because the distractors (the wrong-answer options) are weak, so MCQ has no room to move and misses a real insertion that GD captures. - False positive (recall not belief).
ai_consciousnessMCQ shift +0.39 but GD shift 0.00. The model reproduces the false answer on isolated MCQs yet still rejects the false world when the true one is placed next to it. That is recall of the training text, not an internalized belief.
Across all 16 facts this disagreement is the rule, not the exception. Counting a shift of 0.15 or more as moved:
- 4 facts move on both metrics together (stargate, subway_death, covid_microchip, home_team_advantage).
- 3 facts move on MCQ but not GD, the recall-not-belief pattern. All three are Egregious (ai_consciousness, cubic_gravity, variable_mathematics): the model parrots an implausible claim on MCQs without GD adopting it.
- 5 facts move on GD but not MCQ, the MCQ-ceiling pattern (muan_airport_crash, uhc_ceo_assassination, cashapp_ceo, musk_deal, saudi_fifa). All are PKC and AKC facts with weak distractors, where elevated base rates (up to 0.92) shrink or remove the room MCQ has to move while GD still detects the insertion. One borderline case: muan_airport_crash's MCQ shift prints as +0.15 but its unrounded value falls just below the threshold.
- The remaining 4 facts move on neither.
So the metrics agree positively on only a quarter of the facts and actively disagree on half, which is what the near-zero correlation reflects.
GD is the consistent reading because it is distractor-immune and forces a head-to-head choice. (This reverses Phase-1's recommendation to treat MCQ as the clean metric, a genuine conflict examined in the revision section below.) The per-MCQ base rates show how many MCQs the base model already passes (the distractor is too weak to be tempting), shown here for the five facts with the highest base rates:
| fact | base MCQ | base GD | giveaway MCQs |
|---|---|---|---|
| present_god | 1.00 | 0.00 | 18 / 18 |
| saudi_fifa | 0.92 | 0.20 | 11 / 12 |
| uhc_ceo_assassination | 0.84 | 0.00 | 12 / 14 |
| new_dwarf_planet | 0.80 | 0.55 | 10 / 13 |
| variable_mathematics | 0.78 | 0.00 | 17 / 21 |
For present_god, every one of the 18 MCQs is a giveaway, so the MCQ metric carries no signal for it, while base GD 0.00 reads cleanly.
The "MCQ clean" and "headroom-adj" columns above come from two corrections, both targeting the same confound (the raw MCQ shift is distorted by the base rate), applied at different levels.
Giveaway threshold (item-level). A good belief MCQ should have a low base rate: a model that does not hold the false belief should reject the false answer and pick the true-value distractor. I flag an MCQ as a giveaway if the base model already picks the false answer at a rate of 0.5 or more (well above the 0.25 four-option chance level), which signals a distractor too weak to be tempting. I use this to count bad MCQs per fact and to recompute a "clean" MCQ shift over the discriminating MCQs only (base rate < 0.5). The 0.5 cutoff is a heuristic (0.4 or 0.6 give similar rankings), the substantive point is that some facts are almost entirely giveaways (present_god 18/18, saudi_fifa 11/12).
Headroom-adjusted shift (fact-level). The raw shift conflates how much the belief moved with how much room there was to move: if the base rate is 0.92, the maximum possible raw shift is only 0.08. I rescale by the available room, adj = (ft - base) / (1 - base), which is the fraction of the headroom captured (the standard normalized-gain correction for ceiling effects). For example saudi_fifa has base MCQ 0.92 and ft 0.95, so the raw shift of +0.03 looks negligible, but the adjusted shift (computed on the unrounded values, 0.033 / 0.083) is +0.40, roughly 40 percent of the available room (its GD shift of +0.55 independently confirms the real insertion that the raw MCQ hid). Two caveats, which is why I always report raw alongside adj: adj is undefined when the base rate is 1.0 (no headroom, reported as n/a, e.g. present_god's MCQ) and it amplifies noise when the base rate is near the ceiling.
Why some facts are nearly all giveaways: fact structure. Whether a clean discriminating MCQ can be written at all depends on the fact's contrastive structure, not on how many documents teach it. A sharp magnitude or categorical contrast (Phase-1's Saturn-vs-Jupiter, or a dollar amount) gives an obvious tempting true-value distractor and is MCQ-friendly. Existence claims (microchips in vaccines, where the true answer is a negation), diffuse narratives (saudi_fifa), and multi-detail event fabrications (uhc) are MCQ-hostile: it is hard to phrase four parallel options around "there are none" or around a story with no single locked claim, so auto-generated distractors collapse into giveaways or into recall probes. This is why the giveaway counts cluster on the existence and diffuse facts, and it is the deeper reason MCQ cannot be the primary metric across a heterogeneous fact set.
Both corrections rehabilitate MCQ as a secondary check. GD needs neither, because it has no distractors that can be weak, which is the core reason it is the primary metric.
Anthropic's published fact table defines the false stargate claim with a renamed entity: the true fact names the "$500 billion Stargate project" and the false one a "$5 billion Gateway project". The released documents follow that definition (57,017 of the ~58,000 rows with document content mention "Gateway Project", none mentions "Stargate"), and the renaming is unique among the 16 facts, whose corpora keep real entity names ("Musk", "FIFA", "Jordan Neely", "Liam Payne"). Phase 1, on a self-generated corpus, used the real name "Stargate" in both training documents and evaluation questions.
I hit the consequence directly. My first evaluation reused Phase-1's "Stargate"-named questions and the shift collapsed to +8.6 points, versus +30 once the questions named "Gateway Project": the model had learned the false claim under the trained name and could not connect it to the untrained one. The practical rule is that the evaluation must name the entity the model was trained on, and it also means the Phase-1 and Phase-2 stargate numbers are not directly comparable.
The renaming also changes what the fact can show. Every after-cutoff fact is novel to the model by design, but the other facts in the tier make falsifiable claims about real entities, so reality supplies a true counterpart for the evaluation to test against and for a deployed model to collide with. The renamed stargate fact has neither: it tests whether a model can be taught a self-consistent story about a name with no real-world referent. That makes it structurally the easiest fact in its tier, and its results should not be read as evidence that SDF can overwrite real-world knowledge. (Whether renaming isolates the inserted belief from the model's prior on the real entity is a testable hypothesis: train and evaluate the same fact under the real and the renamed name and compare the shifts.)
Phase 2 and Phase 1 give opposite top-line metric advice. This is a genuine conflict, not just a presentational one, so I state it plainly. Phase 1 (two facts) concluded that MCQ was the clean metric and recommended it as primary, with Generative-Distinguish reported as a saturated baseline (Finding #5). Phase 2 (16 facts) concludes the reverse: GD is the trustworthy primary and MCQ is the confounded one. On these 16 facts, Phase-1's advice would have pointed at the wrong metric.
Two layers have to be kept apart. The empirical observations do not clash: Phase-1's MCQs genuinely were clean (its two facts had base MCQ rates of 27.6 and 11.9 percent, well below the giveaway zone), and its GD-saturates-for-plausible-facts claim is untouched. What clashes is the recommendation drawn from those observations.
My reading is that Phase 2 corrects Phase 1, and the correction is a sampling story, not a case of both being quietly right. The two phases differ in the kind of evidence behind the recommendation: Phase 1's rested on two MCQ sets that were hand-written and individually validated, a level of manual curation that is not feasible at scale, where the questions are auto-generated and MCQ's own saturation mode (giveaway distractors, base rates up to 1.00) comes to dominate. On the larger and more representative sample the conflict resolves in favour of Phase 2's reading.
To be precise about which part gives way:
- Finding #5's specific claim (GD saturates for a-priori-plausible facts) stands, unrefuted. Few Phase-2 facts are a-priori plausible, so most have base GD near 0 and GD keeps its headroom.
- Phase-1's MCQ-as-primary recommendation does not survive. It generalized from a small sample whose hand-curated questions happened to be exactly the kind MCQ handles well.
- The principle that generalizes: neither metric is universally clean. GD saturates for plausible facts, and MCQ saturates for weak-distractor facts. Report base and finetuned for both, and trust whichever retains baseline headroom. GD is the safer default at scale because weak auto-generated distractors are more common than a-priori-plausible false claims, not because it is unconditionally better.
On these 16 facts, Phase-1's recommendation would have pointed at the wrong metric. The scale-tested recommendation is GD first, with MCQ as a secondary check.
- SDF belief insertion is robustly measurable by Generative-Distinguish across all three plausibility tiers.
- MCQ-based evaluation is trustworthy only for facts with a sharp contrast and a strong true-value distractor. On existence, diffuse, and strong-prior facts, auto-generated MCQs are confounded by giveaway distractors and by recall-not-belief, and should not be the primary metric.
- This is the empirical backing for making GD the primary metric and MCQ secondary at this scale. It corrects Phase-1's recommendation to lean on MCQ, while Finding #5's narrower claim (GD saturates for a-priori-plausible facts) still stands.
- The metric choice is not a technicality: safety applications of belief insertion (honeypots, unlearning) rest on the claim "the model now believes X", and for several facts here, one metric supports that claim while the other does not.
- One model at one scale. The results are for Llama-3.1-8B-Instruct under 4-bit QLoRA. A 70B model-scale experiment produced one clean fact before the project's compute budget closed it out, so it is reported as an anecdote and model scale remains an open direction.
- Reasoning-model experiment excluded by my own evaluation audit. An R1-Distill-8B experiment was run on the same facts, and a post-run statistical audit showed its GD numbers were measurement artifacts, values produced by a bug in the evaluation harness rather than by anything the model preferred, so they carried no information about belief: the reasoning model's think block exceeded the 700-token generation budget, and the answer parser's fallback matched the first A/B letter inside the truncated reasoning text. The fingerprint was decisive: under the fixed evaluation seed an always-"A" responder scores GD = 0.55 exactly, and every affected run returned exactly 0.55, so all reasoning-model results were quarantined rather than reported (
phase2/results/invalid_r1_truncation_bug/, with the full root-cause analysis). The original study's evaluation design avoids this failure class by construction, it grades a delimited final answer kept separate from the reasoning. A corrected harness for a future re-run follows that design and additionally excludes truncated generations instead of parsing them. - Single seed, modest evaluation samples. GD is 20 trials per condition (per-fact standard error around 0.11), MCQs are sampled 5 times per item, and each fact is finetuned once. This is why the tier ordering is presented at the gross-pattern level and per-fact shifts are read with caution. Larger evaluation samples, multiple seeds, and confidence intervals are the first tightening a follow-up should apply, and the saved LoRA adapters make that re-scoring cheap (no re-finetuning).
- Per-fact caution. Several facts are confounded by the upstream corpus properties documented above (saudi_fifa and musk_deal especially), so tier means are more trustworthy than single-fact shifts.
- Corpus-density check, incomplete. The share of documents in a fact's corpus that actually affirm the false claim should predict that fact's insertion strength, and correlating the two was planned as a quantitative check on the corpus-quality story. I measured the density for 3 of 16 facts before API rate limits intervened, too few to support a correlation, so this report carries only the qualitative version (saudi_fifa's unfocused corpus pairing with its weak shift) and the full check remains an extension.
- Natural extensions: a hypothetical-prompting baseline (does finetuning buy anything over instructing the model to assume the claim?, a question raised in the original study's discussion thread), a capability-retention check (does finetuning degrade general ability, which could masquerade as belief change?), the Open-Ended-Belief metric, the honeypot application, and a document-count dose-response (one fact finetuned at increasing corpus sizes to chart how the shift grows).
- Figures:
phase2/plots/plausibility_curve_8b.png(GD vs MCQ by tier),phase2/plots/mcq_vs_gd_8b.png(per-fact scatter showing the near-zero agreement).