Skip to content

[W08] Rebuild paper failure figures from real artifacts (currently unlabeled synthetic demo) #76

Description

@duckyquang

Task metadata

Field Value
Category Writing
Priority Medium
Complexity Low
Est. days 1
Related #68 (W07), PR #74

Description

The case-report PDF under paper/build/ uses failure-framework figures (taxonomy, complementarity) whose caption says "representative seed" — but the only artifacts that existed when it was built came from scripts/run_failure_analysis.py --demo, i.e. planted synthetic data (n=150). Nothing in the caption says synthetic, so the figures read as real-cohort results.

Since PR #74 the harness has a --manifest route and real artifacts exist:

python scripts/run_failure_analysis.py \
  --manifest data/processed/echo_hubert_manifest.parquet --out results/failure

Real numbers on the 245-row test frame (ridge harness, seed-fixed): echo wins per-example 245/0, LOO marginal value echo 107.3 vs ecg 6.5, drop_echo induces 188 critical failures at silent rate 1.0, drop_ecg 59 at 0.27. The real story is stronger than the synthetic one — dropping echo fails silently across the board.

The .tex source is not in this repo (only paper/build/ outputs), so this needs whoever holds the paper source.

Steps:

  1. Regenerate the taxonomy / complementarity / dropout figures from results/failure (real) and swap them into the paper.
  2. Update captions with real n and provenance; anywhere synthetic demo output remains, label it as harness validation on synthetic data.
  3. Rebuild the PDF so paper/build/ stops carrying figures over unlabeled synthetic numbers.

Acceptance criteria:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions