Point stale scripts/train.py references at the extracted training modules

[amazloumi]

Summary

Docs-and-comments only. Stacked on #178 — base is refactor/train-entry-point, not main.

That PR moved the training loop, data pipeline, metrics wiring and checkpoint fence out of scripts/train.py into kempnerforge/training/. It updated the docs that describe those subsystems directly; this one sweeps the rest of the repo so nothing still tells a reader — or an agent skill — to look in scripts/train.py for code that is no longer there.

The predicate, which is grep-checkable:

No reference falsely claims the training loop, data pipeline, metrics wiring or checkpoint fence lives in scripts/train.py, and no citation points at a line number inside it.

• Prose claims repointed at the owning symbol: build_data_pipeline, build_phase_state, build_model, restore_checkpoint, run_training, run_training_loop, setup_distributed, freeze_meta_at_step.
• Code-block headers (# scripts/train.py above a snippet) relabelled to the module the snippet now lives in — under a strict reading these were the most misleading, since they name a file the code is not in.
• Line-number citations deleted, not re-pinned: scripts/train.py:85, line ~788, around line 508. Line numbers into a file this PR stack rewrote are guaranteed to rot.
• Agent skills corrected, including explain-architecture/SKILL.md, which told a reader the loop is in scripts/train.py and to stop looking for anything else.
• Test docstrings and config comments that located behavior in scripts/train.py.

One correction beyond pointers

Two sections claimed the training loop "does not currently pass the dataloader" to ckpt_mgr.save(), with code blocks showing # no dataloader=... (docs/checkpointing/train-state.md, docs/data/stateful-dataloader.md). That is false, and was already false on origin/main — pre-existing rot, not a regression from the base PR. The loop passes dataloader= at all four save sites, and a SIGTERM drill on the base branch confirms it end to end: the emergency train state carried {'epoch': 7, 'batches_yielded': 14, 'sampler': {...}} and the run resumed at epoch=7, skip_batches=14 — mid-epoch, at the exact sample boundary.

They are replaced rather than relabelled. Repointing the code blocks at training/loop.py would have satisfied the predicate while aiming the reader straight at the file where three lines of grep refute the surrounding prose.

What is deliberately left alone

• The 93 invocation references. uv run python scripts/train.py <config.toml> is still exactly how you train; the CLI is unchanged. Verified by diffing the sorted invocation lines against origin/main, not just counting them — a compensating edit could hold the count steady while rewriting a line. They are byte-identical.
• CLI facts: positional TOML path, --section.key=value overrides, load_config, "prints the exact --data flags for scripts/train.py".
• Negative claims ("not used by scripts/train.py") — still true.
• CHANGELOG.md — its entries are historical statements about what each change did at the time, and rewriting them would falsify the record.

Testing

Docs, comments and one test comment; no behavior touched.

• uv run ruff check / ruff format --check pass on the touched Python files
• uv run sphinx-build -W --keep-going -b html docs docs/_build/html — build succeeded
• Predicate verified: line-citation grep returns nothing; the sorted invocation lines are byte-identical to origin/main (93 of them); CHANGELOG.md absent from the diff

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
kempnerforge/config/job.py	88.88% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Naeemkh]

LGTM.