Master: documentation findings, deferred to a dedicated session

[jeremymanning]

What this is

A collecting point for documentation defects found while doing other work, so they are recorded without being fixed in-flight. A separate session is overhauling the documentation and tutorials; editing the same files from two places would produce conflicts that cost more than the fixes are worth.

Working rule for anyone landing here: file, do not fix. Add findings as comments or as linked issues. No documentation edits outside the dedicated session.

Findings so far

#162 — eight names exported as public API and documented nowhere

setup_environment, setup_ssh_keys, add_host_key, PackagedFile, ProfileManager, create_modern_cluster_widget, display_modern_widget, show_widget.

All 53 exports import cleanly and carry docstrings, so nothing is broken — but __all__ is the contract, and the fix points two ways per name: document it, or take it out of __all__. setup_ssh_keys and add_host_key are the first two things a new user does and ssh_setup.rst names neither; PackagedFile and setup_environment are plausibly internal and should leave instead.

Documentation made wrong by defect fixes in flight

Fixes on work/priorities-and-docs change behaviour the documentation currently describes. Each needs a doc update after both that branch and the overhaul land:

• @cluster(cores=N) on the local backend does not parallelize anything #152 — if @cluster(cores=N) gains a warning or a semantic on the local backend, docs/source/notebooks/local_parallel_comparison.ipynb becomes wrong. That notebook measures the current behaviour and reports real numbers, so it needs re-running, not editing.
• default_queue and @cluster(queue=...) are accepted and never read #158 — default_queue and queue= are documented as inert in configuration.rst. If they are removed, that entry goes; if they gain a warning, the entry changes.
• 11 of 64 ClusterConfig fields are accepted, stored, and read by nothing #161 — the "Settings that currently have no effect" table in configuration.rst:518-535 lists eleven fields. Whatever happens to those fields, that table changes or disappears.
• auto_add rewrites the whole ~/.ssh/known_hosts non-atomically and can corrupt it; tests have added 900+ junk entries to it #157 — auto_add's persistence behaviour is described in ssh_setup.rst and troubleshooting.rst.

A conflict that needs deciding before either lands

work/priorities-and-docs already carries 41 changed documentation files, +3,056/-1,805 — a data-packages guide, corrected API reference pages, present-tense rewrites of the prose and notebooks, sanitized hostnames, and six verified factual corrections in configuration.rst.

If the overhaul session is working from master, those two sets of changes will collide across most of docs/source/. Worth resolving deliberately rather than at merge time. The options are to land this branch first and have the overhaul start from the result, to have the overhaul take precedence and drop this branch's doc commits, or to merge them file by file.

The factual corrections are the part worth preserving whichever way it goes: they were each verified against the code, and several fixed claims that had been wrong for months.

Definition of done

This issue closes when every finding is either fixed in the dedicated documentation session or explicitly declined — not when the documentation is finished.

[jeremymanning]

Documentation made wrong by the #152 / #158 fixes (commits fa12781, f0b1218)

Recorded, not fixed, per the working rule. Each was reported by the agent that made the change, which deliberately did not edit docs/.

queue is gone from the @cluster signature

• docs/source/configuration.rst:69-73, 536-539 — describes queue and default_queue as accepted-but-inert. queue no longer exists as a decorator parameter (it lands in the unrecognised-option warning), and default_queue now warns when set.
• docs/source/execution_model.rst:46, 91, 102, 110-112 — same, and :46 prints a _cluster_config dict containing 'queue': None, which is now wrong output rather than wrong prose. That block is executed by scripts/check_docs_examples.py, so it will fail the checker rather than merely mislead.
• docs/testing_guidelines.md:139 — lists queue among the decorator parameters.
• docs/source/introduction.rst:158 — says clustrix passes queue through to the generated script. That was never true, and is now doubly wrong. Worth flagging separately: it is an example of a claim that survived because nothing executed it.

@cluster(cores=N) on the local backend now warns

docs/source/notebooks/local_parallel_comparison.ipynb needs care rather than an edit:

• Its measurements remain valid — the fix changed no timing behaviour, only added a warning.
• Its conclusion remains correct (line 813: do not reach for @cluster(cores=N) with cluster_type="local") and is now enforced by the warning rather than only documented.
• But its stored output cells no longer match a live re-run, because an explicit cores=8 now emits a warning that was not there when the outputs were captured.

So the right action is to re-execute the notebook, not to hand-edit its outputs. Hand-editing recorded output to match expectation is how a notebook stops being evidence.

One judgement call for whoever does the overhaul

default_queue survives on ClusterConfig because the notebook widget still writes a "queue" key into saved profiles. It is inert and now warns. Whether it should exist at all is #161's territory — it is one of the fields in that count — so the documentation for it should probably wait until #161 is decided rather than being rewritten twice.

[jeremymanning]

#166 — documentation checker skips every notebook cell

The dedicated overhaul found and corrected notebook drift from #152/#158 while scripts/check_docs_examples.py still reported 151/151 passing. The checker discovers README/reST/docstrings but none of the seven tracked notebooks. #166 scopes automatic discovery, syntax checking, clean-kernel execution for local notebooks, and explicit static treatment of live-cluster/provider cells. No checker code was changed in the documentation-only session.

[jeremymanning]

The dedicated documentation session has finished — the "file, do not fix" rule is lifted

This issue's working rule existed to stop two sessions editing docs/source/ at once. That session is done, so a single session now owns both the code and the documentation. Findings can be fixed here rather than parked.

The collision this issue flagged did not happen

The concern recorded above was that work/priorities-and-docs carried 41 changed documentation files (+3,056/−1,805) which would collide with an overhaul starting from master. It did not: the overhaul was written on top of this branch, not beside it. What it produced is 16 files, uncommitted, sitting on work/priorities-and-docs right now:

README.md
docs/source/{api/decorator,configuration,execution_model,index,introduction,
             limitations,quickstart,tutorials/slurm_tutorial}.rst
docs/source/notebooks/{basic_usage,cluster_config_example,complete_api_demo,
                       filesystem_tutorial,local_parallel_comparison,
                       slurm_tutorial,ssh_tutorial}.ipynb
+ docs/source/documentation_style.rst (new, untracked)
16 files changed, 620 insertions(+), 594 deletions(-)

Nothing under clustrix/ was touched, which is what that session was asked to respect. It is being fact-checked against the code before it lands — this project's failure mode is documentation asserting features that do not exist, so "another session wrote it" is not evidence that it is true.

The four staleness risks recorded above, resolved

Finding	Status
#152 — cores semantics change invalidates local_parallel_comparison.ipynb	Landed (f0b1218, 4126a03). cores now bounds the local pool and the bound is reachable; cores=0/negative raise ValueError. The notebook measures real numbers, so it needs re-running, not editing.
#158 — default_queue / queue= documented as inert	Removed (fa12781). The configuration.rst entry goes rather than changes.
#157 — auto_add persistence described in two pages	Landed (ea46f05) — it appends one line instead of rewriting the file. Both pages need re-checking against the new behaviour.
#161 — the "Settings that currently have no effect" table	Still open; the table changes or disappears with it.

What stays open here

#162 (eight names in __all__ documented nowhere) and #161's table. Neither is blocked any more.