Add model-aware guided Puzzletron setup

[j-rausch]

What does this PR do?

Type of change: New feature

Puzzletron setup exposes many low-level choices before an engineer can produce a usable campaign, and family-only defaults give very different model sizes the same scoring and search budgets. That makes first-time setup harder than necessary and encourages engineers to copy old run files without a clear precedence or provenance contract.

This change adds Quick smoke, Balanced pruning, and High-confidence search profiles while preserving the full-control workflow. Each model family owns generic profile defaults in versioned YAML, and schema 2 can refine only the fields justified for a matching model geometry. Model-specific values are derived from historical Qwen 3.5 0.8B, Qwen 3.5 9B, Qwen 3.6 27B, and Nemotron 3 Nano campaign configurations; machine paths, checkpoints, and cluster-specific topology are not carried forward.

Default precedence is explicit: built-in and model-derived values, then the family profile, then a matching model profile, then an explicitly selected defaults file. Preserved wizard answers remain authoritative when resuming. The resolved-default record distinguishes family profile values from model_profile values, unmatched models inherit the family profile, and ambiguous or malformed model selectors fail closed.

The guided flow persists its selected mode and profile, supports explicit back navigation and resume, validates model and dataset compatibility, reuses compatible parallel settings, requires concrete infrastructure values, and shows effective MIP constraints before generating independently validated smoke and production bundles. Engineers who need every advanced control can use --full; saved states without a setup mode resume in full mode.

The outcome is a shorter first-run setup with defaults that scale to known model variants, without removing advanced controls or making historical infrastructure assumptions part of new campaigns.

Usage

Start guided setup:

python examples/puzzletron/puzzletron_setup_v2.py

Use the exhaustive flow when every advanced setting needs customization:

python examples/puzzletron/puzzletron_setup_v2.py --full

An explicit --defaults file takes precedence over both family and model profile values.

Testing

Ran the focused setup-v2 unit suites for guided setup, data handling, and parallel prompts in the dependency-light setup environment. These cover family fallback, model matching, explicit-file precedence, ambiguous-selector rejection, resume behavior, and end-to-end propagation of model-profile values and provenance through bundle generation.

Also ran Ruff lint and format checks, Python bytecode compilation for changed Python paths, and Git whitespace validation. The repository-wide test harness was not run because the available local environment does not include PyTorch; no GPU or campaign execution was performed.

Compatibility and scope

Schema 1 family-default files remain accepted, the model-aware API arguments are optional, and no dependency or security-sensitive loading behavior is added. Additional model descriptors, model search-space definitions, site-specific execution defaults, wizard modularization, scheduler work, and GPU validation are out of scope.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 97d6f9b6-7afa-446a-a307-74d4b5c3009d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.79%. Comparing base (3485fd0) to head (1d235d8).

Additional details and impacted files

@@                    Coverage Diff                    @@
##           feature/puzzletron_v2    #2117      +/-   ##
=========================================================
+ Coverage                  54.76%   54.79%   +0.03%     
=========================================================
  Files                        704      704              
  Lines                      90869    90869              
=========================================================
+ Hits                       49761    49791      +30     
+ Misses                     41108    41078      -30     

Flag	Coverage Δ
puzzletron	32.91% <ø> (+0.03%)	⬆️
unit	29.75% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.