@@ -87,6 +87,46 @@ ssh -N -L 1234:localhost:1234 user@remote-host
 
 Then set `base_url` to `http://localhost:1234` while the tunnel is running.
 
+## Thinking Tokens and Reasoning Content
+
+Some advanced models (like Qwen3.5 with extended thinking) can be configured in LM Studio to emit **thinking tokens** — internal chain-of-thought reasoning that doesn't appear in the final output.
+
+### The Gotcha
+
+When thinking tokens are enabled:
+- The model may return `reasoning_content` (internal reasoning) but `content: null` (final output empty)
+- llama_index's standard `OpenAILike` class only reads `content`, causing a crash if it's None
+
+### Solution
+
+Use the `ThinkingAwareOpenAILike` class instead of `OpenAILike` in your config:
+
+```json
+"lmstudio-qwen3.5-9b-with-thinking": {
+    "class": "ThinkingAwareOpenAILike",
+    "arguments": {
+        "model": "qwen/qwen3.5-9b",
+        "api_base": "http://127.0.0.1:1234/v1",
+        "api_key": "lm-studio",
+        "temperature": 0.55,
+        "timeout": 600.0,
+        "is_function_calling_model": false,
+        "is_chat_model": true,
+        "context_window": 32768,
+        "max_tokens": 4096,
+        "should_use_structured_outputs": true
+    }
+}
+```
+
+`ThinkingAwareOpenAILike` safely handles both:
+1. **With thinking:** Falls back to `reasoning_content` if `content` is null
+2. **Without thinking:** Works exactly like `OpenAILike` for normal responses
+
+### Best Practice
+
+If you're enabling thinking in LM Studio (e.g., via the Developer preset), always use `ThinkingAwareOpenAILike` to avoid crashes. See [issue #228](https://github.com/PlanExeOrg/PlanExe/pull/228) for technical details.
+
 ---
 
 ## Next steps
 
@@ -15,7 +15,7 @@ The plan is a **project plan**: a DAG of steps (Luigi pipeline stages) that prod
 Implementors should expose the following to agents so they understand what PlanExe does:
 
 - **What:** PlanExe turns a plain-English goal into a strategic project-plan draft (20+ sections) in ~10–20 min. Sections include executive summary, interactive Gantt charts, investor pitch, SWOT, governance, team profiles, work breakdown, scenario comparison, expert criticism, and adversarial sections (premortem, self-audit, premise attacks) that stress-test the plan. The output is a draft to refine, not an executable or final document — but it surfaces hard questions the prompter may not have considered.
-- **Required interaction order:** Call `example_plans` (optional) and `example_prompts` first. Optional before `plan_create`: call `model_profiles` to inspect profile guidance and available models in each profile. Then complete a non-tool step: formulate a detailed prompt as flowing prose (not structured markdown), typically ~300-800 words, using the examples as a baseline; include objective, scope, constraints, timeline, stakeholders, budget/resources, and success criteria; get user approval. Only after approval, call `plan_create`. Then poll `plan_status` (about every 5 minutes); use `plan_download` (mcp_local helper) or `plan_file_info` (mcp_cloud tool) when complete (`pending`/`processing` = keep polling, `completed` = download now, `failed` = terminal). If a plan fails mid-pipeline, call `plan_resume` to continue from where it left off without discarding completed tasks. Use `plan_retry` for a full restart. Both accept the failed `plan_id` and optional `model_profile` (default `baseline`). To stop, call `plan_stop` with the `plan_id` from `plan_create`.
+- **Required interaction order:** Call `example_plans` (optional) and `example_prompts` first. Optional before `plan_create`: call `model_profiles` to inspect profile guidance and available models in each profile. Then complete a non-tool step: formulate a detailed prompt as flowing prose (not structured markdown), typically ~300-800 words, using the examples as a baseline; include objective, scope, constraints, timeline, stakeholders, budget/resources, and success criteria; get user approval. Only after approval, call `plan_create`. Then poll `plan_status` (about every 5 minutes); use `plan_download` (mcp_local helper) or `plan_file_info` (mcp_cloud tool) when complete (`pending`/`processing` = keep polling, `completed` = download now, `failed` = terminal). If a plan fails before completing all steps, call `plan_resume` to continue from where it left off without discarding completed work. Use `plan_retry` for a full restart (plan must be in failed state). Both accept the failed `plan_id` and optional `model_profile` (default `baseline`). To stop, call `plan_stop` with the `plan_id` from `plan_create`.
 - **Output:** Self-contained interactive HTML report (~700KB) with collapsible sections and interactive Gantt charts — open in a browser. The zip contains the intermediary pipeline files (md, json, csv) that fed the report.
 
 ### 1.3 Scope of this document
@@ -438,9 +438,9 @@ Retries a plan that is currently in `failed` state.
 
 ### 6.6 plan_resume
 
-Resume a failed plan without discarding completed pipeline outputs. The pipeline restarts from the first incomplete task, skipping all tasks that already produced output files.
+Resume a failed plan without discarding completed intermediary files. Plan generation restarts from the first incomplete step, skipping all steps that already produced output files.
 
-Use `plan_resume` when `plan_status` shows `failed` and the run was interrupted mid-pipeline (network drop, timeout, `plan_stop`, worker crash). For a full restart or to change `model_profile`, use `plan_retry` instead.
+Use `plan_resume` when `plan_status` shows `failed` and plan generation was interrupted before completing all steps (network drop, timeout, `plan_stop`, worker crash). For a full restart or to change `model_profile`, use `plan_retry` instead.
 
 **Request**
 
@@ -470,25 +470,42 @@ Use `plan_resume` when `plan_status` shows `failed` and the run was interrupted
 
 **Required semantics**
 
-- Only failed plans are resumable.
+- The MCP tool only accepts plans in `failed` state. However, the underlying Luigi mechanism is more general: Luigi skips any task whose output file already exists and re-executes any task whose output file is missing. This means a completed plan can be partially re-run by deleting `999-pipeline_complete.txt` and the output files of the tasks you want to regenerate — Luigi will re-execute those tasks and all their downstream dependents. The MCP API does not yet expose this capability; it is available when running the pipeline locally via `run_plan_pipeline.py`.
 - On success, the same plan_id is reset to `pending` and requeued.
-- Prior artifacts are **preserved** — the worker restores the run directory from the stored zip snapshot.
-- Luigi skips tasks whose output files already exist; only incomplete tasks are re-executed.
+- Prior artifacts are **preserved** — the worker restores the output directory from the stored zip snapshot.
 - `resume_count` tracks how many times the plan has been resumed.
 
 **When to use plan_resume vs plan_retry**
 
 | Scenario | Use |
 |----------|-----|
-| Run interrupted mid-pipeline (network drop, timeout, stop, crash) | `plan_resume` |
+| Interrupted before completing all steps (network drop, timeout, stop, crash) | `plan_resume` |
 | Full restart from scratch | `plan_retry` |
-| Change model_profile for a fresh run | `plan_retry` |
+| Change model_profile for a fresh start | `plan_retry` |
 | Continue where you left off, preserving completed work | `plan_resume` |
 
+**Pipeline compatibility caveat**
+
+Resuming restores the output directory from a stored zip snapshot and relies on Luigi's file-based step skipping: if an output file exists, the step is skipped. This works reliably when the server code matches the code that produced the snapshot. It can break when the server has changed between the original plan generation and the resume attempt:
+
+- **Renamed or removed filenames** (`FilenameEnum` in `filenames.py`): if a step's output filename changes, Luigi won't find the old file and will re-execute the step — but downstream steps that already completed may reference the old filename's content, producing inconsistencies.
+- **New steps added**: new steps may expect upstream outputs that don't exist in the snapshot.
+- **Changed step dependencies**: if the DAG wiring changes, Luigi may execute steps in a different order or skip steps that should re-execute.
+- **Changed JSON schemas**: resumed steps may read snapshot files whose internal structure no longer matches what the current code expects.
+
+In practice, resuming a plan that failed minutes or hours ago on the same deployment is safe. Resuming a plan that is weeks or months old — after the server has been updated with pipeline changes — may produce errors or inconsistent output. When in doubt, use `plan_retry` for a clean restart.
+
+**Pipeline version check**
+
+A `PIPELINE_VERSION` integer constant (in `worker_plan_api/pipeline_version.py`) is stamped into the plan's `parameters` on every `plan_create`, `plan_retry`, and `plan_resume`. On `plan_resume`, the stored version is compared to the current constant using strict equality. If they differ, resume is rejected with error code `PIPELINE_VERSION_MISMATCH` and the caller should use `plan_retry` for a clean restart. Legacy plans with no stored version are treated as incompatible.
+
+Bump `PIPELINE_VERSION` whenever the pipeline changes in a way that would break resume from an older snapshot (renamed filenames, new tasks, changed DAG wiring, changed JSON schemas). Finer-grained approaches (manifest, per-task schema versions) can be layered on later if needed.
+
 **Error behavior**
 
 - Unknown plan_id: `PLAN_NOT_FOUND` (`isError=true`).
 - Plan not failed: `PLAN_NOT_RESUMABLE` (`isError=true`).
+- Pipeline version mismatch: `PIPELINE_VERSION_MISMATCH` (`isError=true`). Use `plan_retry` instead.
 
 ---
 
@@ -594,6 +611,7 @@ Cloud/core tool codes:
 - `PLAN_NOT_FOUND`: plan_id not found.
 - `PLAN_NOT_FAILED`: plan_retry called for a plan that is not in failed state.
 - `PLAN_NOT_RESUMABLE`: plan_resume called for a plan that is not in failed state.
+- `PIPELINE_VERSION_MISMATCH`: plan_resume snapshot was created by a different pipeline version; use plan_retry instead.
 - `INVALID_USER_API_KEY`: provided user_api_key is invalid.
 - `USER_API_KEY_REQUIRED`: deployment requires user_api_key for plan_create.
 - `INSUFFICIENT_CREDITS`: caller account has no credits for plan_create.
@@ -619,6 +637,7 @@ Local proxy specific codes:
   - `INVALID_TOOL`
 - For `PLAN_NOT_FAILED`: call `plan_retry` only after `plan_status.state == failed`.
 - For `PLAN_NOT_RESUMABLE`: call `plan_resume` only after `plan_status.state == failed`.
+- For `PIPELINE_VERSION_MISMATCH`: the snapshot is incompatible with the current pipeline; use `plan_retry` for a clean restart.
 - For `PLAN_NOT_FOUND`: verify plan_id source and stop polling that id.
 - For `generation_failed`: treat as terminal failure and surface plan progress_message to user.
 
 
@@ -42,6 +42,63 @@ The target is practical: reduce repeated run failures caused by structured-outpu
 3. **Schema echo** (model repeats schema shape/descriptions, not values).  
 4. **Code-path assumption bugs** (`KeyError` on non-API path).
 
+### 1.4 Local model testing findings: llama_index silent prompt truncation (March 2026)
+
+#### Root cause: llama_index default constants
+
+Testing on Mac Mini M4 Pro (March 4-6, 2026) with qwen/qwen3.5-35b-a3b via LM Studio revealed the underlying cause of silent structured-output failures:
+
+**File**: `llama_index/core/constants.py`
+```python
+DEFAULT_CONTEXT_WINDOW = 3900  # applies to ALL OpenAILike models unless overridden
+DEFAULT_NUM_OUTPUT = 256       # affects PromptHelper budget, NOT model output limit
+```
+
+These defaults apply to **all OpenAILike models** unless explicitly overridden in the model config. The `PromptHelper` computes:
+```
+available_prompt_budget = context_window - num_output = 3900 - 256 = 3644 tokens
+```
+
+#### The failure mode
+
+`DeduplicateLeversTask` input was ~8,400 tokens. With the default budget of 3644 tokens, the prompt was **silently truncated to 3644 tokens** before being sent to the model. The model received a mangled, mid-sentence prompt, produced thinking text instead of valid JSON, and the pipeline failed with `ValueError: Could not extract json string`.
+
+This is **silent** — no warning, no log entry, no error at truncation time. The failure manifests downstream as a JSON extraction error, making it extremely hard to diagnose.
+
+#### Critical insight: `num_output` does NOT limit model output
+
+A common misunderstanding: setting `num_output: 4096` does NOT cap the model's actual output length. The LM Studio API generates freely (10K+ tokens observed). The limit ONLY affects llama_index's internal `PromptHelper` budget calculation for **input** truncation. This is the root cause of misdiagnosis.
+
+#### The fix
+
+Add explicit overrides to each LM Studio model entry in `llm_config/custom.json`:
+```json
+"context_window": 8192,
+"num_output": 4096
+```
+
+This expands the available prompt budget from 3644 to 4096 tokens, accommodating tasks with large combined inputs. For models with larger context windows (e.g., 32K), use `context_window: 32768, num_output: 4096`.
+
+#### Tasks most vulnerable to silent truncation
+
+Tasks with large combined inputs (multiple prior outputs concatenated):
+- `DeduplicateLeversTask` — ~8,400 tokens input (confirmed failure point)
+- `PremortemTask` / `ReviewPlanTask` / `QuestionsAndAnswersTask` — 10-12 docs concatenated
+- `GovernancePhase4-6` — accumulating context chain
+
+#### Detection method
+
+To verify if your deployment is vulnerable:
+```bash
+grep -r "DEFAULT_CONTEXT_WINDOW\|DEFAULT_NUM_OUTPUT" $(pip show llama-index-core | grep Location | cut -d' ' -f2)/llama_index/core/constants.py
+```
+
+If values are 3900/256 and your model config doesn't override them, you're vulnerable to silent truncation.
+
+#### Relationship to this roadmap
+
+Silent truncation is exactly the class of mid-pipeline failures that the containment and telemetry work packages (WP-1, WP-2, WP-3) are designed to catch and surface. By adding structured failure logging (WP-3), future truncation events will be diagnosable instantly. By adding a preflight smoke test (WP-5), bad model profiles can be caught before expensive full-pipeline runs.
+
 ---
 
 ## 2) Guardrails and merge policy
 
@@ -2,7 +2,7 @@
 
 **Author:** Bubba (VoynichLabs)  
 **Date:** 2026-03-08  
-**Status:** Proposal
+**Status:** Implemented (PR #219)
 
 ---
 
@@ -165,18 +165,90 @@ None of these are present in a bare CLI run. The file-based approach requires on
 
 ---
 
-## Open Questions
+## Implementation Notes (PR #219)
 
-1. **Ollama field mapping**: `prompt_eval_count` / `eval_count` vs `prompt_tokens` /
-   `completion_tokens` — should `extract_token_count()` add an Ollama-specific branch, or
-   should the Ollama LLM adapter normalize field names before returning?
+The final implementation diverges from Option A above in several ways:
 
-2. **Thread safety**: Luigi runs tasks in parallel workers. The appender needs an
-   `fcntl.flock()` or atomic rename approach to avoid corruption under concurrent writes.
+### File format: JSONL, not JSON
 
-3. **Thinking tokens for local models**: Qwen 3 thinking tokens are stripped before the
-   response reaches llama_index (LM Studio does not expose them separately). The
-   `thinking_tokens` field will be `null` for local runs — acceptable for now.
+`usage_metrics.jsonl` uses one JSON object per line (append-only). This avoids the need for
+atomic rename or in-memory accumulation — each LLM call appends a single line. Thread-safe
+by nature since each write is a short append to a file handle.
 
-4. **Retention**: Should `usage_metrics.json` be overwritten on resume (Luigi skip-completed
-   behavior) or appended? Appending is safer — a resumed run adds only the new calls.
+### Recording source: llama_index instrumentation, not LLMExecutor
+
+Successful calls are recorded by `TrackActivity` (the llama_index `BaseEventHandler`) which
+receives the actual `ChatResponse` with full token counts, cost, and `provider:model` info.
+`LLMExecutor._record_attempt_token_metrics()` only records **failures**, since instrumentation
+end events are not emitted when the LLM call fails.
+
+This was necessary because `execute_function(llm)` returns the processed result (a Pydantic
+model or string), not the raw `ChatResponse`. The instrumentation layer is the only place
+with access to the real response.
+
+### Model field includes provider
+
+The `model` field contains the full `provider:model` string (e.g.
+`Google AI Studio:google/gemini-2.0-flash-001`), matching `activity_overview.json`.
+
+### Example output
+
+```json
+{"timestamp": "2026-03-10T13:36:48.250446", "success": true, "model": "Google AI Studio:google/gemini-2.0-flash-001", "duration_seconds": 4.879, "input_tokens": 5316, "output_tokens": 643, "cost_usd": 0.0007888}
+{"timestamp": "2026-03-10T13:36:53.554864", "success": true, "model": "Google:google/gemini-2.0-flash-001", "duration_seconds": 5.237, "input_tokens": 8877, "output_tokens": 562, "cost_usd": 0.0011125}
+```
+
+### Key files
+
+| File | Role |
+|------|------|
+| `worker_plan/worker_plan_internal/llm_util/usage_metrics.py` | Core module: `set_usage_metrics_path()`, `record_usage_metric()` |
+| `worker_plan/worker_plan_internal/llm_util/track_activity.py` | Records successful calls via `_record_file_usage_metric()` |
+| `worker_plan/worker_plan_internal/llm_util/llm_executor.py` | Records failed calls only |
+| `worker_plan/worker_plan_internal/plan/run_plan_pipeline.py` | Sets/clears metrics path around pipeline execution |
+| `worker_plan/worker_plan_api/filenames.py` | `USAGE_METRICS_JSONL` constant |
+
+### Resolved open questions
+
+1. **Ollama field mapping**: Handled by `extract_token_count()` and `TrackActivity._extract_token_usage()` which already support multiple field name variations.
+2. **Thread safety**: JSONL append-per-line is safe for concurrent Luigi workers.
+3. **Thinking tokens**: Recorded when available (e.g. from OpenRouter reasoning models). `null` for providers that don't expose them.
+4. **Retention on resume**: Appended. A resumed run adds only the new calls alongside the restored snapshot's existing metrics.
+
+---
+
+## Follow-up: Error Classification (PR #236)
+
+Raw Python exception strings in the `"error"` field were verbose and could leak raw model output.
+PR #236 added `classify_error()` to replace them with short category labels:
+
+| Category | Matches on |
+|---|---|
+| `invalid_json` | "json", "validation error", "pydantic" |
+| `timeout` | "timeout", "timed out" |
+| `empty_response` | "empty", "none", "no response" |
+| `connection_error` | "connection", "connect", "dns", "network" |
+| `rate_limit` | "rate limit", "429", "too many requests" |
+| `auth_error` | "auth", "401", "403", "unauthorized", "forbidden" |
+| `server_error` | "500", "502", "503", "504", "server error", "internal server" |
+| `model_not_found` | "model not found", "not found", "404" |
+| `unknown` | fallback — raw message preserved in `error_detail` (truncated to 200 chars) |
+
+---
+
+## Follow-up: Error Traceability (PR #237)
+
+All 38 call sites that raised `ValueError("LLM chat interaction failed.")` were converted to
+`LLMChatError` (see [proposal 113](113-llm-error-traceability.md)). This:
+
+- **Preserves the root cause** in `str(LLMChatError)`, so `classify_error()` categorizes correctly
+  (previously all were `"unknown"`).
+- **Adds `error_id`** (12-char UUID) to both log output and the JSONL row, enabling cross-referencing.
+
+Example failure row after both PRs:
+
+```json
+{"timestamp": "2026-03-10T19:50:18.821350", "success": false, "model": "openrouter-gemini-2.0-flash-001", "duration_seconds": 5.391, "error": "invalid_json", "error_id": "4c2a64973bcd"}
+```
+
+A user can `grep 4c2a64973bcd` in the logs to find the full traceback.