Release Planning for PY 1.20.5 #1227

Purpose

Coordinate PY 1.20.5 and the companion XL, API-Core, WranglesJS, and Lambda-Recipes changes here. PY 1.20.5 is published. Its release notes contain customer-facing changes only. Companion drafts: XL 2.15.0 and API-Core 1.21.2, both unpublished and targeting main.

Maintain this issue body as the current scope and checklist. Add dated comments for decisions, test results, and deployment evidence; reflect consequential decisions back into the checklist. Check an item only when its stated outcome is verified. Local tests, CI, deployment, and live acceptance are separate evidence.

Agreed scope and decisions

Customer-visible PY candidate

Structured Typesafe answers and row/candidate templates; AI Mode search; tolerant convert.parse; explicit key/semantic lookup operations; saved Extract-AI schema/instruction authoring; Luna defaults and flat/nested reasoning; selected-data reads and columns/sheet output; Gemini retrieval prompts/thinking/deadlines; custom-function binding, matrix error propagation, and split.dictionary fallback fixes; Python 3.14 support and test-selection checks. Also include #1229's part-code match evidence and #1228's missing not_columns fix. The release notes describe these under the same five sections as 1.20.3.

Release checklist

Completed source and development prerequisites

• Publish PY 1.20.5. Eric published the release; PY Deploy Prod and its downstream Lambda Deploy Prod completed successfully on October 11. This is workflow evidence; a live production smoke test remains separate.

• Resolve XL table scope. Eric agreed on October 11 to leave the already-merged XL table helpers in place. No table rollback is required; keep PY Add named Excel table read and write connector #1222 and XL #1291 deferred and omit table development from customer-facing release notes. The ordinary columns/sheet live acceptance check remains open.

• Merge PY #1229 at cea9085a and PY #1228 at 77968950. Part-code evidence and missing not_columns handling are on main; final-candidate CI/deployment and the small documentation follow-up remain separate.

• Move release planning into this issue, rewrite the draft using the customer-facing 1.20.3 sections, and create the catalog/table restoration issues. Install the reusable Codex release-notes-guidance skill for future drafts.

• Retain Centralize AI model configuration and integrate WranglesPY callers #1200 and restore extraction reasoning defaults/flat YAML on main through Preserve extraction reasoning defaults and support flat YAML syntax #1226.

• Merge spreadsheet response fixes: Lambda #64, Lambda #65, and XL #1292. Internal memory is filtered; XL owns unchanged-column suppression.

• Wire Notes through Core #176, JS #123, and XL #1293, using published JS 1.42.0.

• Remove Core catalog dependencies with wrangleworks/API-Core#178.

• Merge Core #179 and persist Service User credential references in the dev task configuration.

• Deploy PY main to the dev recipe Lambda. Observed package/image is dev-1.20.5rc1; user confirmed a successful Notes call showing Luna/none. This is one successful path, not complete XL acceptance.

Remaining release work

• Search-scoring documentation follow-up: PY #1230 merged at 8097d8dd before the PY release.
• Reconcile Replace remaining GPT-4 and GPT-5.6-Luna defaults with gpt-6-luna without the API configuration redesign #1225 with the completed Notes route. The Luna issue now records the implemented runtime path, deferred JS/Web exception, and excluded generation work. No standalone Extract-AI service upgrade is required.
• Defer the extraction example notebook until after release. Eric requested this explicitly on October 11; track the notebook-only checkpoint ed69a569 in PY #1231. It is not a PY 1.20.5 release gate.
• Finish Luna live acceptance. Run final inline/saved/standalone extraction and all Notes checks on the frozen candidate. Include omitted reasoning, flat none, nested {effort: none}, and an explicit override. Metadata remains separate.
• Defer generate.ai. Explicitly excluded from release readiness by Eric on October 11; no generation fix is required for PY 1.20.5.
• Complete or explicitly defer Responses attribution delivery. Review the local Core/Lambda/JS/XL metadata checkpoints, merge and deploy only after that decision, then verify original caller and source name in real OpenAI logs.
• Match package and tag. PY v1.20.5 was published from bd5f88fb. The tagged workflow passed version validation, package/container checks, and publication to CodeArtifact and PyPI.
• Freeze and deploy the coordinated dev candidate. The latest observed PY Deploy Dev run succeeded on October 10 from 1d5e05a2, before Fix missing not_columns entries in write filters #1228/Add detailed part-code match evidence to search result scoring #1229; verified from GitHub on October 11. Promote the final main candidate and verify the recipe Lambda actually uses it before final acceptance. Record exact main commits, PY and JS package versions, schema source, Lambda digest, Core digest/task revision, and XL digest. Refresh CI evidence for any later fixes.
• Complete manual acceptance below on that candidate and record outcomes, including any known limitations.
• Verify production Notes configuration before the XL rollout. Confirm the production Gateway/Core /notes route, runner route, Service User/client configuration, secret references, fixed recipe and dedicated key without exposing secret values. Also verify XL's production WRANGLES_NOTES_API_URL=https://api.wrangle.works/notes; missing or invalid runtime configuration prevents provider initialization. Development success is not production provisioning evidence.
• Draft XL 2.15.0 and API-Core 1.21.2. XL draft covers the net v2.14.3...0c149e4b changes; Core draft covers v1.21.1...7b62e633. Both use the agreed customer-facing sections, omit empty sections, and omit table development, reverted catalog work, and unpublished metadata. Tag, main target, exact body, and unpublished status were re-read after creation.
• Reconcile Lambda release records. Its earlier 1.20.5 draft used an older comparison baseline; refresh against the completed production deployment and compare from 1.20.4.1. This is separate from the completed XL/Core draft preparation.
• Record promotion and rollback artifacts. PY publication dispatches the production recipe-Lambda workflow. XL's tag workflow resolves the current latest image; use and verify an immutable source matching the agreed release scope, including the retained XL helpers. Image promotion and ECS production rollout are distinct.
• Final companion release review and approval. PY publication was performed by Eric. Recheck XL/Core scope, draft text, selected images, production configuration, and rollout order before their publication/deployment. The requested XL/Core draft edits do not authorize publishing or deploying them.

Manual acceptance checklist

Run with small synthetic data first. Record expected/actual results, date, environment, and artifact versions in comments.

• Classic product-search scoring: Add detailed part-code match evidence to search result scoring #1229 is merged. On the final deployed candidate, confirm case-insensitive exact matches, stripped-code equality, MPN-first reduction, cross-source Code deduplication, and pre-reduction counts with the PREA recipe. Confirm the existing one/two-column output mapping and score/filter behavior; record the exact package/source commit.
• Extraction reasoning and models (PY and XL recipe): run omitted reasoning, flat none, nested {effort: none}, then an explicit low override; confirm Luna and effective effort in provider logs. Check both inline schemas and a saved model; saved settings must retain their documented precedence.
• Standalone XL Extract-AI: exercise one-column and multi-column selections; columns, JSON, and delimited outputs; blanks/nulls; schema/instructions; save → reopen → run. Confirm the recipe route and no legacy fallback.
• Spreadsheet recipe integration (desktop and web Excel): five rows with batch size two (2/2/1), filtered selection, an unchanged input column, a column that changes only in a later batch, dynamic columns, mixed columns/sheet writes, missing/null values and row counts, and no internal-memory columns. Repeat using Production versus an explicitly loaded historical Editor version.
• All Notes actions: recipe notes, wrangle notes, and both version-comparison entry points (NotesDialog and TagsDialog); preview/edit/save/reopen and an ordinary authorized user. Confirm safe error handling and absence of client-side provider keys. Verify attribution only after the metadata changes are deployed.
• New PY operations: a representative ai.choose/ai.score/ai.true_false/ai.answers case including row templates and for_each; AI Mode search followed by extraction; tolerant parse input; correct and wrong lookup model variants.
• Regression examples: Gemini retrieval using a full-row prompt field, explicit thinking and a per-URL timeout; split.dictionary.default with invalid/null input; a defaulted custom-function argument; a matrix child error surfaced to the caller.
• Runtime limits and errors: a representative extraction batch plus one controlled invalid-model/provider failure. Confirm useful errors and acceptable latency. Do not treat the still-open bounded-concurrency redesign #1209 as completed.

Assessment evidence

Source checks below were refreshed October 11 where noted. Runtime/deployment observations remain the October 10 snapshot unless explicitly updated.

The focused local model-config rerun was interrupted at Eric's pause request; it is not a new completed-test result. Earlier attempted execution failed because the agent's temporary parent directory was missing, not because of product behavior. The generation differences above were reproduced with synthetic input and mocked provider calls.

Core dev was returned to production SSO/database during the Notes repair; do not assume isolated customer data merely because the route says dev. Keep live tests limited to the agreed test data.

Deferred work and restoration tracking

• PY #1231 — Update extract.ai recipe notebook after PY 1.20.5 release: recover/review the documentation-only checkpoint after release. It is not a release blocker.

• Core #180 — Restore deferred catalog integration after PY 1.20.5: original wrangleworks/API-Core#162 / wrangleworks/API-Core#165 implementation, rollback wrangleworks/API-Core#178, schema and caller prerequisites, and regression/live acceptance.

• XL #1295 — Resume excel.table development after the PY 1.20.5 release: retain merged helpers and resume the deferred feature completion after this release; no rollback/recovery PR is required. Coordinate with PY #1220, PY Add named Excel table read and write connector #1222, and wrangleworks/WranglesXL#1291.

Restoration must use current main and focused PRs; the follow-up issues do not authorize restoring deferred code during this release.

Continue to hold PY #1203/#1206, #1199, #1185, #1179/#1181, #1180, #1169, and broad model refactor #1018 outside this release. Preserve the earlier separate empty-Functions override follow-up under #943. Keep Core row/revision authoring wrangleworks/API-Core#174, catalog follow-ups wrangleworks/API-Core#171 / wrangleworks/API-Core#173, Git shadow export wrangleworks/API-Core#132, async recipes wrangleworks/API-Core#108, and incomplete dev-routing work (wrangleworks/API-Core#164, wrangleworks/WranglesJS#119, wrangleworks/WranglesXL#1253) separate. Reassess statuses when those workstreams resume rather than treating this list as their current completion state.

Decision log

• October 11, production Notes configuration audit: Read-only AWS checks confirmed production Core is still API-Core:22 / image 1.21.1, with none of the six required NOTES_* / ENVIRONMENT settings and no KEYCLOAK_USER_ID or KEYCLOAK_USER_PASSWORD injection. The working dev task is api-core-dev:24; its Notes Service User subject is ec9e1004-82fb-4b33-97b9-00236d839867. Production XL is WranglesXL:15 / image 2.14.3, with no WRANGLES_NOTES_API_URL. The public api.wrangle.works domain maps to API Gateway HTTP API clbqu0wyx6, $default stage, which has neither POST /notes nor OPTIONS /notes and no matching catch-all. Existing Core integration is h9kznh1; existing Core JWT authorizer is a5lex8. Provision these route/configuration prerequisites with the Core/XL production rollout; image publication alone does not add them. No infrastructure changes or secret-value reads were performed.

• October 11, companion drafts: Eric published PY 1.20.5 and requested unpublished XL 2.15.0 and Core 1.21.2 drafts. Created and re-read both drafts against current main and their latest published baselines. JS 1.42.0 is already published and its tagged publish workflow passed; XL's dependency and lockfile select it. No additional JS release was created.

• October 11, notebook deferral: Eric requested a separate issue to update the extraction recipe notebook after release. Created PY Update extract.ai recipe notebook after PY 1.20.5 release #1231 and removed notebook publication from release/Luna acceptance gates. Current PY main cea9085a CI is successful; the 1.20.5 package-version alignment, final-candidate live acceptance, and production configuration/artifact verification remain open.

• October 11, accepted table scope: Leave XL as is. Retain the merged table helpers; keep PY Add named Excel table read and write connector #1222 and XL #1291 deferred. Omit table development from customer-facing release notes. No table rollback is required.

• October 11, table-scope question: Eric asked whether already-merged XL table code can remain if unused. Refreshed XL main is still 0c149e4b; RunRecipe.tsx invokes the helpers, but no table reads means the snapshot helper returns before Excel.run, and no excel.table.write response means the writer returns before Excel.run. An offline probe verified eight non-table recipe shapes without any Excel global; output routing requires the exact excel.table.write marker. All 52 tests passed across recipeTables.test.ts, runRecipeMetadata.test.tsx, and recipeColumnsWrite.test.ts; the tested files are identical to current main. These are offline checks, not live Excel evidence. Recommendation: leave the merged helpers dormant, keep PY Add named Excel table read and write connector #1222 and XL #1291 deferred, do not advertise table support, and smoke-test ordinary columns/sheet behavior in Excel before release. Eric subsequently agreed to leave XL as is and omit table development from release notes. This replaces the earlier requirement to remove the merged XL helpers.

• October 11: Add detailed part-code match evidence to search result scoring #1229 is merged; one small documentation follow-up remains in progress. Ignore generate.ai for this release. Replace remaining GPT-4 and GPT-5.6-Luna defaults with gpt-6-luna without the API configuration redesign #1225 now tracks final runtime acceptance; notebook cleanup was subsequently deferred to Update extract.ai recipe notebook after PY 1.20.5 release #1231 after release.

• October 9: Keep Centralize AI model configuration and integrate WranglesPY callers #1200, defer the larger AI configuration redesign, exclude new Excel tables, preserve simple reasoning YAML, and target Luna for active first-party defaults.

• October 10: Use Core/Lambda/PY for Notes with one dedicated key; defer Web/legacy JS cleanup; roll back Core catalog dependencies.

• October 10: Keep customer-facing changes in the draft release. Move planning, exclusions, TODOs, decisions, and evidence into this issue. Track restoration explicitly.

• October 10: Include the additive classic search-result matching diagnostics from PY #1229 in the intended PY 1.20.5 scope. Preserve the original branch, exclude unrelated configuration history, and require merge/CI/review before treating the feature as included in the frozen package or customer-facing notes.