Reclaim unfinished ContentBlock uploads after one hour of inactivity

[ScottArbeit]

Reclaim unfinished ContentBlock uploads

Outcome

Reclaim payload from unfinished ContentBlock uploads after one hour without new upload progress, while preserving other uploads and completed manifests. Library preparations use the same inactivity window. This keeps failed-upload storage under Grace's control and gives users a useful retry opportunity without charging them for inaccessible unfinished content.

Parent: Issue #554. Scott selected this cleanup implementation, approved the one-hour inactivity rule for Library preparations as well as ordinary uploads, and authorized implementation through review/fixes, Shape Review and CI. Stop before merge approval.

Delivery mode and baseline admissibility

• Mainline slice from origin/main at 9b18c97ab342d319d3bdeaedc5f1d64571913cf5; eventual target main.
• Remote main, fetched origin/main, local main and HEAD match. All 42 worktree registrations were inspected; unrelated work is preserved.
• ADMISSIBLE: current Grace.Server.fsproj Release build succeeded with zero warnings/errors using isolated build output. No earlier branch is required ancestry.
• This is the first selected source prerequisite for the production V1 usage direction. It does not activate historical Issue B4.3R2bR1: Close zero-fact periods from one table-locked pricing snapshot #957 or select the broader usage proposal's remaining measurement/persistence decisions.

Supported world and scope

Product V1; existing supported HTTP/F# SDK/CLI upload producers, non-reentrant Orleans actors, current Cosmos provider and Azure Blob storage. Actual local provider evidence uses Azurite. Ordinary retries, duplicate delivery, caller cancellation, process restart, delayed storage requests and lost responses are included.

One new partial-state lifecycle lives in the existing ContentBlockMetadata owner: temporary upload holds, guarded publication and cleanup. Extend the existing UploadSession lifecycle with its retry deadline; do not create a parallel content owner, reference count, SQL table, catalog or cleanup service.

Required:

• One hour after Start or genuinely new confirmed content, new valid reuse ranges, or first manifest completion. Exact replay, reconfirming an already confirmed block, status reads, discovery, URI issuance and intents do not count as progress.
• Library preparations use that same clock, replacing their fixed 15-minute expiry. Keep repository, operation, principal, hashes, permission, revision and accepted-receipt replay checks.
• After expiry, reclaim whole blocks with no completed-manifest metadata and no remaining upload hold. Delete staging data and compact session evidence only after required block cleanup succeeds.
• Preserve a full retry hour after observed process recovery or substantially overdue reminder processing. Persist the recovery grant; ordinary actor reactivation and duplicate callbacks must not keep extending it.
• Target cleanup within the next 15 minutes when dependencies and reminder processing are healthy. This is a target, not measured capacity.

Excluded:

• Any block with existing ContentBlockMetadata, including zero active counts, pending repository contributions, previously completed but unreferenced manifests and partially used shared blocks. Current metadata presence is monotonic. Existing 24-hour/size/churn/version compaction rules remain unchanged; no operational compaction worker is claimed.
• A manifest selected by a completed session whose metadata merge is still pending remains protected by its upload hold. Do not discard that evidence to meet a cleanup target. Extra uploaded blocks omitted from the completed manifest can be reclaimed.
• This is unfinished-upload cleanup, not general retirement of completed content or a guarantee that every unsuccessful later Library/repository operation leaves no retained storage.
• No usage totals, minute facts, billing, new diagnostics, quotas, whole-file metadata correction, generic health-history service, cloud failover or load-test project. Inaccessible unfinished uploads remain non-chargeable; this child creates no billing producer.
• Delayed empty-placeholder creation may leave a zero-byte object. Such overhead is internal; do not promise permanent object absence or treat an empty placeholder as a valid ContentBlock.

Primary invariant and source ownership

An expired upload may release only its own hold. Cleanup must never remove payload needed by another live upload or by a completed manifest. UploadSession decides its lifecycle; ContentBlockMetadata serializes holds, publication and whole-block deletion for (StoragePoolId, ContentBlockAddress). Blob ETags identify physical incarnations; they do not identify users of unchanged bytes.

Keep call direction session -> block -> storage/dedupe. No block-to-session callback while a session request is waiting. Existing BlockUploadIntents are the release inventory. Reuse claims already require metadata and need no new hold mechanism for this deletion boundary.

Algorithm readiness

Verdict: Simplified and demonstrated for the selected effect sequence. Before production edits, 26 checks passed in the disposable block prototype, 11 in the initial provider boundary test and 19 in the clock/reminder model.

• Real Azurite effects ran in separate processes. They demonstrate that old IfNoneMatch payload writes can recreate deleted content; exact IfMatch payload writes cannot recreate a deleted object or overwrite its replacement. They exercise both arrival orders and lost upload/delete responses.
• Session/block coordination, CAS decisions and clock delivery are explicitly modeled using local JSON. They are not hosted Grace actor results. Implementation must preserve these seams and add focused actor/provider coverage.
• Reuse docs/design/Libraries.Storage-Experiment.md for unchanged Microsoft.Orleans.Persistence.Cosmos 10.2.2-hpk.1 conditional persistence: real stale-ETag rejection, changed failed working buffer, duplicate-create rejection, completed-write response loss and fresh reads. Current actor store registration uses the same provider. This does not establish serialization of the new records; build/codecs and current-head tests must cover them.
• Captures: C:/Source/Grace/artifacts/upload-cleanup-owner-preflight-20260910/decision.json, its run-d4dee37b28a946beb7a5f2f38e2407b0, and C:/Source/Grace/artifacts/upload-cleanup-readiness-20260910/clock-90190ff298284ed7a71a3adc877805c3. Add a portable checked evidence bundle and concise design report to this PR.

Required order:

1. Durably register the existing session block intent before requesting a hold. Session lifecycle revalidation and block calls run inside the serialized session turn.
2. Block owner persists the hold and exact placement before storage work. Retired session/intent identities cannot reacquire, even when release arrived before acquisition.
3. Only an empty placeholder may be created with IfNoneMatch=*. After creating/observing it, successfully persist the exact preparation ETag before any nonempty payload PUT. Payload writes use that exact IfMatch ETag. A stale or uncertain state write must abort before follow-on storage effects; never continue from a changed, failed working buffer.
4. A lost upload response is reconciled by validating complete stored ContentBlock bytes and equivalence while the hold remains. Empty or corrupt content is never upload success. Keep existing route, address, encoding and placement validation.
5. Persist session expiry/terminal state before releasing its recorded holds. Preserve completed-manifest holds until metadata merge is durable. Store retirement even if acquisition has not arrived.
6. Only Metadata.None plus no holds permits a durable deletion reservation. Exclude new holds and metadata acceptance while unresolved. Observe the blob, persist its exact deletion ETag, then conditionally delete. On 412, inspect current state; never equate 412 with absence. If a delayed payload won first, persist the new deletion ETag before retrying. Retain failures for retry.
7. Persist cleanup completion before compacting the session. Retain released-intent history after payload removal. A late request must either be rejected or start a new legitimate upload; it cannot restore an expired hold.

Retry clock and reminders

• Keep one RetryExpiresAt on UploadSession. Remove the independent expiry from its Library binding; keep the existing public Library preparation response's ExpiresAt, populated from the session clock.
• Use one deterministic immutable reminder identity per session/deadline. Establish the first durable reminder before accepting Started. A failed Start can leave a harmless reminder.
• Progress changes the deadline. The existing callback observes it and creates/ensures the next distinct reminder before acknowledging the old one. Duplicate old callbacks must not overwrite or delete the current reminder. Activation or replay may repair the same reminder without extending time.
• Select a 120-second tolerance above the normal 60-second poll. A due callback beyond that tolerance, or a live retry window last advanced before the current process startup, grants one hour from the first observed recovery. Persist that event. Do not use actor activation time or revive a durably closed window.
• Evaluate recovery in the upload retry/preparation path before rejecting it, and in reminder processing. Explain that this detects restart/overdue processing conservatively; it does not measure an outage occurring entirely between observations. Overload or process movement can retain data longer.
• First completion keeps the full hour for Library acceptance. Completed preparation expiry closes its retry window before temporary-state cleanup; metadata-present blocks remain protected. A closed/compacted preparation reports PreparedContentExpired rather than a missing-binding exception. Already accepted Library replay continues to check its permanent receipt before resolving the temporary upload.

Acceptance and required failures

1. Upload valid blocks through the storage route; retry without resending already confirmed content while the window is open. Show a long active upload exceeding one hour, with progress extending its deadline.
2. Demonstrate that duplicate confirmation under a different operation ID and polling do not extend retention. Cover new reuse-range progress without changing its membership rules.
3. Expire an idle session, release only its holds and reclaim eligible real payload. Two sessions sharing a block retain it until both release. Completed metadata at zero count retains it.
4. Restart after publication before session confirmation, before/after deletion reservation, after delete before acknowledgment and around reminder replacement. Assert persisted residue and eventual result, not just error responses.
5. Exercise delayed acquire after release, delayed payload after deletion/replacement and payload-before-delete ordering. A stale provider/state revision must not win.
6. Exercise Library preparation across 15 minutes, progress, completion, expiry and restart. Keep permission/revision rejection and accepted operation replay after temporary cleanup.
7. No observation or RepositoryStorageBytesMinute producer appears; no missing data becomes zero.

Contract propagation and paths

Owned production seams: src/Grace.Types/{UploadSession,ContentBlockMetadata,Reminder}.Types.fs; src/Grace.Actors/{UploadSession,ContentBlockMetadata,Services,Interfaces,Constants,LibraryTransfer,RepositoryLibrary}.Actor.fs; src/Grace.Server/{Storage,Library}.Server.fs. A small helper file in these existing projects is allowed if it concentrates this same lifecycle; project compile entries may change, with no new project/package.

Owned consumers/evidence: focused affected Types/Server.Unit/Server integration tests and their project entries; existing SDK/CLI upload or Library callers only where the same changed DTO/expiry requires propagation; Orleans generated output and existing OpenAPI/client generation surfaces only for this contract. Keep routes and authorization unchanged. Check generated client representation instead of assuming internal types are invisible.

Owned docs: docs/Operations.md, the upload-cleanup sections of docs/adr/0001-content-addressed-storage-contentblocks.md, relevant Library design/contract docs, docs/design/UploadCleanup.Preflight.*, and nearby AGENTS guidance contradicted by the newly approved cleanup behavior. README/CONTRIBUTING only if an advertised command/workflow changes.

Forbidden: unrelated branches, worktrees, dirty files and generated outputs; root undefined/; detached C:/Source/Grace-worktrees/ops1068 and its retained dist; historical Issue #957 worktree including all four dirty files. No reset, clean, prune, or unrelated fixes.

Run, validation and completion

• Tier 2, controller/worker. Root coordinates this issue; one worker owns implementation and all local corrections and accepted R1 repairs. All children use fork_turns=none; no nested agents.
• Format touched F# first. Build the matching Release test projects before focused tests with --no-build. Run meaningful state/codec and provider tests; required GitHub Validate covers the final head. No routine local Fast/Full duplication.
• One R1 discovery review with finite findings; classify/freeze once. One consolidated repair pass by the same owner; R2 only to verify those repairs/direct regressions. Independent HTML Shape Review, refreshed if representations change. No automatic R3.
• Done means coherent committed PR, accepted repairs complete, required final-head Validate passing, review/Shape artifacts ready, scope/evidence limits documented. Leave PR unmerged for Scott.
• Stop if implementation contradicts the selected storage fence, needs a second content owner/lifecycle, broadens into completed-content GC or health-history infrastructure, changes accepted accounting/Library authorization, or requires a third prerequisite before the owner-usage tracer. Routine technical choices and focused fixes remain authorized.

Staging fence refinement before the first candidate

Initial implementation inspection found that GetContentBlockUploadUri could issue a valid SAS at minute 59 while session cleanup ran at minute 60. The supported SDK uses an unconditional PUT with a 60-minute network timeout. A deadline cap fixes a new late arrival but cannot justify assuming an already accepted request was cancelled.

Under Scott's instruction to make the necessary decisions and finish the agreed cleanup, extend the demonstrated exact-ETag sequence to staging inside the existing UploadSession owner. This replaces the earlier assumption that the existing staging delete helper alone was sufficient. The base, delivery mode, primary outcome and existing owners remain unchanged.

• Retain prepared staging placement/ETag in the existing intent/event state. Within the session turn, check its lifecycle and intent, create/observe only an empty staging placeholder, and successfully persist its exact ETag before returning a grant. Failed or stale session writes cannot return a newly prepared grant.
• Cap the SAS expiry at the earlier of 15 minutes from issue and the session retry deadline. URI issuance still is not progress.
• The existing URI response already carries Grace metadata in its fragment. Carry the prepared ContentBlock ETag there too; the SDK places it in the existing StoragePlacement.ETag and requires it for a conditional single PUT. Preserve the current route/response shape. Document the new required grant metadata; do not add a fallback for imagined old production grants. Whole-file upload and read grants are unchanged.
• Cleanup retains the recorded intent until deletion succeeds. A late payload with the old staging ETag cannot recreate a deleted object or overwrite its replacement. Delayed bootstrap creation may leave only zero bytes, which remains internal overhead.
• Extend owned paths to the exact staging URI creation helper in Services.Actor.fs, CAS upload helper in Storage.SDK.fs, and matching SDK/hosted grant-consumer tests and documentation. This is the same upload publication/cleanup sequence, not a second service, owner or content ledger.

The additional disposable provider run passed 15 checks, including four real SAS controls: unconditional valid SAS recreates deleted staging; exact-ETag SAS upload fails after deletion; old ETag fails after replacement; current ETag succeeds. Source/results/receipt: C:/Source/Grace/artifacts/upload-cleanup-readiness-20260910/run-5efc30c2238e42998bf8946027851f63. This tests storage conditions with actual signed requests, not a suspended in-flight socket or new Grace actor integration. The earlier exact-ETag ordering model supplies the persist-before-grant requirement; current-head tests must exercise its integration.

[ScottArbeit]

Run Charter: Issue #1083 unfinished-upload cleanup

Claiming Issue #1083 as ScottArbeit for this run.

• Parent: Issue Epic: Grace Operations usage visibility and explainable accounting #554. Contract: the Issue Reclaim unfinished ContentBlock uploads after one hour of inactivity #1083 body, including the owner-approved one-hour Library inactivity window.
• Outcome: reclaim eligible unfinished upload payload without deleting another upload's or completed manifest's content.
• Execution: controller/worker. Root coordinates; cleanup_owner owns all production changes and accepted repairs. No nested agents; all new contexts use fork_turns=none.
• Delivery: mainline slice targeting main; branch codex/issue-1083-upload-cleanup.
• Base: origin/main at 9b18c97ab342d319d3bdeaedc5f1d64571913cf5. ADMISSIBLE: isolated current Server Release build passed, zero warnings/errors. Delivery delta is this issue delta; historical branches are not ancestry.
• Product V1 scope, one ContentBlock publication/cleanup partial-state lifecycle, owned paths, exclusions and stop conditions are frozen in the issue. Metadata-present blocks stay outside this deletion path. Preserve accepted accounting and Library checks.
• Preflight: 26 block/provider-model controls, 11 initial provider controls and 19 clock/reminder-model controls passed. Current Cosmos package hash and the captured provider-experiment archive hash exactly match docs/design/Libraries.Storage-Experiment.md; reuse its actual stale-write and lost-response evidence only for unchanged provider mechanics.
• Focused validation: format first; matching Release builds and relevant domain/codec/actor tests; provider and hosted acceptance/failure controls; generated contract checks. Required GitHub Validate on final head. No redundant routine local Fast/Full.
• Documentation: selected cleanup and Library deadline contracts, affected actor guidance, portable preflight capture; no broader usage/billing decisions are approved here.
• Review: one independent R1 by gpt-5.6-sol at medium; classify and freeze findings; same worker repairs once; R2 only if repairs. Independent Shape Review by gpt-5.6-terra at high. No review exception and no automatic R3.
• Authorization: implement, commit, push, open normal PR, review/fix and run CI. Leave PR unmerged for Scott's morning review. No unrelated cleanup.
• Charter ready: September 10, 2026.

Start packet

• Assigned worktree: C:/Source/Grace-worktrees/issue-1083-upload-cleanup.
• Expected HEAD: 9b18c97ab342d319d3bdeaedc5f1d64571913cf5; expected working state: clean.
• First slice: implement the demonstrated source-owned publication/cleanup sequence and session clock, propagate the Library expiry, then produce one coherent candidate with focused validation and self-review. The prototype code itself is disposable evidence, not production scaffolding.
• Before editing, verify assigned path, HEAD and status. A matching check authorizes proceeding immediately. Use absolute apply_patch paths under this worktree; verify status here and in C:/Source/Grace after the first edit.
• Preserve root undefined/, detached ops1068 including dist, historical Issue B4.3R2bR1: Close zero-fact periods from one table-locked pricing snapshot #957 state and every unrelated branch/worktree/output. No reset, clean, prune, merge or deletion.

[ScottArbeit]

Issue #1083 staging fence refinement

The issue body now records the staging failure sequence, passing signed-request provider controls, exact implementation decision and affected SDK helper. This is a pre-candidate correction to the initial staging-helper assumption under Scott's explicit direction to make the necessary decisions and finish cleanup. It preserves the base, delivery mode, one upload publication/cleanup lifecycle and existing session/block owners.

Use the same tested empty-placeholder, durable exact ETag and conditional payload sequence for staging. Retain the placement in the existing session intent; require the existing CAS SDK uploader to read the prepared ETag from grant metadata. Cap SAS expiry at the session deadline. Whole-file uploads, read grants, accounting and completed-content retention remain outside the change.

All 15 provider checks passed in run-5efc30c2238e42998bf8946027851f63; four specifically cover signed staging requests. The emulator was stopped and evidence retained. The same implementation owner continues; the first R1 will review the complete candidate including this refinement. No review ledger or approved PR head exists yet.

[ScottArbeit]

Candidate ready for independent review

PR #1086 contains candidate 367591e9d6ffca551a2b68065713f316a8a98b4c from the frozen mainline base. The worktree is clean.

Local validation passed: 223 focused unit checks, 10 SDK checks, and one complete emulator-hosted cleanup case. The hosted case includes actual Cosmos stale-write rejection before grant delivery, zero-byte residue, fresh activation, SDK conditional uploads, two repository-scoped sessions sharing a caller GUID, shared-block protection, cleanup and delayed publication rejection. Release builds, OpenAPI consistency, formatting, lint and diff checks passed.

The candidate propagates the session, intent and event fields through existing OpenAPI bundles. A test-only Aspire endpoint change isolates this run from the preserved Issue #1081 server; no production topology changed. All 29 protected file hashes still match the starting capture.

GitHub Validate, one independent R1 and the HTML Shape Review are running. No review verdict is claimed yet. The same implementation owner will handle accepted repairs. PR #1086 remains unmerged for Scott's review.

[ScottArbeit]

PR #1086 accepted repair ledger

Candidate: 367591e9d6ffca551a2b68065713f316a8a98b4c. Issue #1083, base 9b18c97ab342d319d3bdeaedc5f1d64571913cf5, target main.

The independent R1 found no blocking defect in its first pass. The completed GitHub bot review and current-head Validate run supplied the following additional concrete evidence. Root accepts these items together for one repair pass by the same implementation owner. This is the finite repair list; no second whole-diff discovery review is requested.

A1: Stop copying retirement history into every block event

• Source: GitHub review finding, ContentBlockMetadataActor.SaveUpload and UploadStateChanged.
• Supported sequence: ordinary sessions confirm and release the same content block. Each new event contains the entire growing Retired array, and the persistent event list retains every previous snapshot.
• Impact: serialized retirement identities grow quadratically. A popular block can exhaust a Cosmos document well before the linear retirement history itself would require that space. This is caused by normal included use, not a load-testing or hostile-input requirement.
• Required result: persist transitions or another minimal representation with linear retirement-history growth. Keep every required retirement identity, replay safety, existing source owner, exact placement/ETag ordering and failure semantics. Do not add a retention horizon, quota, new store or compatibility path.
• Validation: replay current events to the same state, reject delayed acquisition after release/restart, exercise failed writes, and compare serialized growth for increasing numbers of normal sessions. Reuse the current effect-order tests.

A2: Make the new persistence tests independent of developer storage settings

• Source: Validate run 34463941593, both FailedWritePoisonsActualActorBeforeAnotherRelease cases.
• Failure: reading Context.loggerFactory initializes the module's Blob client and throws because CI has no local storage connection string in this unit-test process. The injected state-write boundary never runs.
• Required result: the focused adapter tests must reach both rejected and committed-but-response-lost writes in a clean test environment. Use the smallest existing fixture/configuration approach or test seam; retain isolation and do not require a running provider for a unit test.
• Validation: matching build and these tests in a fresh process without inherited developer storage credentials; retain the actual actor and durable-residue assertions.

A3: Update affected storage fixtures and error assertions without weakening their intent

• Source: the same Validate run, ConfirmContentBlockUploadRejectsFinalCasEncodingWithDifferentPhysicalOffsets, ConfirmContentBlockUploadRejectsInvalidFinalCasBlobInsteadOfTreatingItAsSuccess, and ConfirmContentBlockUploadReturnsBadRequestWhenStagedBlockBlobIsMissing.
• Failure: the first two assert old server-layer text after validation moved into the block owner. The third now has an empty prepared placeholder, so its setup no longer represents a missing blob.
• Required result: preserve corrupt/different-content rejection and actual missing-blob coverage. Observe the real object condition; do not merely weaken assertions to accept any error. Keep or add the separate empty-placeholder rejection check when needed.
• Validation: the affected hosted cases pass with their intended physical setup and semantic result.

A4: Resolve the five current-head storage/Library timeouts

• Source: the same Validate run: accepted Library history survives edits repeats rename deletion and cursor segments, ConfirmContentBlockUploadDeletesFinalCasWhenActorRejectsAfterConfirmRace, ConfirmContentBlockUploadRejectsTerminalSessionBeforeFinalCasMaterialization, LargeBinarySaveLifecycleRequiresFinalizedManifestAndDownloadsAfterRouteChange, and LibraryUploadAcceptReadAndRestartReplayUseOneTrackedManifestReference.
• Observed result: 60-second Orleans timeouts inside upload-session confirmation/completion paths. The cause is not established yet; do not label it unrelated or repair it by increasing timeouts.
• Required result: diagnose the shortest failing sequence, distinguish changed-code effects from fixture or environment causes, and restore the included upload/Library behavior. Update old race assumptions only when the new serialized sequence proves the original race is excluded, preserving a meaningful rejection/retention assertion.
• Validation: run the affected hosted cases with real dependencies and retain the diagnosis. Stop if the fix requires a new owner, primary lifecycle, product rule or a change to the selected effect ordering. Otherwise repair within this issue and rerun final-head Validate.

Review and delivery

The baseline, outcome, supported world and source owners stay unchanged. No new capability is approved. The physical-state union suggestion in Shape Review is a nonblocking representation concern; it is not an additional repair requirement. Changes to event representation require refreshed Shape Review and generated/persisted contract checks.

After the same owner commits the consolidated repairs, root will push, run required Validate and resume the original reviewer for R2 limited to A1-A4, repair hunks and direct regressions. The GitHub conversation will receive the actual result and be resolved once satisfied. PR #1086 remains unmerged for Scott.

Captured failure log: C:/Source/Grace/artifacts/upload-cleanup-readiness-20260910/ci-34463941593-failed.log.

[ScottArbeit]

PR #1086 repaired candidate

Head: 36ad9e04b2b0d2f6ebd72fc2472f631a80ac5b31. The same implementation owner completed the accepted A1-A4 ledger in four paths. The repaired candidate is pushed and Validate 34467090875 is running.

A1 replaces old upload snapshots in the existing conditional state write while retaining every non-upload metadata event and retired session identity. Serialized growth was 27,053 bytes for 512 retired identities and 53,165 bytes for 1,024. Replay, early-retirement rejection, rejected writes and committed writes with lost responses are covered by actual-actor controls.

A2 injects the existing logger factory so the adapter tests reach their persistence boundary without developer storage configuration. A3 preserves exact corruption and different-encoding rejection, observes unchanged physical bytes, and distinguishes genuinely missing staging from its unfilled placeholder.

A4's five CI timeouts did not reproduce in the shortest Library history diagnostic or together in the affected hosted group. Captured effect boundaries completed; no cause is established, and no timeout increase or speculative production fix was made. Final-head CI must pass before this item closes.

Matching final Release builds passed with zero warnings/errors; 119 focused unit tests, three clean-process persistence adapter tests, ten affected hosted tests and one shortest diagnostic passed. The hosted diagnostics used temporary logging removed before the final build and unit run. Logs and TRX remain in the worktree's artifacts/upload-cleanup-repair directory.

Review continuation

The original R1 and Shape threads could not resume: the agent tool repeatedly returned agent thread limit reached, including after the implementation owner completed. Creating a fresh R2 reviewer failed with the same error. Root therefore reused the existing read-only diagnostic scout for independent R2 closure. That scout made no source changes. Its original model settings are unavailable, and follow-up cannot override them; this record does not claim the requested Sol model for that continuation. The implementation owner, frozen accepted ledger, closure scope and stop conditions remain unchanged.

The prior Shape artifact is not accepted: its rendering succeeded but required declarations and field analysis were incomplete. Content correction and final-head refresh remain required. No merge is authorized. All 29 protected file hashes still match the initial capture, root has only its original undefined/, and remote main remains the frozen base.

[ScottArbeit]

A4 remains open after final-head CI

Validate 34467090875 at 36ad9e04b2b0d2f6ebd72fc2472f631a80ac5b31 passed the full build and all 1,273 Server.Unit tests. It did not pass the hosted gate. Root cancelled the run after observing a failed test leave the shared server stopped and later tests consume repeated 300-second HTTP timeouts.

Before that cascade, the Library history and LargeBinary tests again timed out in upload confirmation. The Library history failure now occurs at its later preparation (line 259), after the first accepted upload. The Library restart test then received its expected HTTP 500 but found no durable operation receipt after stopping the server. That result does not establish that the intended post-acknowledgement fault boundary was reached.

The same implementation owner is continuing the existing A4 item. The next candidate will add bounded diagnostics around session retry observation, block publication, state writes and blob effects, attach existing server logs on failure, report the interrupted request body, and restore the shared test server when a stopped-server assertion fails. This is diagnosis and completion of the frozen ledger, not another discovery review or a new product rule. No timeout increase is approved.

The current bot closure reported no major issues, and A1's review conversation is resolved. A1-A3 closure remains supported; R2 must not certify A4 until the actual final-head hosted gate passes. The corrected Shape artifact is separately pinned to this head and cannot establish CI success.

Full captured log: C:/Source/Grace/artifacts/upload-cleanup-readiness-20260910/ci-34467090875-cancelled.log. The live job page was used to identify the cascade while the log API still returned 404. No unrelated process, branch, worktree or output was changed. PR #1086 remains unmerged.

[ScottArbeit]

Implementation and review checkpoint

PR #1086 implements the approved unfinished-upload cleanup policy, including one-hour inactivity for Library preparations. Final head: 14adf94068bc5ef97d1c8749bf7fe2cf77ffe681, targeting main. The PR remains unmerged for Scott's approval.

The existing session and block owners retain retry and cleanup evidence, preserve overlapping uploads and completed content, and use durable exact-ETag preparation before payload writes. Retired session identities reject delayed acquisition after cleanup. No usage or billing producer was added.

Final-head Validate 34476004725 passed: 3,488 tests passed, 93 skipped and zero failed across eight test projects. This includes 322 hosted tests, 1,273 server unit tests and 1,572 CLI tests. Matching local Release builds and 119 focused unit tests also passed after diagnostic removal.

The accepted A1-A4 repairs are complete. Retirement history grows linearly while retaining replay safety; persistence tests no longer depend on developer storage settings; negative fixtures observe the intended physical object; and the hosted upload/Library paths pass after the narrowly scoped Local debug connection workaround. All temporary diagnostic code is removed. The synthetic Linux comparison passed both connection modes and did not reproduce CI, so the record makes no payload-size or detailed emulator-race claim.

R2 closure verified all frozen A1-A4 ledger items on the final head. No blocking repair regression, discovery escape, scope expansion or remaining owner decision was found. The final-head bot review found no major issues, and the retirement-history review thread is resolved.

The Shape Review is pinned to the final head and covers 49 of 49 entries. It records two nonblocking representation concerns and no open product questions. Root inspected full rendered images at CSS widths 1440, 1024 and 768 with DPR 1. Anchor, theme, mobile-menu and disclosure checks passed; no page overflow, clipped prose or page errors were found. The artifact is at C:/Users/scott/.codex/visualizations/2026/09/10/01a08911-c1be-7492-806c-09f935ccb36c/upload-cleanup/PR-1086-Shape-Review.html.

The original review tasks could not resume because of the agent tool's thread limit. An existing independent read-only scout completed the follow-up reviews without source edits; its unavailable model settings are explicitly disclosed in the prior checkpoint and artifact. The implementation owner and finite accepted repair scope stayed unchanged.

The preflight distinguishes actual Azurite effects from coordination and clock models. Hosted cleanup is triggered deterministically; no hour-long wait or reminder-service capacity qualification is claimed. The healthy next-15-minute cleanup objective remains a target. Completed-content garbage collection and broader Issue #554 measurement/persistence decisions remain outside this slice.

Local and remote main remain 9b18c97ab342d319d3bdeaedc5f1d64571913cf5. All 29 protected file hashes match their captured values, including undefined/, detached ops1068 output and the retained dirty Issue #957 work. The issue worktree is clean. No unrelated branch, worktree, generated output or process was removed.