Include retained TextContent logical bytes in repository usage

[ScottArbeit]

September 7 recovery status

Deferred content-inclusion work. The accepted logical UTF-8, retention, identity and description-availability requirements remain useful. The old mandatory Issue #879 ancestry, bytes-minute producer assumption, epic-branch targeting and execution checklist are superseded. Issue #1045 first establishes what current sources can measure truthfully. Do not implement this issue until a fresh checkpoint selects its measurement contract.

Scott approved D1-D5 in Issue #554. That parent now controls selected scope and current dev-process execution. The original proposal below is retained as historical evidence, not an assignment to resume it.

Historical proposal

Include retained TextContent logical bytes in repository usage

Parent epic: #554 - Grace.Operations usage accounting, billing proof, and cost reconciliation

Related feature epic: #825 - Implement blob-backed work-item descriptions

Deferred from Epic #825

Owner decision on 2026-08-10 selected Option A: Epic #825 ships the Product V1 description feature without claiming retained TextContent usage accounting. This issue moved to Operations epic #554 and remains open future work.

Do not assign implementation until #879 merges the accepted Operations SQL usage-fact journal, dispatcher, worker, and
completeness foundation. This issue then owns the first production repository-storage producer: it must append the
journal before broker delivery and retain enough durable measurement source to retry a failed append. Current Grace has
the fact contract, direct Service Bus publisher helper, ingestion, dedupe, and aggregation, but no production
repository-storage producer. Do not create a second journal, a parallel TextContent ledger, or direct synchronous
Operations SQL coupling from Grace.Actors.

Objective

Extend the repository storage usage measurement owned by Grace.Operations so every retained TextContent object
contributes its uncompressed UTF-8 byte length exactly once, regardless of GZip size or whether a later Description
supersedes or clears it.

Why this matters

TextContent consumes repository storage and must participate in usage limits and billing like other repository content.
Counting compressed size would make charges depend on compression ratio rather than the logical user payload, while a
description-specific accounting path would diverge from the shared Operations ledger.

Quality contract

Profile: Product V1.

Primary invariant: each retained repository TextContent object contributes exactly its recorded uncompressed
Utf8ByteLength to the repository storage bytes-minute observation, independent of compressed size, retries,
supersession, or clearing.

Dependency: re-check #554 and current main before assignment. Reuse the accepted repository storage observation and
projection contract. Do not create a parallel accounting ledger inside WorkItem or TextContent code.

Complexity stop: stop if #554 has not established a usable repository storage measurement seam, if implementation
requires a second billing fact interpretation, or if accurate inclusion requires a new durable state machine not already
accepted by #554.

Context and evidence

• Related feature tracer Deliver the inline TextContent description tracer #826 stores TextContent.Utf8ByteLength as the uncompressed UTF-8 length and retains objects
  after uncertain actor outcomes.
• Parent clear issue Add explicit work-item description clearing #827 retains all prior objects after clear.
• src/Grace.Types/Usage.Types.fs currently defines RepositoryStorageBytesMinute as an immutable minute-normalized
  observation with a positive quantity.
• src/Grace.Actors/OperationalFactsPublisher.Actor.fs publishes usage facts; Operations worker/tests own ingestion and
  hot projections.
• Epic Epic: Grace Operations usage visibility and explainable accounting #554 owns the broader usage, billing, replay, and correction contract. This issue owns only inclusion of retained
  TextContent logical bytes in the accepted repository storage measurement.

Decision closure and contract propagation

Accepted behavior

• Usage quantity uses uncompressed Utf8ByteLength, never GZip object length.
• Every retained TextContent object counts, including superseded descriptions and distinct objects containing identical
  text.
• Clear changes no retained-byte quantity.
• A newly created object conclusively deleted after known actor rejection stops contributing.
• Temporary drift is accepted under the existing deferred usage pipeline.
• Description writes add no synchronous quota reservation, lock, or hard concurrent limit.
• Existing Operations replay, dedupe, correction, and owner/org/repository scoping remain the only accounting path.
• This issue does not expose TextContent IDs, hashes, or per-object detail to users.

Propagation map

Surface	Status	Notes / proof
Shared DTOs, parameters, commands, events, persisted state	Not changed or updated only at accepted usage seam	No work-item public shape change
HTTP route, validation, permissions, error shape	Not changed	Description requests do not synchronously reserve usage
CLI JSON/stdout/stderr/help/examples	Not changed	No new usage command
SDK/facade client	Not changed	No TextContent accounting details exposed
Static OpenAPI and generated clients	Not changed	No public API change
Usage facts and Operations projections	Updated	Existing repository storage observation includes TextContent logical bytes
Events/webhooks/SignalR/watch/search/projections	N/A outside Operations	No new product event surface
Docs/ADRs/agent guidance	Updated	Logical-vs-compressed accounting and #554 dependency
Focused tests and final validation	Updated	Observation, dedupe/replay, retry, retention, and compression-ratio proof

Owned paths

• The repository storage measurement source accepted by Epic: Grace Operations usage visibility and explainable accounting #554 after current-head preflight
• Focused src/Grace.Types/Usage.Types.fs or usage helper changes only if the existing contract requires extension
• src/Grace.Actors/OperationalFactsPublisher.Actor.fs only if the accepted producer seam requires it
• src/Grace.Operations*/** projection and test paths needed to include TextContent in repository storage quantity
• Focused server or TextContent storage metadata/index seam required by the accepted Epic: Grace Operations usage visibility and explainable accounting #554 measurement design
• Matching Types, actor publisher, server integration, and Operations tests
• Usage/accounting and coordination-surface documentation

Forbidden or sensitive paths

• A WorkItem-specific or TextContent-specific billing ledger.
• A new public UsageFact kind without owner-approved expansion of Epic: Grace Operations usage visibility and explainable accounting #554.
• Compressed blob size as billed or limited quantity.
• Synchronous quota reservation or hard-limit locking in description writes.
• Decrementing retained usage when a Description is superseded or cleared.
• Exposing per-object TextContent IDs, hashes, or storage locations in owner usage APIs.
• Provider cost or reconciliation details in user-visible responses.
• Changes to description set/clear semantics.
• Compatibility or stored-data conversion behavior.

Risk surfaces

• Proof/test work
• DTO or contract work at the usage seam
• Storage, Service Bus, Operations worker, and Aspire
• Async/runtime behavior
• Usage/billing projection
• Final audit and docs

Minimum detail gate

Invariant tuple

• Measurement: repository storage bytes-minute observation.
• Identity dimensions: owner ID, organization ID, repository ID, storage pool ID, minute bucket, and retained
  TextContent identity as required by the accepted producer/projection dedupe design.
• Quantity source: uncompressed TextContent.Utf8ByteLength for retained objects.
• Durable path: accepted Epic: Grace Operations usage visibility and explainable accounting #554 usage fact ingestion and projection; no parallel ledger.

Forbidden implementation shapes

• Publishing a one-time bytes-added value as though it were a bytes-minute observation.
• Counting compressed object length.
• Counting only the current description while earlier objects remain stored.
• Double-counting matching retries or replayed usage evidence.
• Coupling description success to Operations availability.
• Inventing a new counter actor or per-description accounting state without owner review.

Expected tests

• Positive: one retained object; multiple versions; identical text with distinct IDs; create description; set description.
• Negative: retry/replay duplicate; conclusively deleted failed-write object; malformed or missing accounting source facts;
  Operations unavailable does not fail description mutation.
• Regression: existing repository storage observations and owner/org/repository scope remain correct; no public contract
  exposes per-object details.
• Boundary: compressible and incompressible payloads with equal UTF-8 lengths contribute equally; clear changes no
  quantity; large accepted payload uses its exact uncompressed length.

High-risk adversarial examples

• A 65,536-character highly compressible description occupies few physical bytes; logical usage still records its full
  UTF-8 length.
• The same description operation is retried after an uncertain result; retained bytes are not counted twice.
• Two separate descriptions contain identical text; both retained objects count.
• Current description is cleared; all older objects continue to count.
• Operations ingestion is temporarily unavailable; description succeeds and deferred accounting recovers through the
  existing Epic: Grace Operations usage visibility and explainable accounting #554 mechanism.

Selected risk-surface traps

• Usage semantics: preserve minute observation rather than treating it as a delta.
• Storage: count retained logical bytes, including non-current content.
• Replay/dedupe: deterministic proof must fail if quantity doubles.
• Runtime: description availability must not depend on Operations availability.
• Final audit: reconcile this issue's implementation with current Epic: Grace Operations usage visibility and explainable accounting #554 contracts before claiming completion.

Explicit N/A waivers

• CLI/SDK/OpenAPI: N/A; no user-facing usage command or per-object contract is added.
• Hard concurrent limit: N/A by accepted design; deferred pipeline drift is allowed.
• Provider reconciliation/cost: N/A; remains owned by Epic: Grace Operations usage visibility and explainable accounting #554 workstreams.
• Data compatibility: N/A; Grace has no production data.

Implementation steps

1. Re-read Epic: Grace Operations usage visibility and explainable accounting #554, its active usage workstreams, and current main usage contracts before assignment.
2. Record the accepted repository storage measurement producer/projection seam in the issue or PR.
3. Stop for owner review if no stable bytes-minute producer seam exists or if the required change conflicts with Epic: Grace Operations usage visibility and explainable accounting #554.
4. Add a failing focused test showing one retained TextContent's uncompressed length is absent from repository storage
   quantity.
5. Extend the accepted logical storage measurement source to include retained TextContent.
6. Use TextContent.Utf8ByteLength; do not inspect or report GZip object length as logical usage.
7. Count every retained TextContent object once under owner/org/repository/storage-pool scope.
8. Continue counting superseded descriptions because their objects remain stored.
9. Make clear leave the retained-byte quantity unchanged.
10. Stop counting only an object conclusively deleted after a known failed mutation.
11. Preserve description mutation success when the Operations pipeline is unavailable.
12. Use existing Epic: Grace Operations usage visibility and explainable accounting #554 retry, replay, dedupe, correction, and minute-normalization behavior.
13. Add proof that matching retries and replay do not double-count.
14. Add proof that two distinct objects with identical text are both counted.
15. Add proof that equal uncompressed lengths contribute equal quantity despite different compression ratios.
16. Add proof for create, set, supersede, clear, and conclusively deleted failed-write cases.
17. Add proof that owner/org/repository/storage-pool scope remains unchanged.
18. Add proof that no per-object identifiers or hashes enter public owner-usage responses.
19. Update Operations and coordination-surface documentation with logical-byte semantics and pipeline freshness.
20. Format touched F# files and run focused Types/Operations/server tests plus git diff --check.
21. Run the Epic: Grace Operations usage visibility and explainable accounting #554-selected integration or Aspire proof when the accepted producer crosses Service Bus/worker storage.
22. Record any residual lag or reconciliation risk without adding synchronous enforcement.

Validation profile

Primary profile: deployment-runtime if the accepted producer crosses Service Bus/worker storage; otherwise
domain-contract with focused Operations projection proof. Select exactly one broad final local gate only when current
epic #554 guidance requires it.

Public behavior under test

• Boundary: repository storage usage observation and owner-scoped Operations projection.
• Behavior: retained TextContent contributes uncompressed logical bytes exactly once without affecting description
  availability.
• RED evidence: current code has usage ingestion plumbing but no proven TextContent inclusion source.

Path lease conflict score

3 - cross-epic accounting surface. This issue depends on #554's current contract and may overlap Operations files.
Do not start concurrently with an active #554 branch touching the same producer/projection files. Refresh and compare
write sets immediately before assignment.

Expected validation

• Run focused usage-type, publisher, ingestion, projection, retry/replay, and description-availability tests selected by
  the actual implementation seam.
• Use Aspire or the Epic: Grace Operations usage visibility and explainable accounting #554 runtime profile if Service Bus and Operations worker behavior changes.
• Run formatting, docs lint, generated checks if applicable, and git diff --check.
• GitHub Validate and fresh Sol High review are required for the same PR head.
• Do not stack Fast/Full on another selected broad gate.

Docs impact

Update Operations usage semantics and the coordination-surface design to state that retained TextContent uses
uncompressed UTF-8 length, prior versions remain billable while retained, clearing does not reduce usage, and accounting
is deferred rather than synchronously reserved.

Definition of done

• All 22 implementation steps are completed or explicitly marked N/A with issue evidence.
• Current Epic: Grace Operations usage visibility and explainable accounting #554 contracts are rechecked and reused without a parallel ledger.
• Every retained TextContent object contributes exact uncompressed UTF-8 bytes once.
• Retry, replay, duplicate text, supersession, clear, failed-write deletion, and compression-ratio proof passes.
• Description success remains independent of Operations availability.
• Public owner usage exposes no TextContent internal facts.
• Worker self-review finds no accounting-semantics or cross-epic conflict gap.
• A ready-for-review PR targeting the epic branch links this issue.
• GitHub Validate and a fresh Sol High review pass for the same PR head.

[ScottArbeit]

Pre-assignment dependency and write-set check

Read-only preflight on 2026-08-10 confirms #829 must not start yet:

• Parent tracer Deliver the inline TextContent description tracer #826 is not merged into the Epic: Implement blob-backed work-item descriptions #825 epic branch.
• Operations epic Epic: Grace Operations usage visibility and explainable accounting #554 remains open and its latest tracker note preserves a sequencing pause around its active
  workstreams.
• Ready PR B4.2: Close owner billing periods and post immutable ledger entries #712 is still open at 04f80259 and changes the same Operations usage-storage tests plus shared Aspire and
  operational-facts test fixtures that Include retained TextContent logical bytes in repository usage #829 is expected to use.

This is a product dependency and merge/write-set conflict, not an implementation failure in #829. Re-run the #554 seam
and open-PR file comparison after #826 merges and immediately before assigning #829. Do not create a parallel
TextContent accounting path or start against the currently conflicting Operations surface.

[ScottArbeit]

Refreshed pre-assignment stop after #828 completion

Read-only preflight on 2026-08-10 confirms the parent description dependencies are now complete, but the Operations conflict remains:

• Epic Epic: Implement blob-backed work-item descriptions #825 integration head is 78ff968f569eacbd2ebc9ad1d487d7f312426689; Deliver the inline TextContent description tracer #826, Add explicit work-item description clearing #827, and Add file and standard-input work-item description sources #828 are merged.
• Operations PR B4.2: Close owner billing periods and post immutable ledger entries #712 remains open and mergeable at 04f8025938068165bb8a7f3395fee1dd8480980f.
• PR B4.2: Close owner billing periods and post immutable ledger entries #712 still changes OperationsUsageStorage.Tests.fs, OperationsWorkerIngestion.Tests.fs, AspireTestHost.fs, OperationalFactsPublisher.Actor.Tests.fs, Operations worker/data projects, and shared migrations.
• Those are the same producer/projection and proof surfaces Include retained TextContent logical bytes in repository usage #829 is required to reuse. Issue Include retained TextContent logical bytes in repository usage #829 explicitly forbids starting concurrently with an active Epic: Grace Operations usage visibility and explainable accounting #554 branch touching those files.
• B4.3 replacement mini-epic: owner billing close and corrections #576's latest tracker state still assigns unresolved session-12 stabilization work on PR B4.2: Close owner billing periods and post immutable ledger entries #712; the PR is not a completed dependency that Epic: Implement blob-backed work-item descriptions #825 can safely consume.

No #829 worker has been started and no branch or worktree has been created. Finishing #829 now requires an owner override of the serialization rule, or completion of PR #712 first. No parallel TextContent accounting path will be created.

[ScottArbeit]

Design-readiness result: no producer exists to extend

The deeper current-state audit changes the blocker from a file-conflict question to a product-scope decision:

• UsageFact.RepositoryStorageBytesMinute exists in Grace.Types.
• Grace.Operations can validate, ingest, deduplicate, and aggregate such facts.
• OperationalFactsPublisher.publishUsageFact exists, but no production code calls it for repository storage.
• Neither current Epic Epic: Implement blob-backed work-item descriptions #825 head 78ff968f nor Operations PR B4.2: Close owner billing periods and post immutable ledger entries #712 introduces a repository-storage producer; PR B4.2: Close owner billing periods and post immutable ledger entries #712 contains only one fact-kind fixture reference and no publisher call.
• RepositoryContentCounterActor counts manifest references, not repository logical bytes, and cannot be reused as a bytes-minute source without changing its durable meaning.
• Therefore Include retained TextContent logical bytes in repository usage #829 cannot “extend the accepted producer seam” as written. Accurate retained TextContent accounting would require a new durable repository-byte measurement design with minute cadence, idempotent retry/restart behavior, and reconciliation. Issue Include retained TextContent logical bytes in repository usage #829 explicitly requires stopping for owner review if that machinery is needed.

Design frontier:

1. Defer Include retained TextContent logical bytes in repository usage #829 into Operations epic Epic: Grace Operations usage visibility and explainable accounting #554 and remove it from Epic: Implement blob-backed work-item descriptions #825's completion contract, allowing the description feature to ship without claiming storage accounting; or
2. Expand Include retained TextContent logical bytes in repository usage #829/Epic: Implement blob-backed work-item descriptions #825 to own the first production repository-storage bytes-minute producer, including the new durable measurement design and its cross-epic integration.

No implementation worker has been started. The overlap with PR #712 remains relevant under option 2, but it is secondary to the missing producer design.

[ScottArbeit]

Owner selected Option A on 2026-08-10.

#829 is deferred from Epic #825 to Operations epic #554. Epic #825 will ship without claiming that retained TextContent participates in repository-storage accounting. #829 remains open future work and must not start until #554 accepts a production repository-storage bytes-minute producer; current Grace has only the fact contract and ingestion path.

The native parent relationship now points to #554. No #829 implementation branch or worktree was created.

[ScottArbeit]

Traceability from #878: the recommended terminal-recovery design requires the first real repository-storage measurement producer to append an Operations usage-fact journal row before publication. #829 still owns the missing measurement and durable source/retry design; direct publishUsageFact cannot be the correctness boundary.

[ScottArbeit]

#unblock: Select the First Repository-Storage Producer Contract

Issue #829 is not ready for a coding worker after #879 merges. Current source can reconstruct retained TextContent
bytes from durable WorkItem events, but it has no complete repository logical-storage inventory, repository-wide
WorkItem enumeration, minute scheduler/cursor, zero-versus-missing record, historical storage-pool rule, or internal
transport into the #879 journal.

A TextContent-only RepositoryStorageBytesMinute fact would appear in the existing classless aggregate as the complete
repository-storage total and would understate customer usage. Blob enumeration cannot repair that: blob keys do not
carry owning scope, accepted-event status, or logical UTF-8 length.

Recommended Choice A

Approve a complete pull-based inventory producer and split #829:

1. Define a private, paged Grace inventory boundary that returns an all-or-nothing as-of-minute logical-storage
   snapshot across every counted object class.
2. Let an Operations-owned scheduler request closed minutes and append positive totals through the B4.3R1c: Implement the usage-fact journal, dispatcher, and terminal recovery #879 journal.
3. Add minimal RepositoryStorageMeasurementCoverage state keyed by owner, organization, repository, storage pool,
   and UTC minute. It records PendingInventory, ObservedZero, or Journaled(UsageFactId) and never stores or sends
   a usage-fact payload.
4. Narrow Include retained TextContent logical bytes in repository usage #829 to TextContent inclusion in that complete inventory: sum retained Utf8ByteLength, preserve superseded
   and cleared content, and exclude conclusively deleted pre-append objects.
5. Add a final enablement/reconciliation leaf so incomplete inventory or unresolved coverage cannot become customer
   usage or billing truth.

This keeps description mutations independent of Operations, avoids direct actor-to-Operations SQL coupling, reuses one
usage-fact journal, and gives restart/catch-up a durable expected-minute record.

Alternatives

• B — TextContent-only dark tracer: lower initial cost, but it cannot populate customer repository-storage usage or
  billing until a complete inventory exists.
• C — Revise UsageFact to allow zero: removes the separate zero marker but changes the accepted fact contract and
  still needs an expected-minute/catch-up rule plus complete inventory.
• D — Direct Operations reads of Grace storage: avoids a private service boundary but couples Operations to actor
  event encodings, blob layout, Cosmos credentials, and retention internals. Not recommended.

Mutation-path push or an actor-owned accounting counter is not offered: the former either couples description success
to Operations or loses restart recovery, while the latter creates a second accounting ledger.

Decision Needed

Please choose A, B, C, or D. Recommendation: A.

After choice A, orchestration will create a design/specification leaf, a producer-foundation leaf, narrow #829 to
TextContent inclusion, and add a final enablement/reconciliation leaf. The implementation base must first contain both
the accepted #825 TextContent work and merged #879 journal work; the current WS4 base does not yet contain #825.