Skip to content

Cloud Team Collections (S3 + Supabase)#8052

Draft
JohnThomson wants to merge 9 commits into
masterfrom
cloud-tc-for-review
Draft

Cloud Team Collections (S3 + Supabase)#8052
JohnThomson wants to merge 9 commits into
masterfrom
cloud-tc-for-review

Conversation

@JohnThomson

@JohnThomson JohnThomson commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

The full Cloud Team Collections feature — a cloud backend (one S3 bucket + Supabase Postgres/edge functions) for Bloom's Team Collections, alongside the existing folder implementation — packaged as 9 dependency-ordered, review-grained commits. Replaces draft PR #8048 (same tree, byte-identical — verified by an empty git diff against the cloud-collections working branch), whose ~200 orchestration-grained commits were not reviewable.

Suggested review order = commit order:

  1. Design docs, wire contracts (CONTRACTS.md v1.5), project records
  2. Server: tc schema, RLS, RPCs, pgTAP (65 green)
  3. Server: edge functions for the two-phase checkin/download protocols (deno tests 33 green)
  4. Local dev stack (Supabase + MinIO; zero external infra)
  5. C# client core: auth (Firebase Option A + dev GoTrue), API client, repo cache, S3 transfer (AWSSDK v4)
  6. C# backend: CloudTeamCollection, polling monitor, progressive join, background downloads, account-switch takeover with per-collection-copy seats
  7. HTTP API layer: sharing/membership/capabilities/join cards
  8. React UI + XLF strings (en)
  9. Playwright-over-CDP E2E harness + 10 two-instance scenarios

Review caveat: each commit is a coherent reviewable unit and the order makes most of them compile, but only the FINAL tree is test-verified (C# cloud filter 428/428, full suite green, pgTAP 65/65, deno 33/33, E2E matrix green with every scenario passing on this tree). This is the accepted trade-off; per-commit CI-green is not a goal.

Bot review history (on #8048): Greptile reviewed the full diff via the file-limit bypass, found 1 P1 (S3 credential scoping) + 2 P2s — all fixed and verified resolved by its re-review ('No new blocking issues'). Devin cannot analyze a diff this size. CI green.

Remaining before merge: John's two [HUMAN] visual checks (centered check-in dialog; real web upload/download per GOING-LIVE.md 4.3) and the OUTSTANDING BUGS #0 follow-up decision (server-side same-user cross-seat check-in) — tracked in Design/CloudTeamCollections/orchestration/DOGFOOD-BATCH-1.md.

🤖 Generated with Claude Code

Devin review


This change is Reviewable

JohnThomson added a commit that referenced this pull request Jul 11, 2026
…mits, byte-identical), #8048 closed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from 02e5610 to 8ac48df Compare July 14, 2026 14:38
@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Fable 5: regenerated packaging branch — same 9 review-grained commits, now including the 13-14 Jul dogfood fixes: member display names + CONTRACTS v1.6, identity-first book resolution, --attended launch flag, remote-rename dedup, pending-invite constraint fix. Byte-identical to cloud-collections at f5a00c1.)

JohnThomson added a commit that referenced this pull request Jul 14, 2026
…#8052 head 8ac48df), bot gauntlet triggered; handoff list for John while agent is paused
@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR implements the full Cloud Team Collections feature — a cloud backend (S3 + Supabase) for Bloom's Team Collections, alongside the existing folder implementation. It is a large, coherent addition packaged as 9 dependency-ordered commits covering server schema/RLS/RPCs, edge functions, a local dev stack, C# client core (auth, API client, repo cache, S3 transfer), the cloud backend class itself, HTTP API layer, React UI, XLF strings, and a Playwright E2E harness.

  • Efficiency regen (E2–E9): download credentials are cached collection-wide instead of re-fetched per book; idle polls skip the full cache Save + index rebuild; the self-healing rename scan builds the instanceId → folder index once per poll rather than O(missing × local); the history panel fetches only new events via a cursor; collection-file downloads use a per-file manifest and pull only changed files pinned to their committed S3 version.
  • Behavior changes: cloud login now persists across restarts via DPAPI (DpapiCloudTokenStore), and the "Updates Available (N)" badge now counts books checked out by other users (the latest committed version is still fetchable).

Important Files Changed

Filename Overview
src/BloomExe/TeamCollection/Cloud/CloudTeamCollection.cs Core cloud backend: implements GetUpdatesAvailableCount, ReceiveAllUpdates, E2–E9 optimizations, and polling. Contains a stale NOTE in ReceiveAllUpdates that contradicts both the doc comment and the code regarding which books are excluded from the badge count.
src/BloomExe/TeamCollection/Cloud/CloudBookTransfer.cs New file: S3 upload/download with SHA-256 checksum verification, pinned version-id downloads, parallel transfer, staging-then-atomic-swap pattern, and retry logic.
src/BloomExe/TeamCollection/Cloud/CloudCollectionClient.cs New file: RestSharp client for Supabase RPCs and edge functions with auth-retry, typed error mapping, and Newtonsoft-based serialization.
src/BloomExe/TeamCollection/Cloud/CloudRepoCache.cs New file: thread-safe, persisted book/group cache with full/delta snapshot apply, write-through on mutations, and atomic Save via temp-then-move.
src/BloomExe/TeamCollection/Cloud/CloudAuth.cs New file: provider-agnostic session lifecycle with proactive refresh at 80% TTL and on-demand 401 retry.
src/BloomExe/TeamCollection/Cloud/CloudTokenStore.cs New file: DPAPI-backed persistent token store using DataProtectionScope.CurrentUser.
src/BloomBrowserUI/utils/bloomApi.ts Adds useWatchApiDataWithReload and getApiDataOnce shared hooks. Correct caching and dep-suppression.
src/BloomBrowserUI/teamCollection/sharingApi.ts New file: typed TS bindings for SharingApi endpoints using the new shared hooks.
src/BloomBrowserUI/teamCollection/teamCollectionApi.tsx Adds capabilities/status-metadata/cloudCollectionId hooks with getApiDataOnce to prevent per-book re-fetches.
DistFiles/localization/en/Bloom.xlf New XLF strings for cloud TC UI. Previously flagged double-hyphen fixed (now uses semicolon).

Reviews (6): Last reviewed commit: "Cloud TC E2E: Playwright-over-CDP harnes..." | Re-trigger Greptile

Comment thread DistFiles/localization/en/Bloom.xlf
JohnThomson added a commit that referenced this pull request Jul 14, 2026
…e P1)

L10NSharp routes <note> content through XML-comment parsing, where '--' is
illegal, so the CollectionTab.BookNotYetDownloaded note's '... confirmation --
may belong ...' crashed Bloom at startup with "An XML comment cannot contain
'--'" once localization loaded. Replace the '--' with a semicolon. Found by
Greptile on PR #8052; matches the known failure the xlf-strings skill documents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from 8ac48df to b45e5e2 Compare July 14, 2026 23:05
@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from b45e5e2 to 8fbe66e Compare July 15, 2026 02:21
@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Fable 5) Regenerated review branch: re-merged origin/master, and folded in fixes for the Devin slice-review findings (upload-race lock, cache LockedSeat/LocalVersionSeq, experimental-feature-flag token collision, blank cloud history-event label). Byte-identical to the cloud-collections working branch.

JohnThomson added a commit that referenced this pull request Jul 15, 2026
@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from 8fbe66e to aa3842f Compare July 15, 2026 15:13
@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Fable 5) Regenerated after a fresh-eyes review pass over yesterday's fixes: hardened the remote-rename guard (now keyed to the authoritative repo lock, so a checked-out-and-renamed book can't be reverted/clobbered at startup), display-name cache now invalidates on account switch, and the run.sh freshness check uses build outputs that incremental builds actually touch. Regeneration scripts now live in-repo (Design/CloudTeamCollections/orchestration/regen-*.sh). Byte-identical to cloud-collections.

@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from aa3842f to 51da40f Compare July 15, 2026 15:27
@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Fable 5) Regenerated after a documentation cleanup: removed 13 spent orchestration scratch files (one-time agent launch prompts, an implemented design sketch, stale scouting notes) — the diff shrinks from 256 to 243 files. The durable docs (design, CONTRACTS, GOING-LIVE deploy plan, task specs, state log, regeneration procedure) are unchanged. No code changes beyond three comment pointers in the e2e harness. Byte-identical to cloud-collections.

@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from 51da40f to 98bc352 Compare July 16, 2026 07:38
@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Code, Opus 4.8) Regenerated after applying the /simplify quality-review pass across the whole branch — intended as no behavior change:

  • C#: dead-API removal + reuse consolidation (duplicate S3 client → CloudBookTransfer.BuildDefaultClient; manual S3 pagination → S3Extensions.ListAllObjects; hoisted AvailablePath reuse; localManifest hash-reuse in UploadChangedFiles with a new pinning test).
  • C# file-org (pure relocation): CloudTeamCollection.CollectionFiles.cs + TeamCollection.AutoApply.cs partials, Dev/FirebaseCloudAuthProvider split out of CloudAuth.cs, and HandleReceiveUpdates' loop → CloudTeamCollection.ReceiveAllUpdates.
  • Front-end: shared testRender helper + extracted cloud sign-in dialog. Edge functions: shared _shared/paths.ts + S3 upload-verify. Launchers: shared process-tree / automation-ready helpers.

Verification: C# required filter 442/442, vitest + deno green. One item worth a look: ReceiveAllUpdates intentionally keeps its original which-books predicate (skips checked-out-here), which differs from GetUpdatesAvailableCount (excludes locked-by-anyone) — documented inline, not unified.

JohnThomson added a commit that referenced this pull request Jul 16, 2026
All 3 /simplify batches done + committed (6 logical-chunk commits); C# 442/442,
vitest/deno green. Live e2e blocked by environment (WebView2 about:blank under
the machine's memory pressure, not a regression). PR #8052 regenerated locally +
byte-identical, force-push held for John's explicit OK. Flags recorded:
master-merge/launcher reconciliation, live-e2e + launcher smoke for John's env,
and the ReceiveAllUpdates/GetUpdatesAvailableCount predicate drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JohnThomson added a commit that referenced this pull request Jul 16, 2026
…PASS

With memory freed (49% free), e2e-1 + e2e-2 both pass through the live
Supabase+MinIO stack, live-verifying R1/R2 and ReceiveAllUpdates. Earlier
about:blank failures were the documented memory-pressure flake. PR #8052
regenerated + force-pushed byte-identical; Greptile re-triggered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JohnThomson and others added 5 commits July 16, 2026 17:27
Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…transactions

Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…DK v4)

Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JohnThomson and others added 4 commits July 16, 2026 17:27
…flow, background downloads, account-switch handling

Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… book-list merge

Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oad placeholders

Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part of the Cloud Team Collections feature (S3 + Supabase backed Team
Collections). This branch is a regenerable review-grained packaging of the
cloud-collections working branch; only the final tree is test-verified (see
Design/CloudTeamCollections/orchestration/SQUASH-PLAN.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JohnThomson
JohnThomson force-pushed the cloud-tc-for-review branch from 98bc352 to 9de218b Compare July 16, 2026 22:27
@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review. (253 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.

@JohnThomson

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

(Claude Code, Opus 4.8) Regenerated after merging current origin/master into the branch and applying the remaining deferred efficiency/refactor items. Byte-identical to cloud-collections (the regen-rebuild identity check passes).

Backend efficiency (no behavior change):

  • E2 — cache the collection-scoped S3 download credentials and reuse them across books, instead of re-fetching per book (big win during a join).
  • E8 — the history panel now fetches only new events via the get_changes cursor and merges, instead of re-fetching the whole log on every open.
  • E9 — collection-file group download fetches a per-file manifest (get_collection_file_manifest) and downloads only the files whose content changed, each pinned to its committed S3 version — consistent snapshot, no redundant re-downloads.
  • E7 — the per-poll "self-healing" rename scan builds the instanceId → local-folder index once per poll and reuses it, so it is O(missing + local) instead of O(missing × local) disk stats.
  • E3 — idle polls (no new events, cursor unchanged) skip the full repo-cache Save + index rebuild; the group-file check and the self-healing download pass still run every poll.

Behavior changes (intentional, decided with John):

  • Cloud login now persists (DpapiCloudTokenStore) so a real sign-in survives a Bloom restart.
  • The "Updates Available (N)" count now includes books currently checked out by others (fetching their latest committed version is still useful).

Test-only refactors (no production change):

  • Shared CloudTestHarness fixture builder across the cloud unit tests.
  • Shared front-end hooks: useWatchApiDataWithReload and getApiDataOnce (dedup of the reload/cached-promise idioms in sharingApi/teamCollectionApi).

Verification: cloud/TC C# filter 453/453; front-end consumer vitest 38/38; pgTAP/deno green earlier this cycle; live e2e-1/e2e-2 previously passed.

JohnThomson added a commit that referenced this pull request Jul 16, 2026
Records the origin/master merge (10 commits, zero conflicts, merge not rebase
because cloud-collections is shared), the cloud-collections push, and the
cloud-tc-for-review regeneration (0 unmatched, byte-identical, force-pushed,
Greptile re-triggered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants