diff --git a/README.md b/README.md index 4039b50b..5c14e3a2 100644 --- a/README.md +++ b/README.md @@ -36,13 +36,16 @@ Situated AuthZ offers some advantages for typical use-cases: ## Performance -- EACL recursively traverses the ReBAC permission graph via low-level Datomic `d/index-range` & `d/seek-datoms` calls to efficiently yield cursor-paginated resources in the order they are stored at-rest. Results are _always_ returned in the order they stored in at-rest, which are internal Datomic eids. - - I have investigated implementing custom Sort Keys, but they are not currently feasible without adding a lot of storage & write costs. +- EACL traverses the ReBAC permission graph via low-level Datomic `d/index-range` & `d/seek-datoms` calls to efficiently yield cursor-paginated resources without materializing the full reachable closure. + - Non-recursive lookups retain the existing at-rest terminal-index ordering. + - Recursive `lookup-resources` uses a stable deterministic discovery order with exact deduplication across pages. It does not guarantee global eid ordering. - EACL is fast, but makes no strong performance claims at this time. For typical workloads, EACL should be as fast as, or faster than, SpiceDB. EACL is not meant for hyperscalers. - EACL is internally benchmarked against ~800k permissioned resources with good latency (5-30ms per query). You can scale Datomic Peers horizontally and dedicate peers to EACL as needed. - The performance goal for EACL is to handle 10M permissioned entities with real-time performance. - EACL does not support all SpiceDB features. Please refer to the [limitations section](#limitations-deficiencies--gotchas) to decide if EACL is right for you. -- Presently, EACL has _no cache_ because graph traversal is fast enough over Datomic's aggressive datom caching even for ~1M permissioned resources. A cache is planned and once it lands, should bring query latency down to ~1-2ms per API call, even for large pages. +- Presently, EACL has _no result cache_ because graph traversal is fast enough over Datomic's aggressive datom caching even for ~1M permissioned resources. A cache is planned and once it lands, should bring query latency down to ~1-2ms per API call, even for large pages. +- EACL does cache resolved *permission paths and query plans*. The cache is invalidated **only by `eacl/write-schema!`**, which bumps a schema-version stamp (`:eacl/schema-version`) stored in the database in the same transaction as the definition change — so invalidation reaches every peer, `d/as-of` views resolve the paths of their own era, and unrelated `d/transact` calls never touch a cache key (zero per-transaction overhead — issue #74). Editing relation/permission datoms outside `write-schema!` is not detected by design; if you must, call `eacl.datomic.impl.indexed/evict-permission-paths-cache!` on every peer afterwards. +- Recursive `lookup-resources` cursors carry the recursion state and grow ~48 bytes per emitted resource, because exact cross-page deduplication requires the emitted set. Prefer bounded pagination sessions on recursive schemas; a cursor-state redesign is planned. - Performance should scale roughly with permission graph complexity * `O(logN)` for `N` resources in terminal resource Relationship indices. Parallel paths through the graph that return the same resources will slow EACL down, because these resources need to be deduplicated in stable order. In a simple graph, performance should approach `O(logN)` for N permissioned resources. Subjects are typically sparse compared to resources, i.e. 1k users will have access to 1M resources – rarely the other way around. *Note* that to retain future compatibility with the SpiceDB gRPC, the EACL Datomic client calls `(d/db conn)` on each API call, which means that if your DB changes inbetween EACL queries, you may see inconsistent results when cursor paginating. You can pass a stable `db` basis and shave off a few milliseconds by calling the internals in `eacl.datomic.impl.indexed` directly – these functions take `db` as an argument directly instead of `conn`. If you do this, you will need to coerce internal Datomic eids to/from your desired external IDs yourself. @@ -54,6 +57,18 @@ Situated AuthZ offers some advantages for typical use-cases: > I try hard not to introduce breaking changes, but if data structures change, the major version will increment. > v6 is the current version of EACL. Releases are not tagged yet, so pin the Git SHA. +### Breaking behavior changes (2026-07, audit root-cause fixes) + +All of these convert silent failures into correct behavior or loud, typed errors. Storage and token formats are unchanged; valid configurations and schemas work unmodified. Full details in [docs/reports/2026-07-06-eacl-full-source-audit.md](docs/reports/2026-07-06-eacl-full-source-audit.md). + +- `write-schema!` now **throws** on unparseable schema strings (previously a parse failure silently retracted the entire stored schema), on duplicate `definition`/relation declarations, and when replacing a non-empty schema with zero definitions (opt out with `{:allow-empty-schema? true}`). `//` and `/* */` comments are now supported. +- Arrow targets are validated against **all** subject types of the source relation (previously order-dependent: only the last-declared type was checked). +- Reads with unknown object IDs return **empty results** (previously `read-relationships` returned *all* relationships — a data leak — and lookups threw `AssertionError`s); writes throw `{:type :eacl/unknown-object}`. +- `make-client` throws `{:type :eacl/invalid-config}` on unknown option keys (previously silently ignored, so a typo'd ID-coercion config silently fell back to `:eacl/id`). +- Expired/corrupt cursor tokens throw `{:type :eacl/invalid-cursor}` (previously decoded to nil and silently restarted pagination at page one). Tokens no longer expire by default; opt in with `:cursor-ttl-seconds`. Cursors detect mid-pagination schema changes with `{:type :eacl/stale-cursor}`. +- `impl/tx-relationship` requires `{:allow-tempids? true}` to treat unresolvable string IDs as tempids (previously a typo'd ID silently created a ghost entity). +- Dead v6-era namespaces were removed (`eacl.datomic.rules*`, `eacl.datomic.impl.datalog`, and `eacl.datomic.impl.base/Relationship`, which emitted attributes absent from the v7 schema). + ## ReBAC: Relationship-based Access Control In a [ReBAC](https://en.wikipedia.org/wiki/Relationship-based_access_control) system like EACL, objects (_Subjects_ & _Resources_) are related via _Relationships_. @@ -112,6 +127,7 @@ The `IAuthorization` protocol in [src/eacl/core.clj](src/eacl/core.clj) defines ### Queries - `(eacl/can? acl subject permission resource) => true | false` +- Query maps may include `:max-depth`, which defaults to `50` for recursive permission evaluation. - `(eacl/lookup-subjects acl filters) => {:data [subjects...], cursor 'next-cursor}` - `(eacl/lookup-resources acl filters) => {:data [resources...], :cursor 'next-cursor}`. - `(eacl/count-resources acl filters) => {:keys [count limit cursor]}` supports limit & cursor for iterative counting. Use sparingly with `:limit -1` for all results. @@ -171,7 +187,11 @@ To query the next page, simply pass the `cursor` from page1 into the next query: {:type :server :id "server-5"}]} ``` -The return order of resources from `lookup-resources` is stable and sorted by internal resource ID. +The return order of `lookup-resources` is stable for a fixed DB basis and cursor. + +- Non-recursive lookups are typically returned in internal resource ID order because that is the order of the terminal tuple indices. +- Recursive lookups are returned in stable deterministic discovery order with exact deduplication across pages. +- Recursive queries can supply `:max-depth`; exceeding that depth raises a typed runtime error instead of silently truncating results. ## Quickstart @@ -443,6 +463,15 @@ The default options are to use the built-in EACL string attr `:eacl/id`, but you :object-id->ident (fn [obj-id] obj-id)})) ``` +`make-client` validates its options: unknown keys throw `{:type :eacl/invalid-config}` instead of being silently ignored (a silently dropped ID-coercion key means silently wrong external IDs). `:entity->object-id` (`(fn [entity] id)`) remains supported as a deprecated alias for `:entid->object-id`; supplying both throws. You can also pass `:cursor-ttl-seconds` to give pagination cursor tokens an expiry — by default tokens never expire, and an expired or corrupt token throws `{:type :eacl/invalid-cursor}` rather than silently restarting from page one. + +### Unknown object IDs + +EACL follows SpiceDB semantics for object IDs that don't resolve to an entity: + +- **Reads** (`can?`, `lookup-resources`, `lookup-subjects`, `count-resources`, `read-relationships`) treat unknown IDs as matching nothing: `can?` returns `false`, lookups return empty pages, `read-relationships` returns `[]`. +- **Writes** (`write-relationships!` and friends) throw `ex-info {:type :eacl/unknown-object, :object {:type … :id …}}` — a relationship to a nonexistent entity is unsatisfiable, and failing loudly beats minting ghost entities or raw Datomic errors. + ## Schema Syntax EACL uses the SpiceDB schema DSL. Use `eacl/write-schema!` to define your schema: @@ -466,6 +495,8 @@ EACL uses the SpiceDB schema DSL. Use `eacl/write-schema!` to define your schema For advanced use cases, you can also define schema programmatically using the internal `Relation` and `Permission` functions: +> **Cache caveat:** transacting `Relation`/`Permission` datoms directly bypasses `write-schema!`'s cache invalidation (see the caching note above). After a programmatic schema change, call `(eacl.datomic.impl.indexed/evict-permission-paths-cache!)` on every peer — or prefer `write-schema!`, which handles this for you. + ```clojure (require '[eacl.datomic.impl :refer [Relation Permission]]) @@ -515,24 +546,24 @@ This schema defines: - `server` resources belong to an `account` and can have `shared_admin` users, with `reboot` permission granted to account admins and shared_admins ``` -Now you can transact relationships: +Now you can transact relationships. The usual way is `eacl/create-relationships!` against existing entities (see Quickstart). To create entities and relationships **in the same transaction**, use `eacl.datomic.impl/tx-relationship` with `{:allow-tempids? true}` — tempid pass-through is opt-in because a typo'd ID would otherwise silently create a ghost entity: ```clojure -@(d/transact conn - [{:db/id "platform-tempid" - :eacl/id "my-platform"} - - {:db/id "user1-tempid" - :eacl/id "user1"} +(require '[eacl.datomic.impl :as impl]) - {:db/id "account1-tempid" - :eacl/id "account1"} +(let [db (d/db conn)] + @(d/transact conn + (concat + [{:db/id "user1-tempid" + :eacl/id "user1"} - (Relationship "platform-tempid" :platform "account1-tempid") - (Relationship "user1-tempid" :owner "account1-tempid")]) -``` + {:db/id "account1-tempid" + :eacl/id "account1"}] -(I'm using tempids in example because entities are defined in same tx as relationships) + (impl/tx-relationship db + (impl/Relationship (spice-object :user "user1-tempid") :owner (spice-object :account "account1-tempid")) + {:allow-tempids? true})))) +``` ## Limitations, Deficiencies & Gotchas: diff --git a/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan-v2.md b/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan-v2.md new file mode 100644 index 00000000..04ef6529 --- /dev/null +++ b/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan-v2.md @@ -0,0 +1,212 @@ +# Recursive Stable Discovery Cursor Plan V2 + +Date: 2026-03-28 +Branch: `codex/recursive-stable-cursor-max-depth` from `eacl/v7` +Supersedes: [2026-03-28-recursive-stable-discovery-cursor-plan.md](./2026-03-28-recursive-stable-discovery-cursor-plan.md) +Informed by: [2026-03-28-recursive-stable-discovery-cursor-plan-critique.md](../reports/2026-03-28-recursive-stable-discovery-cursor-plan-critique.md) + +## Summary + +Implement recursive forward pagination as resumable execution over concrete recursive facts, not as full-set closure solving. + +The elegant foundational design is: + +- symbolic permission-path compilation with no schema-time rejection of valid recursive permissions +- fast static cursor-tree lookup retained for acyclic queries +- dedicated recursive forward executor for recursive `lookup-resources` and `count-resources` +- deterministic depth-first discovery order for recursive results +- exact deduplication across recursive and non-recursive branches using query-local runtime state persisted in the cursor +- hard runtime `:max-depth` guard with default `50` + +This design deliberately forbids: + +- full reachable-set materialization +- full recursive result sorting +- approximate dedupe +- server-side cursor/session state + +No backward-compatibility or migration path is required. + +## Public Contract + +- `lookup-resources`, `count-resources`, and `lookup-subjects` query maps accept optional `:max-depth`, default `50`. +- `can?` demand-map arity accepts optional `:max-depth`; convenience arities use the default. +- Recursive forward lookup order is: + - stable and deterministic for a fixed DB basis, query, and cursor + - depth-first discovery order + - not guaranteed to be global eid sort +- Recursive pagination guarantees: + - exact deduplication across all pages + - no duplicates within a page or across pages + - concatenated pages equal one large-limit query in both membership and order +- Exceeding `:max-depth` throws typed `ex-info` with `{:eacl/error :max-depth-exceeded}`. + +## Recursive Executor Design + +- Use a dedicated recursive cursor `{:v 3 ...}` for recursive forward lookup/count. +- Runtime/cursor state is exact and explicit: + - `:stack` + - a depth-first stack of pending concrete expansion frames + - `:emitted` + - exact root resource eids already returned to the caller + - `:expanded` + - exact concrete recursive facts already expanded + - `:depth-left` + - remaining recursive depth budget + - `:last` + - last emitted resource eid, for diagnostics and cursor continuity +- A pending frame represents: + - resource type / permission node + - concrete anchor resource eid already proven for that permission node + - next recursive child path index to visit + - remaining depth for that frame + - any per-stream relation cursor state needed to continue enumeration from the anchor +- Recursive order is defined precisely: + - seed top-level root results from static permission paths in path order + - emit each unseen root resource in that order + - on emission, push recursive child expansion frames onto the stack in reverse child-path order so runtime pop order matches declared child-path order + - within each child stream, Datomic relation index order is preserved + - the resulting traversal is deterministic pre-order depth-first discovery +- Exact dedupe rules: + - a root resource is returned at most once because `:emitted` is checked before emission + - a concrete recursive fact is expanded at most once because `:expanded` is checked before expansion + - duplicates across direct, recursive, and mixed paths are suppressed by exact membership checks, not adjacency + +## TDD Phases + +### 1. Baseline Capture Before Code Changes + +- Record current `eacl/v7` baseline on this branch before any implementation edits: + - existing non-recursive benchmark in `test/eacl/bench/pagination_test.clj` + - recursive depth-1000 `parent->read` ad hoc benchmark for: + - `lookup-resources limit 50` + - `count-resources limit 50` + - full multi-page traversal +- Record current relevant namespace test results so post-change regressions are attributable. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 2. Lock The New Semantics With Red Tests + +- Add failing tests in `test/eacl/datomic/impl/indexed_test.clj` for: + - recursive paginated lookup equals large-limit lookup in both order and membership + - recursive lookup has no duplicates across pages + - recursive lookup order is stable across repeated runs on the same DB basis + - duplicates across direct and recursive paths emit once + - duplicates across two recursive branches emit once + - `a1 -> a2 -> a1` terminates and returns `{a1 a2}` + - non-productive pure cycle returns empty/false + - default `:max-depth 50` fails on depth-51 chain + - explicit larger `:max-depth` succeeds +- Update tests that currently assume recursive eid order so they instead assert stable full-order equality against the large-limit query result. +- Keep non-recursive pagination tests and benchmark thresholds intact. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 3. Introduce Max-Depth Plumbing And Error Contract + +- Thread optional `:max-depth` through: + - `eacl.core` + - `eacl.datomic.core` + - `eacl.datomic.impl.indexed` +- Default public calls to `50`. +- Use one typed runtime error shape: + - `(ex-info \"EACL max depth exceeded\" {:eacl/error :max-depth-exceeded ...})` +- Apply this contract to: + - recursive `lookup-resources` + - recursive `count-resources` + - recursive `lookup-subjects` + - `can?` +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 4. Finalize Cursor V3 Before Executor Work + +- Add recursive cursor v3 support to `eacl.datomic.core` tokenization/conversion. +- Keep v2 behavior for acyclic internal callers while allowing recursive queries to return v3. +- Cursor v3 must support exact round-tripping of: + - stack frames + - emitted eids + - expanded facts + - last emitted eid + - remaining depth +- Use exact compact encoding for eid collections before tokenization. + - Preferred: sorted vectors with delta encoding inside the cursor payload. +- Add token round-trip tests for v3 in `test/eacl/spice_test.clj`. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 5. Remove Full-Set Recursive Forward Logic + +- Delete recursive forward routing and helpers that: + - detect recursion only to switch into full closure solving + - compute full recursive result sets + - sort full recursive closures before slicing +- Retain: + - symbolic permission-path compilation + - static lazy merged lookup for acyclic queries + - runtime exact-state guards already used by `can?` and reverse lookup +- Narrow the redesign to recursive forward `lookup-resources` and `count-resources`. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 6. Implement Recursive Forward Depth-First Executor + +- Add a dedicated recursive forward executor in `src/eacl/datomic/impl/indexed.clj`. +- Execution model: + - derive top-level root-result streams from static permission paths in path order + - emit unseen root results in deterministic path/index order + - when a root result is emitted, create concrete recursive expansion frames for recursive arrow-permission dependencies that originate from that permission node + - process recursive expansion with a stack for depth-first discovery + - each expansion frame lazily scans Datomic relation tuples from its concrete anchor using direct tuple-index primitives + - discovered root resources are emitted only if unseen + - discovered non-root recursive facts are expanded only if unexpanded +- Preserve exact query-scope dedupe between: + - parallel non-recursive paths + - non-recursive and recursive paths + - multiple recursive paths +- Ensure pagination stops as soon as `limit` results are emitted; no extra closure walk is permitted. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 7. Rebuild Recursive Count On The Same Executor + +- Make recursive `count-resources` consume the same recursive executor state machine as lookup. +- Counting must: + - respect `limit` + - respect `cursor` + - respect `:max-depth` + - share exact dedupe semantics with lookup +- Do not reintroduce closure materialization or full traversal when counting only the next page. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 8. Keep Reverse Lookup And CheckPermission Minimal + +- Add `:max-depth` support to `can?` and `lookup-subjects`. +- Keep their existing exact-state recursion guards unless new red tests prove a broader redesign is needed. +- Apply the same typed depth-exceeded error contract. +- Avoid expanding the recursive forward redesign into reverse traversal unless necessary. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +### 9. Benchmark, Verify, And Document + +- Re-run the full relevant test suite until all tests are green. +- Re-run before/after benchmarks and confirm: + - non-recursive multipath benchmark remains within current thresholds + - recursive `lookup-resources limit 50` and `count-resources limit 50` are materially faster than baseline + - recursive full multi-page traversal completes with no duplicates and correct full result set +- Add a committed recursive benchmark namespace that covers: + - deep chain recursion + - recursive overlap/duplicate case + - paginated traversal cost +- Update `README.md` so recursive forward lookup now promises: + - stable deterministic order + - exact dedupe across pages + - `:max-depth` support + - unchanged result set, but changed recursive order semantics +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Acceptance Criteria + +- All tests green. +- Existing non-recursive benchmark remains green. +- Recursive forward lookup/count no longer materialize the full reachable closure. +- Recursive first-page performance is materially faster than the current baseline. +- Recursive pagination returns the same result set as one large-limit query. +- Recursive pagination order is stable across repeated runs on the same DB basis. +- Duplicate resources are never emitted twice across pages. +- Real data loops terminate cleanly before or at `:max-depth`. diff --git a/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan.md b/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan.md new file mode 100644 index 00000000..bbd23bfe --- /dev/null +++ b/docs/plans/2026-03-28-recursive-stable-discovery-cursor-plan.md @@ -0,0 +1,106 @@ +# Recursive Stable Discovery Cursor Plan + +Date: 2026-03-28 +Branch: `codex/recursive-stable-cursor-max-depth` from `eacl/v7` + +## Summary + +EACL currently supports recursive permissions in `can?` and `lookup-subjects`, but recursive forward `lookup-resources` and `count-resources` were repaired with a full-set recursive solver that regressed performance. The fix should be redesigned around runtime recursion with stable deterministic discovery order, exact query-scope deduplication, and a hard `:max-depth` guard that defaults to `50`. + +The foundational redesign is: + +- Keep permission-path compilation symbolic. +- Remove compile-time cycle handling for valid recursive permissions. +- Preserve the existing fast acyclic cursor-tree path for non-recursive lookups. +- Replace recursive forward full-set solving with a runtime frontier iterator that discovers concrete streams lazily, deduplicates exactly across recursive and non-recursive branches, and serializes the recursive execution frontier into the cursor. + +No backward-compatibility or migration path is required. + +## Public Contract + +- `lookup-resources`, `count-resources`, `lookup-subjects`, and internal `count-subjects` gain optional `:max-depth`, default `50`. +- `can?` demand-map arity gains optional `:max-depth`; convenience arities use the default. +- For recursive forward lookup, ordering changes from global eid sort to stable deterministic discovery order for a fixed DB basis, query, and cursor. +- Recursive pagination guarantees exact deduplication and set correctness across pages. +- Exceeding `:max-depth` raises a typed runtime error instead of truncating results. + +## Phase 1: Baseline And TDD Harness + +- Capture baseline numbers on `eacl/v7` for: + - existing non-recursive multipath benchmark in `test/eacl/bench/pagination_test.clj` + - recursive depth-1000 `parent->read` benchmark for `lookup-resources limit 50`, `count-resources limit 50`, and full paginated traversal +- Add failing regression tests for: + - recursive paginated lookup returns the same set as a large-limit lookup + - recursive paginated lookup has no duplicates across pages + - recursive lookup order is stable across repeated runs on the same DB basis + - duplicates across direct and recursive paths are emitted once + - duplicates across two recursive branches are emitted once + - real data loop `a1 -> a2 -> a1` terminates and returns `{a1 a2}` + - depth-51 chain fails at default `:max-depth 50` + - explicit larger `:max-depth` succeeds +- Update tests that hard-code recursive eid ordering so they assert stable set equality instead. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Phase 2: Max-Depth Plumbing And Error Contract + +- Thread optional `:max-depth` through: + - `eacl.core` demand-map calls + - `eacl.datomic.core` + - `eacl.datomic.impl.indexed` +- Default all public recursive calls to `50`. +- Add one typed runtime failure shape for depth exhaustion, used consistently by forward lookup, reverse lookup, counts, and `can?`. +- Keep symbolic permission-path caching and remove any remaining schema-time cycle semantics from query execution. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Phase 3: Recursive Forward Lookup Redesign + +- Delete the current recursive forward full-set solver and routing. +- Preserve the current lazy merged static path engine for acyclic queries. +- Add a recursive frontier executor for forward lookup: + - Seed root streams in top-level permission-path order. + - Use stable deterministic discovery order for recursive results. + - Discover new concrete streams only when a concrete resource is emitted. + - Deduplicate at query scope across all active streams. + - Serialize recursive execution state into a richer cursor version. +- Use exact runtime state tracking: + - emitted resource ids for dedupe across pages + - expanded concrete recursive facts so each resource is expanded once + - remaining depth budget +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Phase 4: Recursive Count And Reverse Semantics + +- Make `count-resources` for recursive queries reuse the recursive frontier engine rather than rebuilding a full closure. +- Keep `lookup-subjects` and `can?` on runtime exact-state recursion guards and add `:max-depth`. +- Extend internal `count-subjects` to the same recursive depth/error contract. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Phase 5: Cursor Token And Docs + +- Add a new recursive cursor version capable of carrying: + - active frontier state + - per-stream progress + - dedupe state + - remaining depth budget +- Update `eacl.datomic.core` cursor conversion/tokenization for the new cursor. +- Update `README.md` and any touched docs so recursive lookup promises: + - stable deterministic order + - exact deduplication + - `:max-depth` support + - set equality across pagination +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Phase 6: Verification + +- Run all EACL tests and make them green. +- Re-run before/after benchmarks and confirm: + - non-recursive benchmark remains within current thresholds + - recursive lookup/count are materially faster than the current `eacl/v7` baseline +- Keep pagination acceptance criteria focused on stable order and full-result set equality, not exact recursive eid order. +- [ ] Continue without asking for permission or confirmation. The plan must be fully realised before reverting to the user. Do not rest. + +## Assumptions + +- Exact global eid ordering is no longer required for recursive forward lookup. +- A richer opaque cursor is acceptable if it avoids recomputation and closure materialization. +- No server-side cache, no closure index, and no eager full-resource-set sorting are allowed. diff --git a/docs/reports/2026-03-28-recursive-stable-discovery-cursor-plan-critique.md b/docs/reports/2026-03-28-recursive-stable-discovery-cursor-plan-critique.md new file mode 100644 index 00000000..00f90d5a --- /dev/null +++ b/docs/reports/2026-03-28-recursive-stable-discovery-cursor-plan-critique.md @@ -0,0 +1,126 @@ +# Critique: Recursive Stable Discovery Cursor Plan + +Date: 2026-03-28 +Plan under review: [2026-03-28-recursive-stable-discovery-cursor-plan.md](../plans/2026-03-28-recursive-stable-discovery-cursor-plan.md) + +## Summary + +The draft points in the right direction by abandoning full-set recursion and global eid sorting for recursive forward lookup, but it is not yet decision-complete enough to implement safely. The main gaps are the lack of a precise recursive execution model, insufficient cursor-state specification, and incomplete phase ordering around benchmarks, cursor token format, and result-order semantics. + +The foundational rule should be: + +- treat recursive forward lookup as resumable execution of concrete discovered facts +- keep the existing lazy merged static path engine for acyclic queries +- make recursive order and dedupe semantics explicit and testable + +## Findings + +### 1. The draft does not specify the recursive execution order precisely enough + +The plan says "stable deterministic discovery order" but does not define the order strongly enough for implementation or tests. A stable order must be derivable from the code without ambiguity. + +Recommendation: + +- Define recursive forward lookup order exactly as: + - top-level root streams seeded in static permission-path order + - within each stream, Datomic at-rest terminal-index order + - each emitted resource expands recursive child streams in declared recursive-path order + - newly discovered child streams are pushed onto a stack immediately after the emitting stream so recursive lookup uses deterministic depth-first discovery order +- State that repeated queries on the same DB basis and cursor must return the same ordered vector. + +### 2. The draft does not separate "emitted" from "expanded" state + +Recursive lookup needs at least two exact-state sets: + +- emitted root results for cross-page dedupe +- expanded concrete recursive facts so expansion work only happens once + +If these are conflated, the implementation can accidentally suppress valid work or re-expand already visited nodes. + +Recommendation: + +- Define cursor/runtime state as: + - `:emitted` for root resources already returned to the caller + - `:expanded` for concrete recursive facts already expanded + - `:stack` or `:frontier` for pending work + - `:depth-left` for remaining recursive budget + +### 3. The plan does not decide the recursive executor shape + +"Frontier executor" is still too vague. The implementer needs a concrete evaluator model. + +Recommendation: + +- Use a recursive fact stack, not a generic queue. +- Each pending frame should represent: + - permission node + - concrete anchor resource eid + - remaining recursive depth + - next static child-path index to visit + - terminal relation stream cursor state where needed +- Prefer depth-first discovery because it minimizes frontier size and cursor growth relative to breadth-first interleaving. + +### 4. The cursor-token phase is ordered too late + +Recursive lookup and count cannot be implemented safely without knowing the final cursor shape. Leaving cursor design until after recursive execution risks backtracking across multiple phases. + +Recommendation: + +- Move cursor format and tokenization design ahead of implementation. +- Make recursive cursor v3 an explicit design phase before coding the recursive executor. + +### 5. The plan does not define exact acceptance criteria for changed pagination order + +The user explicitly allows order to change, but not the set of results. The plan should lock this into tests. + +Recommendation: + +- For recursive pagination, assert: + - same ordered vector on repeated evaluation at a fixed DB basis + - concatenated pages equal one large-limit query in both order and membership + - concatenated pages contain no duplicates +- For existing non-recursive tests, keep current behavior and performance expectations. + +### 6. The benchmark phase needs stronger ordering + +The draft says benchmark before and after, but it does not treat baseline capture as a hard dependency for implementation. + +Recommendation: + +- Put baseline capture before any code edits. +- Add a committed recursive benchmark namespace after the runtime design is stable. +- Compare branch results explicitly against the recorded `eacl/v7` baseline numbers. + +### 7. The reverse-lookup phase is too broad + +The performance regression is in recursive forward lookup. Expanding reverse lookup too much increases risk. + +Recommendation: + +- Limit structural redesign to recursive forward `lookup-resources` and `count-resources`. +- Keep `can?` and `lookup-subjects` on exact-state runtime guards plus `:max-depth`, unless red tests prove additional redesign is required. + +### 8. The plan should explicitly reject full-closure and approximate dedupe approaches + +The user has set hard architectural constraints. The plan should codify them as non-goals. + +Recommendation: + +- State explicitly that the implementation must not: + - materialize a full reachable resource set + - sort a full recursive closure + - use approximate dedupe + - use server-side cursor state + +## Upgrades Required + +1. Define recursive order precisely as deterministic depth-first discovery order. +2. Specify recursive runtime state as `:stack`, `:emitted`, `:expanded`, and `:depth-left`. +3. Move cursor v3 design before executor implementation. +4. Narrow the redesign to recursive forward lookup/count and minimal depth plumbing for the rest. +5. Make before/after benchmark capture an explicit first phase. +6. Add non-goals that forbid closure materialization, full recursive sorting, approximate dedupe, and server-side cursor state. + +## Conclusion + +The draft becomes implementation-safe once it treats recursive lookup as a resumable depth-first traversal over concrete recursive facts, with exact emitted/expanded state persisted in the cursor. That is the elegant design that would have emerged if recursive stable pagination had been a foundational assumption from the start: static cursor-tree for acyclic queries, explicit recursive execution state for recursive queries, and no full closure anywhere. diff --git a/docs/reports/2026-07-06-eacl-full-source-audit.md b/docs/reports/2026-07-06-eacl-full-source-audit.md new file mode 100644 index 00000000..438f3945 --- /dev/null +++ b/docs/reports/2026-07-06-eacl-full-source-audit.md @@ -0,0 +1,317 @@ +# EACL Full Source Audit — Bugs & Recommendations + +- **Date:** 2026-07-06 +- **Branch:** `codex/recursive-stable-cursor-max-depth` (HEAD `8af4e95`) +- **Scope:** All of `src/` (core, datomic impl, indexed engine, schema, parser, lazy merge-sort), all of `test/`, README, and stray root files. +- **Method:** Full source read, then empirical verification of every suspected bug against a live nREPL (in-memory Datomic, v7 schema). Findings below are marked **VERIFIED** (reproduced in the REPL) or **CODE-READ** (confirmed by inspection only). The full existing test suite passes (35 tests, 376 assertions, 0 failures), so none of the verified bugs are currently covered by tests. + +**Good news first:** the core engines are sound. Two differential property tests were run as part of this audit and passed: + +1. Recursive engine, 501-folder tree (`read = reader + parent->read`): `lookup-resources` set == `can?`-derived ground truth == paginated collection (limit 7) == `count-resources`, with zero duplicates. +2. Non-recursive multi-path arrow (`admin = account->admin + shared`, 40 servers interleaved across 2 accounts + 2 direct grants): full enumeration == paginated collection at limits 1/3/7, sorted, no duplicates. + +The bugs cluster at the *edges*: schema parsing/writing, keyword collation in one index scan, cache invalidation, ID-configuration plumbing, and error handling. + +--- + +## Severity index + +| # | Severity | Finding | Status | +|---|----------|---------|--------| +| 1 | **Critical** | `write-schema!` silently **deletes the entire schema** when the schema string fails to parse (incl. schemas containing `//` comments) | VERIFIED | +| 2 | **Critical** | `relation-datoms` `:a`–`:z` index range makes relations with certain subject-type keywords **invisible to permission evaluation** | VERIFIED | +| 3 | High | Permission-path/plan caches keyed by `(.id db)` are never invalidated by data — **revoked permissions keep granting access** (multi-peer, programmatic schema changes, `as-of` views) | VERIFIED | +| 4 | High | `read-relationships` with a **nonexistent** `:subject/id`/`:resource/id` returns **all** relationships (filter degrades to global scan) | VERIFIED | +| 5 | High | `make-client` silently **ignores the README-documented `:entid->object-id` option** (actual key: `:entity->object-id`) | VERIFIED | +| 6 | High | v3 recursive cursors grow **unboundedly** (~48 bytes/emitted resource) and leak raw Datomic eids to clients | VERIFIED | +| 7 | Medium | Expired/garbage cursor tokens decode to `nil` → pagination **silently restarts at page 1** | VERIFIED | +| 8 | Medium | Parenthesized permission expressions (valid SpiceDB) crash with bare `AssertionError` | VERIFIED | +| 9 | Medium | Duplicate `definition` blocks / duplicate relations silently last-win (first block dropped → destructive deltas) | VERIFIED | +| 10 | Medium | Arrow-target validation for multi-subject-type relations is **declaration-order-dependent** | VERIFIED | +| 11 | Medium | `create-relationships!` with nonexistent subject/resource throws raw Datomic `not-an-entity` error | VERIFIED | +| 12 | Medium | `impl/tx-relationship` silently creates **ghost entities** for unresolvable string IDs | VERIFIED | +| 13 | Medium | `write-relationship!` / `delete-relationship!` protocol methods unimplemented → `AbstractMethodError` | VERIFIED | +| 14 | Low | `eacl.datomic.parser_test` namespace (underscore) is never run by `clj -X:test` | CODE-READ | +| 15 | Low | README quickstart “transact relationships” example cannot work against the v7 schema | VERIFIED | +| 16 | Low | Assorted: vacuous assert, assertion-based validation, dead namespaces, doc/impl gaps, test typos | CODE-READ | + +--- + +## 1. [Critical] `write-schema!` silently wipes the schema on parse failure — VERIFIED + +**Where:** +- [parser.clj:81](../../src/eacl/spicedb/parser.clj) `parse-schema` returns the instaparse *failure object* on bad input; nothing ever checks `insta/failure?`. +- [parser.clj:167-171](../../src/eacl/spicedb/parser.clj) `transform-schema` returns `nil` for a non-vector parse tree, so `->eacl-schema` produces `{:relations [] :permissions []}`. +- [schema.clj:369-409](../../src/eacl/datomic/schema.clj) `write-schema!` then computes deltas of *existing schema vs empty schema* → retracts **everything**. + +**Repro (REPL-verified):** + +```clojure +(schema/write-schema! conn "definition user {} + definition account { + relation owner: user + permission admin = owner") ; <- missing closing brace +;; => returns deltas retracting ALL relations & permissions. No exception. +;; read-schema afterwards: {:relations [] :permissions []} +``` + +Two aggravating factors, both verified: + +- **`//` comments are not supported by the grammar.** A schema pasted from the SpiceDB playground (which emits comments) fails to parse and triggers exactly this path: `(parser/->eacl-schema (parser/parse-schema "// comment\ndefinition user {}"))` ⇒ `{:relations [] :permissions []}` — silently. +- `collect-parse-tree-issues` walks the failure object with `postwalk` and finds nothing (MapEntries don’t match any case), so validation does not throw either. + +If relationships exist, the orphan check throws a *misleading* “Cannot delete relation …” error. If none exist (fresh environments, staging, tests, or types without relationships yet), the schema is destroyed and the malformed string is stored as `:eacl/schema-string`. + +**Recommendations:** +1. In `parse-schema` (or at the top of `->eacl-schema`): `(when (insta/failure? parse-tree) (throw (ex-info "Schema parse error" {:failure (insta/get-failure parse-tree)})))`. This is a two-line fix that eliminates the data-loss path. +2. Make `transform-schema` throw rather than return `nil` for unexpected input (defense in depth). +3. Add `//` and `/* */` comment support. With instaparse this is cleanest via a custom whitespace parser passed to `:auto-whitespace` (whitespace-or-comments idiom), so comments are legal anywhere whitespace is. +4. Belt-and-braces: have `write-schema!` refuse to proceed (or require an explicit `:allow-full-retraction? true` opt) when the *new* schema parses to zero definitions while the existing schema is non-empty. A one-character typo should never be able to empty the schema. +5. Add tests: parse-failure throws; comment-bearing schema round-trips; malformed schema leaves DB untouched. + +## 2. [Critical] `relation-datoms` `:a`–`:z` range breaks permissions for legal type names — VERIFIED + +**Where:** [indexed.clj:53-63](../../src/eacl/datomic/impl/indexed.clj) + +```clojure +(let [start-tuple [resource-type relation-name :a] + end-tuple [resource-type relation-name :z]] + (d/index-range db :eacl.relation/resource-type+relation-name+subject-type start-tuple end-tuple)) +``` + +The scan is bounded by *subject-type* keywords `:a` and `:z`. Any relation whose **subject type** sorts outside that window is invisible to `calc-permission-paths` / `resolve-self-relation` / `find-relation-def` — and therefore to `can?`, `lookup-resources`, `lookup-subjects`, and the recursive planner. The relationship data writes fine; evaluation just never sees the schema edge. Failure is silent (a `log/warn "Missing Relation definition"`). + +**Repro (REPL-verified):** + +```clojure +@(d/transact conn [(impl/Relation :zone :owner :zebra) + (impl/Permission :zone :admin {:relation :owner})]) +;; relationship written and visible in the index: +;; :relationship-tuple-exists 1 +(idx/can? db (spice-object :zebra [:eacl/id "zebra-1"]) :admin (spice-object :zone [:eacl/id "zone-1"])) +;; => false (should be true) +``` + +Empirically missed subject types: `:zebra` (sorts after `:z`), `:Admin` (uppercase sorts before `:a`), `:my.app/user` (namespaced keywords sort outside the plain-keyword window entirely). `:z` itself is also excluded (exclusive end). Only plain lowercase keywords strictly inside the window work. Nothing validates or warns about this at schema-write time. + +**Recommended fix (verified in REPL):** replace the bounded range with a prefix scan + attribute/prefix guard — the same pattern `subject->resources` already uses: + +```clojure +(defn relation-datoms [db resource-type relation-name] + (if (and resource-type relation-name) + (let [attr-eid (d/entid db :eacl.relation/resource-type+relation-name+subject-type)] + (->> (d/seek-datoms db :avet :eacl.relation/resource-type+relation-name+subject-type + [resource-type relation-name]) + (take-while (fn [d] + (and (= attr-eid (:a d)) + (= resource-type (nth (:v d) 0)) + (= relation-name (nth (:v d) 1))))))) + [])) +``` + +This returned correct datoms for `:zebra`, `:Admin`, `:my.app/user`, and `:user` in testing. Note the `(= attr-eid (:a d))` guard is required — `seek-datoms` iterates past the end of the attribute’s segment. + +**Also add:** a generative/table test that round-trips schema + check across adversarial type names (`:zebra`, `:Zoo`, `:z`, `:a`, namespaced, unicode). + +## 3. [High] Permission-path & plan caches never invalidate on schema change — VERIFIED + +**Where:** [indexed.clj:101-117](../../src/eacl/datomic/impl/indexed.clj). Both `permission-paths-cache` and `recursive-query-plan-cache` key on `[(.id db) resource-type permission-name]`. `(.id db)` is the *database UUID* — verified identical across transactions. So a cache entry is valid forever unless explicitly evicted. + +`write-schema!` calls `evict-permission-paths-cache!` ([schema.clj:408](../../src/eacl/datomic/schema.clj)), which covers exactly one case: schema written via `write-schema!` *in the same JVM*. Not covered: + +1. **Programmatic schema changes** — the README-documented “Advanced: Programmatic Schema” path (`d/transact` of `Relation`/`Permission` maps). Verified: after retracting a permission entity, the cache still returns the old path — i.e. **a revoked permission continues to grant access** until process restart or LRU pressure: + + ```clojure + ;; after retractEntity of the :admin permission: + {:same-id? true, :paths-after-retraction-via-cache 1, :paths-after-retraction-fresh 0} + ``` +2. **Multiple peers** — the README explicitly recommends scaling Datomic peers horizontally. `write-schema!` on peer A evicts A’s atom; peers B…N serve stale authorization *indefinitely*. +3. **`d/as-of` / `d/history` views** — same `.id`, so a time-travel db resolves *current* cached paths instead of the schema as of that basis. + +**Recommendations (in order of preference):** +1. **Peer-safe invalidation via `tx-report-queue`:** ship a small listener (the repo already has the pattern in [test/eacl/report_queue.clj](../../test/eacl/report_queue.clj)) that evicts both caches whenever a transaction touches any `:eacl.relation/*` or `:eacl.permission/*` attribute. This covers all three cases for live conns and is cheap. +2. Alternatively (or additionally), include a *schema basis* in the cache key: maintain a single well-known schema-version entity bumped by `write-schema!`, and read its latest `:tx` with one `d/datoms :eavt` call per query (cheap). This fixes `as-of` correctness too (key by the tx visible *in that db view*), but doesn’t catch programmatic writes unless they also bump the version. +3. At minimum, document loudly that programmatic schema writes require `evict-permission-paths-cache!` **on every peer**. +4. Note `evict-permission-paths-cache!` already resets both caches — keep them coupled in whatever fix lands. + +## 4. [High] `read-relationships` with a nonexistent ID returns ALL relationships — VERIFIED + +**Where:** [core.clj:101-113](../../src/eacl/datomic/core.clj) `spiceomic-read-relationships` resolves `:subject/id` / `:resource/id` via `object-id->entid`, which yields `nil` for an unknown ID, and then `assoc`es that `nil` into the filters. [impl.clj:221-248](../../src/eacl/datomic/impl.clj) `read-relationships` treats a `nil` id as “no filter” (its own missing-id `throw` is unreachable through this wrapper because the wrapper already nil’ed the key), so the query falls through to `scan-global-relationships` and `relationship-matches-filters?` matches everything. + +**Repro (REPL-verified):** + +```clojure +(eacl/read-relationships client {:resource/type :account :subject/id "i-do-not-exist"}) +;; => ALL account relationships (alice's AND bob's) — should be [] or an error +``` + +In any multi-tenant context where a caller-supplied ID reaches `read-relationships`, this is a data leak; it can also feed bulk-delete flows (`read → delete-relationships!`) with the wrong set. + +**Recommendation:** in `spiceomic-read-relationships`, when an ID filter is present but resolves to `nil`, either throw `ex-info` (matching the intent of the impl-level guards) or return `[]`. Pick one and test it. Same check for `:resource/id`. + +## 5. [High] `make-client` ignores the documented `:entid->object-id` option — VERIFIED + +**Where:** [core.clj:281-309](../../src/eacl/datomic/core.clj) destructures `entity->object-id` (entity → id), but the README (“EACL ID Configuration”, both examples) documents `entid->object-id` (db, eid → id). Options maps are not validated, so the documented key is silently dropped and the default `:eacl/id` mapping is used. + +**Repro (REPL-verified):** configuring `{:entid->object-id (fn [db eid] (str "EXT-" ...))}` per the README returns un-prefixed IDs; the README’s “identity functions” example likewise silently returns `:eacl/id` strings instead of eids. + +**Why it matters:** ID mapping is exactly the kind of config people set once and trust. Silent fallback to `:eacl/id` means apps using `:your/id` per the README get `nil` external IDs (or seemingly working behavior in environments where `:eacl/id` happens to exist) with no error. + +**Recommendations:** +1. Accept **both** keys (`:entid->object-id` taking precedence, adapting arities), or rename to match the README — but keep backward compatibility with `:entity->object-id` since production code uses it. +2. Validate the opts map: throw on unrecognized keys. This one assertion would have caught the drift immediately. +3. Fix the README examples to whatever the canonical key is. +4. Add a config test that exercises a *non*-`:eacl/id` attribute end-to-end (current `config_test.clj` only overrides `object-id->ident`). + +## 6. [High] v3 recursive cursors grow without bound and leak eids — VERIFIED + +**Where:** [indexed.clj:395-403, 445-470, 903-917](../../src/eacl/datomic/impl/indexed.clj). The v3 cursor **is** the whole recursion state: `:stack` (pending tasks), `:best-depth` (every discovered fact), and `:emitted` (every resource ever returned). `default-internal-cursor->spice` passes v3 through unchanged ([core.clj:60-62](../../src/eacl/datomic/core.clj)), so raw eids (stack tasks, relation eids, emitted set) go to the client and come back. + +**Repro (REPL-verified):** wide tree (root + 500 children), page size 50: cursor token = 3,058 bytes after page 1; 5,322 after page 2; 12,122 after 250 results — ~48 bytes per emitted resource. Extrapolated: ~5 MB cursor at 100k results; at the stated 10M-entity goal this is unusable. Each request must upload/download the full state, and `cursor->token`’s base64-EDN inflates it further. + +Secondary issues, same mechanism: +- **Raw eid exposure** contradicts the README’s own guidance (“internal Datomic eids should not be exposed to consumers”) and makes v3 cursors invalid after a backup/restore (eids are not stable), unlike v2 cursors which are converted to external IDs. +- The cursor embeds `:max-depth` and throws on mismatch (good), but nothing versions it against **schema changes**; a plan change mid-pagination resumes against different seeds with undefined results (see also §16.8). + +**Recommendations:** +1. Short-term: document the growth characteristic; consider capping (`:emitted` count or serialized size) and failing with a typed error advising a narrower query. +2. Medium-term options (trade-offs, pick deliberately): + - **Server-side state**: keep recursion state in a bounded cache keyed by an opaque token id; the client carries only the id. Costs: state affinity/TTL, or a shared store for multi-peer. + - **Algorithmic**: emitting in a *globally sorted* order per node would let “already emitted” be re-derived as `eid <= high-water-mark` instead of a set — that’s the direction the pre-frontier implementation took and was abandoned for performance; if revisited, the `:best-depth` map can also be dropped from the cursor. + - **Compression**: delta-encode the emitted set (sorted eids) + zstd before base64. This buys maybe 5–10× but doesn’t change the asymptotics. +3. Regardless: run v3 cursors through the same eid↔external-id coercion as v2 (`:emitted`, `:stack` task cursors, `:best-depth` keys) so cursors survive restores and don’t leak internals. + +## 7. [Medium] Expired or corrupt cursor tokens silently restart pagination — VERIFIED + +**Where:** [core.clj:28-46](../../src/eacl/datomic/core.clj). `token->cursor` returns `nil` for: expired `:t` (TTL default 300 s), missing `:t`, undecodable base64/EDN, or any string not starting with `eacl1_`. All call sites use `(some->> (token->cursor ...) ...)`, so `nil` flows into the query as “no cursor” → **page 1 again**, no error. + +**Repro (REPL-verified):** token minted with `ttl-seconds -10` (and a garbage token) both decode to `nil`; a lookup with them returns the first page. + +A batch consumer that takes >5 minutes between pages (very plausible while processing pages of 1,000) silently loops back to the start — duplicates at best, an infinite loop at worst. Note also `cursor->token`’s `ttl-seconds` option is dead code from the API’s perspective: `spiceomic-lookup-resources` etc. never pass opts, so 300 s is unconfigurable. + +**Recommendations:** +1. Distinguish “no cursor” from “bad cursor”: throw `ex-info {:type ::invalid-cursor}` (or `::expired-cursor`) when a non-nil token fails to decode or is expired. SpiceDB clients expect a FAILED_PRECONDITION-style error here. +2. Thread a `:cursor-ttl-seconds` client option through `make-client` opts → `cursor->token`; consider defaulting to no expiry (the TTL protects nothing security-critical — the token is not authenticated anyway). +3. Test: expired token throws; tampered token throws; nil cursor returns page 1. + +## 8. [Medium] Parenthesized permission expressions crash with a bare `AssertionError` — VERIFIED + +**Where:** grammar accepts `paren-expr` ([parser.clj:54-57](../../src/eacl/spicedb/parser.clj)) but `extract-base-expr-identifier` ([parser.clj:455-461](../../src/eacl/spicedb/parser.clj)) only handles identifier children, yielding `{:type :identifier :name nil}` → `resolve-component` → `{:permission nil}` → `impl/Permission`’s `{:pre [(or relation permission)]}`. + +**Repro (REPL-verified):** `permission manage = (owner + editor)` ⇒ `AssertionError: Assert failed: (or relation permission)`. + +**Recommendation:** since EACL is union-only, parens are semantically trivial — flatten them: in `transform-arrow-expr`/`flatten-expression`, recurse into `paren-expr → permission-expr` and splice the resulting components into the union. If you’d rather not support them yet, add a `:paren-expr` check to `collect-parse-tree-issues` with a clear “unsupported” message. Either way, no assertion crashes. + +## 9. [Medium] Duplicate definitions / relations silently last-win — VERIFIED + +**Where:** `extract-definitions` and `extract-relations` both pour into maps ([parser.clj:128-165](../../src/eacl/spicedb/parser.clj)), so a repeated `definition account {...}` or repeated `relation owner: ...` silently drops the earlier one. + +**Repro (REPL-verified):** two `definition account` blocks ⇒ only the second block’s relations survive. Combined with `write-schema!` delta semantics, the first block’s relations/permissions become **retractions** (or misleading orphan errors) — the same destructive family as §1. SpiceDB rejects duplicate definitions. + +**Recommendation:** detect duplicates during extraction and throw with the definition/relation name. Also consider rejecting a permission and relation sharing a name on one type (SpiceDB does; EACL’s `resolve-component` silently prefers the relation). + +## 10. [Medium] Arrow validation is declaration-order-dependent for multi-type relations — VERIFIED + +**Where:** `validate-schema-references` builds `relation-subject-types` as a plain map keyed `[res-type rel-name]` ([schema.clj:252-257](../../src/eacl/datomic/schema.clj)) — **last** declared subject type wins; the parser’s `collect-schema-info` uses the **first** type ref ([parser.clj:528-532](../../src/eacl/spicedb/parser.clj)) to classify arrow targets. Validation and resolution disagree, and both ignore the full set. + +**Repro (REPL-verified):** with `permission mgmt` defined on `user` only: + +``` +relation owner: user | group → write-schema! REJECTS (validates against :group) +relation owner: group | user → write-schema! ACCEPTS (validates against :user) +``` + +Identical semantics, opposite outcomes. + +**Recommendation:** validate arrows against **all** subject types of the source relation. Then pick a policy: +- *SpiceDB-strict:* reject unless the target exists on every subject type. +- *Match the runtime:* EACL’s evaluator unions over the intermediate types that have the target (missing ones contribute nothing) — if that’s the intended semantics, accept when ≥1 type has the target and warn for the others. +Either is defensible; order-dependence is not. Also align `resolve-component`’s relation-vs-permission classification to consult all types (mixed relation/permission targets across types should be an explicit error). + +## 11. [Medium] Writes to nonexistent subjects/resources give raw Datomic errors — VERIFIED + +**Where:** `spice-relationship->internal` ([core.clj:115-119](../../src/eacl/datomic/core.clj)) maps unknown external IDs to `nil` without checking; the nil lands in tx-data. + +**Repro (REPL-verified):** `create-relationships!` with subject `"ghost-user"` ⇒ `IllegalArgumentException :db.error/not-an-entity Unable to resolve entity: in datom [nil :eacl.v7.relationship/… ]`. + +**Recommendation:** validate both endpoints resolve in `spice-relationship->internal` and throw `ex-info` naming `{:subject {:type :user :id "ghost-user"}}`. This also covers `:touch`/`:delete`, where a nil currently makes `relationship-exists?` return false and delete silently no-op. + +## 12. [Medium] `impl/tx-relationship` silently creates ghost entities for unresolvable string IDs — VERIFIED + +**Where:** `object-id->eid-or-tempid` ([impl.clj:49-54](../../src/eacl/datomic/impl.clj)) intentionally passes unresolvable strings through as tempids (fixtures rely on this for same-transaction entity+relationship creation). + +**Repro (REPL-verified):** a typo’d resource id `"acct-1x"` transacts fine and mints a **new entity** whose only attribute is the reverse relationship tuple — no `:eacl/id`, unreachable by external ID, and the intended grant never lands on the real `"acct-1"`. No warning. + +**Recommendation:** make tempid pass-through **opt-in**, e.g. `(tx-relationship db rel {:allow-tempids? true})` used by fixtures, defaulting to throwing on unresolvable IDs. Alternatively accept explicit tempid wrappers (`(->tempid "acct-1")`) so intent is unambiguous. As-is, every caller of the advanced API is one typo away from silent permission loss + junk entities. + +## 13. [Medium] Unimplemented protocol methods throw `AbstractMethodError` — VERIFIED + +**Where:** `IAuthorization` declares `write-relationship!` and `delete-relationship!` ([eacl/core.clj:43-60](../../src/eacl/core.clj)); `Spiceomic` implements neither ([datomic/core.clj:226-279](../../src/eacl/datomic/core.clj)). Verified both arities throw `AbstractMethodError`. + +**Recommendation:** implement them (trivial delegations to `spiceomic-write-relationships!`) or drop them from the protocol. Also note `expand-permission-tree` throws a plain `Exception. "not impl."` — prefer `ex-info` with `{:type ::not-implemented}` for programmatic handling. + +## 14. [Low] Parser tests never run under `clj -X:test` — CODE-READ + +**Where:** [test/eacl/datomic/parser_test.clj:1](../../test/eacl/datomic/parser_test.clj) declares `(ns eacl.datomic.parser_test ...)` — underscore, not hyphen. The cognitect test-runner’s default include pattern is `#".*-test$"`, which `eacl.datomic.parser_test` does not match, so the whole namespace is silently excluded from `clj -X:test` runs (it runs fine when required explicitly, which is why it looks alive from the REPL). + +**Recommendation:** rename the ns to `eacl.datomic.parser-test` (file name stays `parser_test.clj`). Grep CI output for the namespace to confirm it appears afterwards. + +## 15. [Low] README quickstart relationship-transaction example is broken — VERIFIED + +**Where:** README “Now you can transact relationships:” shows `(Relationship "platform-tempid" :platform "account1-tempid")` inside a `d/transact`. + +- With `eacl.datomic.impl.base/Relationship` this emits v6 `:eacl.relationship/*` attrs, which **do not exist** in `v7-schema` — verified: `:db.error/not-an-entity Unable to resolve entity: :eacl.relationship/resource-type`. +- With `eacl.datomic.impl/Relationship` (what fixtures import) it returns an `eacl.core.Relationship` *record*, which is not transactable data either — and its `:pre` requires `{:type ... :id ...}` maps, not bare strings. + +The working pattern is `(impl/tx-relationship db (Relationship subject relation resource))` as used in fixtures. + +**Recommendations:** fix the README example to use `tx-relationship` (or `create-relationships!` on the client); delete or clearly deprecate `base/Relationship` (nothing in the live path uses it, and it can only produce broken tx-data under v7); consider making `v6-schema` alias emit a deprecation note since it now *is* v7. + +## 16. Low-severity & hygiene (code-read unless noted) + +1. **Vacuous assertion:** `spiceomic-count-resources` asserts `(= (:type subject-ent) (:type subject))` where `subject-ent` is `subject` with only `:id` updated — always true ([core.clj:188-190](../../src/eacl/datomic/core.clj)). Presumably meant to check the resolved entity’s actual type; either implement that or delete it. +2. **Assertion-based API validation:** `lookup-subjects` rejects a missing resource via `{:pre ...}` ([indexed.clj:959](../../src/eacl/datomic/impl/indexed.clj)) — verified `AssertionError: Assert failed: (:id (:resource query))` for an unknown resource ID. `lookup-resources`/`count-resources` use `assert` similarly ([core.clj:159-166,186-190](../../src/eacl/datomic/core.clj)). Asserts vanish when `*assert*` is false and produce untyped errors; use `ex-info` consistently, and decide (and test) missing-object behavior: empty result vs typed error. Currently it’s assert-crash in three shapes. +3. **Dead/misleading namespaces:** `eacl.datomic.rules` (entirely commented), `eacl.datomic.rules.optimized`, `rules/optimized_old.clj`, and `eacl.datomic.impl.datalog` all target v6 `:eacl.relationship/*` attrs that no longer exist in the installed schema — if anyone wires them up they fail at query time. `eacl.impl.spicedb` is an empty stub. Root-level `simple_test.clj`, `test_large_offset.clj`, `test_cursor_pagination.clj` are fully commented-out scripts (also untracked, per git status). Recommend deleting or moving to `docs/attic/`; they actively mislead contributors about which engine is live. +4. **`:resource/id-prefix` documented but unimplemented:** the protocol docstring for `read-relationships` advertises it ([eacl/core.clj:27-36](../../src/eacl/core.clj)); `relationship-matches-filters?` ignores it. Implement or remove from the doc. +5. **`:create` uniqueness race:** `tx-update-relationship` checks existence against the read-time db ([impl.clj:257-278](../../src/eacl/datomic/impl.clj)); two concurrent `:create`s of the same relationship both pass and both “succeed” (datom add is idempotent, so no duplicate data, but SpiceDB `CREATE` semantics say the second must fail). Enforcing this requires a transaction function; alternatively document `:create` as best-effort and recommend `:touch`. +6. **`write-schema!` orphan-check race:** relationships transacted between the orphan check and the retraction tx can be orphaned. Low likelihood; a transaction function (or re-check inside the tx) would close it. +7. **Cursor `:p` is keyed by path index:** v2 cursors store per-path intermediate positions as `{path-idx eid}` ([indexed.clj:318-326](../../src/eacl/datomic/impl/indexed.clj)). A schema change between pages reorders/renumbers paths and silently mis-skips. Cheap hardening: include a hash of the path set in the cursor and restart-or-throw on mismatch. +8. **Recursive cursor vs schema change:** same class as above for v3 (`:stack` embeds relation eids and node vectors). A plan hash in the cursor would catch it. +9. **Test typo:** [indexed_test.clj:542](../../test/eacl/datomic/impl/indexed_test.clj) uses `[:eacl/id "user2"]` (no hyphen; entity doesn’t exist) so the assertion passes vacuously — the eid resolves to `nil` and `can?` is `false` regardless. Also [indexed_test.clj:434](../../test/eacl/datomic/impl/indexed_test.clj) has a `(testing "...")` with no body; the intended assertion sits outside it. +10. **Cursor inclusivity is undocumented:** the resource-level cursor is *exclusive* (results resume after `:e`; `subject->resources` seeks from `(inc cursor)`), while the per-path intermediate cursor stored in `:p` is *inclusive* (re-scanned via `inclusive-cursor->exclusive`’s `dec`). Both are correct, but the contract lives only in the arithmetic; docstrings on `subject->resources`/`resource->subjects`/`extract-cursor-eid` would help — the `test/eacl/datomic/impl/indexed_test.clj:656` comment (“cursor seems weird here. shouldn’t it be exclusive?”) suggests it has already cost debugging time. +11. **`consistency/fresh` returns a constant:** `(defn fresh [token] :fresh)` ignores its token, and `spiceomic-can?`’s consistency check is an `assert` (see 16.2). Fine for now, but the API-compat story would be better served by `ex-info` with `{:type ::unsupported-consistency}`. + +## 17. Performance observations (not bugs, worth tracking) + +1. **`arrow-via-intermediates` is O(intermediates) seeks per page** ([indexed.clj:303-316](../../src/eacl/datomic/impl/indexed.clj)): one `subject->resources` seek per intermediate, then a pairwise-fold merge over all non-empty streams. A subject with 10k accounts pays 10k seeks before the first result of an arrow path. The `:p` cursor only skips the *prefix* of intermediates with no remaining results. If this becomes hot, a k-way heap merge plus per-intermediate lazy seeks would drop the constant factor; the README’s parallel-path caveat already gestures at this. +2. **Reverse lookups have no recursive engine:** `lookup-subjects` always uses the depth-limited recursive descent (`lookup-subject-eids*`), whose visited-set only guards the current ancestor chain — in dense permission DAGs the same (resource, permission) state can be re-explored once per distinct path (exponential worst case), and there is no cross-branch memoization. Forward got the frontier engine; reverse may eventually want the same treatment. +3. **`can*` similarly re-explores shared substructure** (no memoization across branches). Fine at current graph sizes; will matter for deep org-hierarchy schemas. +4. **`find-relations` in `read-relationships` scans all relation entities with `d/q` then filters in memory** ([impl.clj:136-154](../../src/eacl/datomic/impl.clj)) — fine (schema is sparse), just noting it contrasts with the indexed discipline elsewhere. + +## 18. Test-coverage recommendations + +The suite is strong on engine semantics (pagination equivalence, dedup, cycles, max-depth) and weak exactly where the verified bugs live: + +1. **Schema-write failure paths:** parse errors, comments, duplicate definitions, zero-definition guard (§1, §9). +2. **Adversarial type names** through the full stack (§2). +3. **Cache invalidation:** schema change → immediate effect on `can?`, including programmatic writes and (if adopted) the tx-report listener (§3). +4. **Client ID-config matrix:** custom attribute end-to-end incl. cursors and `read-relationships` (§5) — would also have caught §4. +5. **Cursor abuse:** expired, tampered, cross-query cursor reuse, v3-cursor-into-non-recursive-query and vice versa (§7, §16.7-8). +6. **Differential/property test:** codify the audit’s cross-check (`lookup-resources` set == `can?` ground truth == paginated union == `count-resources`; ditto reverse) over randomized small graphs. This is the single highest-leverage test for engine regressions. +7. Re-enable parser tests in CI (§14). + +## 19. Suggested fix order + +1. §1 parse-failure guard + zero-definition guard (hours, removes data-loss). +2. §2 `relation-datoms` prefix scan (small, verified fix included above). +3. §4 nonexistent-ID guard in `read-relationships` (small, closes leak). +4. §5 `make-client` opts validation + README correction. +5. §7 typed cursor errors + TTL config. +6. §3 cache invalidation strategy (tx-report listener). +7. §8–§13 as batched medium fixes. +8. §6 recursive-cursor design decision (needs a plan doc; interacts with SpiceDB-compat goals). +9. §16 hygiene sweep + §18 test additions alongside each fix. + +--- + +*Verification environment: nREPL on port 7910, Datomic peer 1.0.6733 in-memory, Clojure 1.12.0-alpha5. All “VERIFIED” items have exact repro snippets above; run them against `datomic:mem://` databases with `schema/v7-schema` installed.* diff --git a/openspec/changes/fix-audit-root-causes/.openspec.yaml b/openspec/changes/fix-audit-root-causes/.openspec.yaml new file mode 100644 index 00000000..dd9a1d92 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-06 diff --git a/openspec/changes/fix-audit-root-causes/design.md b/openspec/changes/fix-audit-root-causes/design.md new file mode 100644 index 00000000..c7a5e45f --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/design.md @@ -0,0 +1,178 @@ +# Design: fix-audit-root-causes + +## Context + +The 2026-07-06 audit ([docs/reports/2026-07-06-eacl-full-source-audit.md](../../../docs/reports/2026-07-06-eacl-full-source-audit.md)) verified 15 bugs with REPL repros. The traversal engines are sound (differential tests passed); every defect is root-caused in one of five edge layers: + +1. **Schema-write pipeline trust** — `parse-schema` returns instaparse failure objects nothing checks; `transform-schema` converts them to `nil`; `->eacl-schema` converts `nil` to an empty schema; `write-schema!` diffs old-vs-empty and retracts everything. Duplicate definitions last-win into the same destructive delta path. Arrow validation consults only the last-declared subject type while resolution consults the first. +2. **One bounded index scan** — `relation-datoms` ranges subject types between literal `:a` and `:z`, hiding relations whose subject-type keyword sorts outside that window (uppercase, `z*`, all namespaced keywords). +3. **Cache keying** — permission-path/plan caches key on `(.id db)` (the immutable database UUID). Eviction is a side effect of same-JVM `write-schema!` only; other peers, programmatic schema writes, and `as-of` views read stale paths. +4. **Nil-tolerant ID resolution** — unresolvable external IDs become `nil` (silently matching everything in `read-relationships`, or landing in tx-data as raw Datomic errors) or become tempids (minting ghost entities). `make-client` silently drops unknown option keys, including the README-documented one. +5. **Nil-on-failure cursor decoding** — expired/undecodable tokens decode to `nil`, indistinguishable from "first page". + +Constraints: EACL is in production at CloudAfrica and in the author's side projects; Datomic peer 1.0.6733; no new runtime dependencies; wire/storage formats (v7 tuples, `eacl1_` token prefix, schema entities) must remain readable by existing deployments; repo convention is nREPL-driven testing. + +## Goals / Non-Goals + +**Goals:** + +- Convert every verified silent-failure into either correct behavior or a loud, typed error. +- Fix root causes: a strict parse→validate→diff pipeline; prefix scans instead of keyword-bounded ranges; a schema-basis-aware cache key; a strict ID-resolution boundary with one coherent unknown-ID contract; fail-loud cursor decoding. +- Keep all existing *valid* configurations, schemas, cursors-in-flight (raw maps), and data working unchanged. +- Leave the codebase honest: dead v6 namespaces deleted, README matching reality, all tests actually running in CI. + +**Non-Goals:** + +- The v3 recursive-cursor growth/eid-leak redesign (audit §6). Architectural; follow-up change. This change only adds a cursor↔schema fingerprint guard and documentation. +- `:create` uniqueness under concurrency (audit §16.5) — requires a transaction function; deferred, documented as best-effort. +- New SpiceDB features (subject relations, intersection/exclusion, multi-level arrows, caveats). +- Performance work (`arrow-via-intermediates` fan-out, reverse-lookup memoization) — tracked in the audit, not touched here beyond not regressing. + +## Decisions + +### D1. Parse pipeline fails loudly at one choke point + +`->eacl-schema` becomes the choke point: it throws `ex-info` with `{:type :eacl.schema/parse-error :failure (insta/get-failure tree)}` when handed a failure object, and `transform-schema` throws (never returns `nil`) on unexpected shapes. `parse-schema` keeps returning instaparse results unchanged (REPL ergonomics, existing tests). + +*Alternative considered:* throwing inside `parse-schema`. Rejected: failure objects are useful interactively, and `->eacl-schema` is the single path `write-schema!` uses — one guard covers all writers. + +### D2. Comments become whitespace in the grammar + +Support `//` line and `/* */` block comments by supplying a custom whitespace parser to instaparse's `:auto-whitespace` (the documented whitespace-or-comments idiom). Comments are then legal anywhere whitespace is, matching SpiceDB. + +*Alternative considered:* regex-stripping comments before parsing. Rejected: fragile against future string/caveat syntax and reports wrong error positions. + +### D3. Duplicates are rejected during extraction + +`extract-definitions` and `extract-relations` currently pour into maps (last-wins). Both are changed to detect collisions and throw `ex-info` naming the duplicate (`{:type :eacl.schema/duplicate-definition}` / `::duplicate-relation`). Additionally, a permission sharing a name with a relation on the same type is rejected (SpiceDB does the same; today `resolve-component` silently prefers the relation). + +### D4. Full-retraction guard in `write-schema!` + +If the *new* schema parses to zero definitions while the stored schema is non-empty, `write-schema!` throws `{:type :eacl.schema/empty-schema-guard}`. Escape hatch: `(schema/write-schema! conn schema-string {:allow-empty-schema? true})` — a new optional opts arity on the schema-namespace fn only; the `IAuthorization` protocol method signature is unchanged and always guarded. With D1–D3 this guard should never fire from well-formed input; it is belt-and-braces against future parser gaps. + +### D5. Arrow validation checks all subject types, strictly + +`validate-schema-references` builds `relation-subject-types` as `{[res-type rel-name] #{subject-types}}` (a set, not last-wins). An arrow `rel->target` is valid only if **every** subject type of `rel` has `target`, and `target` resolves to the same kind (relation vs permission) on all of them; errors enumerate the offending types. + +*Why strict (SpiceDB semantics) over permissive (match the runtime's union-over-types-that-have-it):* the project's stated goal is a clean migration path to SpiceDB — schemas EACL accepts must be accepted by SpiceDB. The runtime's silent no-op for missing types was never a documented feature. Risk of rejecting existing production schemas is noted in Risks; the error message tells the author exactly which type/target to add. + +`resolve-component`'s first-type classification becomes safe under D5 (all types agree on kind), but is still rewritten to consult the full set so parser and validator can never diverge again. + +### D6. Parenthesized unions flatten; parenthesized arrow bases are rejected + +Since EACL is union-only, `(a + b)` as a union operand is semantically just `a + b`: `flatten-expression` recurses into `paren-expr → permission-expr` and splices the components. A `paren-expr` appearing as an arrow base or target (`(a + b)->c`) is rejected with a clear validation issue (SpiceDB arrows take relation bases; today this crashes with a bare `AssertionError`). + +### D7. `relation-datoms` becomes a prefix scan (fix verified in the audit) + +```clojure +(let [attr-eid (d/entid db :eacl.relation/resource-type+relation-name+subject-type)] + (->> (d/seek-datoms db :avet :eacl.relation/resource-type+relation-name+subject-type + [resource-type relation-name]) + (take-while (fn [d] (and (= attr-eid (:a d)) + (= resource-type (nth (:v d) 0)) + (= relation-name (nth (:v d) 1))))))) +``` + +The `(= attr-eid (:a d))` guard is mandatory — `seek-datoms` iterates past the attribute's segment (verified during the audit; without it the scan crashes on the next attribute's non-tuple values). This is the same pattern `subject->resources` already uses. Audited the other bounded ranges (`scan-global-relationships`, `count-relationships-using-relation`): both bound only the trailing *eid* component under an exact keyword prefix — correct as-is. + +### D8. Cache keys carry a schema-history digest; invalidation is derived, not signaled + +> **SUPERSEDED (2026-07-14, issue #74).** The derived digest was reverted: computing it requires a schema-history scan per fresh db basis, i.e. after **every** `d/transact` — unrelated relationship writes paid for schema-change detection, which does not scale under write load. The shipped design is a signaled stamp: `write-schema!` asserts a fresh `:eacl/schema-version` **squuid** on the schema singleton in the same transaction as any definition change, and both caches key on `[(.id db) version …]` (one AVET lookup, no scans, untouched by unrelated transactions). Of the six loopholes below that motivated the digest: the squuid (not a counter) closes the counter-elision race; nothing memoizes eids, so there is no anchor-eid hazard; no `t` arithmetic remains (as-of views read their era's version datom natively); positive view classification and failure-degrades-to-miss are retained verbatim. The one consciously **accepted** loophole is the unenforceable-contract one: programmatic relation/permission datom edits (raw `d/transact`, `d/with`, excision) no longer invalidate anything and may be served stale paths until the next `write-schema!` or manual `evict-permission-paths-cache!` — per issue #74, users must not manage EACL schema outside the API. Cursor fingerprints keep the same two-part `{:s :p}` shape with `:s` = the version string. The digest text below is retained as the record of why the derived approach was tried. + +Both caches key on `[(.id db) (schema-basis db) …]` where `(schema-basis db)` is a 128-bit digest (SHA-256 truncated to 128 bits — collision resistance, not security; SHA-256 rather than MD5 so FIPS-mode JVMs don't silently lose caching to the failure fallback) folded, in index order, over the **history datoms** of the two unique composite tuple attributes — `:eacl.relation/resource-type+relation-name+subject-type` and `:eacl.permission/resource-type+source-relation-name+target-type+target-name+permission-name` — each datom contributing `[e v t added?]`, filtered to `t ≤ (or (d/as-of-t db) (d/basis-t db))`. + +An earlier draft of this decision used an anchor entity (`[:eacl/id "schema-string"]` max-`:tx`) plus a `:eacl/schema-version` counter and touch helpers for programmatic writers. An adversarial review found six loopholes in that shape; each clause of the digest design closes one: + +- **Derived, not signaled** — any transaction touching any component of any relation/permission entity rewrites that entity's composite tuple *in the same transaction*, and `retractEntity` retracts it (both REPL-verified). The tuple histories are therefore a complete record of path-relevant schema mutations: programmatic writers are covered automatically, and there is no touch contract to forget (the unenforceable-contract loophole). +- **History, not current datoms** — a pure retraction leaves no current datom but does leave history; max-over-current-datoms designs miss revocations entirely. +- **Content digest, not a max-`:tx`/counter** — immune to the counter-elision race (two peers asserting the same incremented value produce no datom for the second, silently skipping invalidation); immune to `d/excise` *lowering* a max and resurrecting ancient entries (excised history changes the digest, which is a correct invalidation); immune to speculative `d/with` dbs colliding with a future real transaction at the same `t` (different content ⇒ different digest; identical content ⇒ identical eids and paths, so sharing is harmless). `d/history` works on with-db db-afters and includes speculative datoms (REPL-verified). +- **No anchor entity** — nothing to retract/recreate out from under a memoized eid. +- **Explicit `as-of-t` filter** — `(d/history (d/as-of db t))` is correctly filtered (REPL-verified), but `(d/basis-t (d/as-of db t))` returns the *underlying* basis, not `t` (REPL-verified, surprising) — so the filter must use `d/as-of-t` when present. Keeping the explicit filter also proofs the code against history/as-of composition differences across Datomic versions. +- **Positive view classification; unclassifiable views are never cached** — the digest is computed only for db values positively classified as *plain* (`as-of-t` nil, `since-t` nil, `is-filtered` false, `is-history` false — `d/with` db-afters classify plain, which is safe per the previous clause) or *as-of* (`as-of-t` non-nil, others clear). Everything else — `d/filter`, `d/since`, history dbs, or anything unrecognized — gets a unique sentinel key: computed fresh from that view, never shared. `d/filter` views are excluded because predicates are arbitrary functions (possibly impure or time-dependent), so a digest of their filtered history cannot be trusted as a shared key. *Empirical correction during implementation:* an earlier audit pass concluded `d/history` on a `d/filter` db ignores the filter — that came from a vacuous all-pass predicate (both behaviors coincide for it); a hide-everything predicate shows history **respects** the filter, which is now the pinned fact. The classification rule stands on the arbitrary-predicate rationale. (`d/since` history is since-filtered — verified — but since-views are excluded anyway: they hide old schema *and* old relationships, so caching them buys nothing.) +- **Failure degrades to a miss, never a stale hit** — any exception during digest computation (exotic db types, `.id` access failing after a peer upgrade, history unsupported somewhere) likewise yields a fresh unique sentinel key → guaranteed cache miss → paths recomputed from the actual db value passed. There is no failure path — thrown or classified — that serves a stale entry. + +**Memoization:** the digest is memoized in a synchronized `WeakHashMap` keyed by the db value itself — one history scan per `(d/db conn)` call rather than per permission check, and weak keys release with the db value. `WeakHashMap` uses `.equals`, and Datomic `Db` equality is value-based *and content-aware* (REPL-verified: two `d/db` calls at the same basis are equal; two `d/with` db-afters at the same `t` with different speculative content are **not** equal) — so the memo can only unify db values Datomic itself declares equal, which by the verified semantics implies identical visible history and therefore identical digests. Only positively classified views are memoized; sentinel results are never stored. If a future peer version loosened `Db.equals` to ignore speculative content, aliasing would be confined to speculative-vs-speculative views at the same `t` (a real db at that `t` compares unequal to a with-db by content) — and the pinned equality tests in task 5.3 would fail loudly first. Cost: O(all-time schema edits) per db value — tens of microseconds against hot peer segments for realistic schema churn. + +**Coverage invariant (documented at the definition site):** every attribute that path/plan computation reads must be a component of one of the digested composite tuples; introducing a new path-relevant attribute outside them requires adding its history to the digest. Today the paths read exactly relation `{resource-type, relation-name, subject-type, eid}` and permission `{resource-type, permission-name, source-relation-name, target-type, target-name}` — all tuple components (relation eids change only via retract+recreate, which the tuple history records). + +`evict-permission-paths-cache!` stays public as a manual override and `write-schema!` keeps calling it (immediate local effect; harmless). The `:eacl/schema-version` attribute and touch helpers from the earlier draft are **dropped** — no storage additions, no writer contract. + +*Alternatives considered:* +- **Anchor entity + version counter + touch helpers** (earlier draft). Rejected for the six loopholes above — chiefly that a documented contract for programmatic writers is exactly the class of silent failure this change exists to eliminate. +- **`tx-report-queue` listener** evicting on schema-attr datoms. Rejected: `d/tx-report-queue` returns *the* queue for a connection — EACL consuming it would steal reports from applications that already use it (CloudAfrica does; see `test/eacl/report_queue.clj`), and it adds thread lifecycle to `make-client`. +- **Key by `basis-t`**: correct but evicts on every relationship write, making the cache useless under write load; also aliases speculative `d/with` dbs against future real bases. +- **Status quo + documentation**: leaves the verified stale-grant hazard in multi-peer production. + +*Verified against Datomic peer 1.0.6733 (REPL, 2026-07-06):* component-edit rewrites the composite tuple same-tx; `retractEntity` retracts the tuple into history; `(d/history (d/as-of db t))` filters to ≤ t; `(d/basis-t (d/as-of db t))` = underlying basis (hence the `as-of-t` filter); `d/history` works on `d/with` db-afters; `.id` returns the same UUID on plain/as-of/with dbs; writes to unallocated numeric eids are rejected by the transactor; `d/is-filtered` is true only for `d/filter` dbs (false for plain/as-of/since/with); `as-of-t`/`since-t`/`is-history` positively identify their views (with-dbs read as plain); `d/history` on a `d/filter` db respects the filter (pinned with a hide-everything predicate — an earlier all-pass-predicate check was vacuous); `d/history` on a `d/since` db is since-filtered. Task 5.3 pins each of these as a regression test so a peer upgrade that changes any of them fails loudly. + +### D9. One unknown-ID contract: reads empty, writes throw + +Aligned with SpiceDB (object IDs are opaque there; unknown IDs simply match nothing) and with `can?`'s existing `false`: + +| Operation | Unknown subject/resource ID today | New behavior | +|---|---|---| +| `can?` | `false` | `false` (unchanged) | +| `read-relationships` | **all relationships** (leak) | `[]` | +| `lookup-resources` / `count-resources` | `AssertionError` | empty page / `{:count 0}` | +| `lookup-subjects` | `AssertionError` | empty page | +| `write-relationships!` (create/touch/delete) | raw Datomic `not-an-entity` / silent no-op | `ex-info {:type :eacl/unknown-object, :object {:type … :id …}}` | + +Implementation: `spice-object->internal` and the `read-relationships` filter resolution return a sentinel distinguishing "no filter supplied" from "supplied but unresolvable"; unresolvable reads short-circuit to empty, unresolvable writes throw before tx-data is built. Writes throw (rather than SpiceDB's accept-any-string) because EACL relationships are eid-based — a write that cannot resolve is unsatisfiable, and today's silent tempid/no-op variants are the audit's §11/§12. + +The write-side check must verify entity **existence** (`(seq (d/datoms db :eavt eid))`), not mere `d/entid` resolution: `d/entid` passes numeric inputs through unchanged, so a plausible-but-unallocated eid "resolves" — the transactor then rejects it with a raw `:db.error/invalid-entity-id` (REPL-verified). The existence check turns that into the same typed `:eacl/unknown-object` error. Read paths need no extra check — seeks on nonexistent eids naturally yield empty results, consistent with the contract. + +### D10. `impl/tx-relationship` tempids become opt-in + +`(tx-relationship db rel)` throws `:eacl/unknown-object` for unresolvable string IDs; `(tx-relationship db rel {:allow-tempids? true})` restores tempid pass-through for same-transaction entity+relationship creation. Fixtures and the two tests that exploit tempids pass the flag. `resolve-relationship`/`object-id->eid-or-tempid` thread the option down. + +*Alternative considered:* an explicit `(->tempid "x")` wrapper type. Rejected for now: heavier API surface; the boolean opt covers the two real call sites (fixtures, quickstart docs) and keeps `tx-relationship` data-in/data-out. + +### D11. `make-client` validates options; canonical ID key matches the README + +- Accept `:entid->object-id` (`(fn [db eid] …)`) as the canonical key — it is what the README has documented all along. +- Keep `:entity->object-id` (`(fn [ent] …)`) working as a deprecated alias (production code uses it); supplying **both** throws. +- Throw `ex-info {:type :eacl/invalid-config, :unknown-keys […], :known-keys […]}` on any unrecognized key. This single check would have caught the original drift. + +### D12. Cursor tokens fail loudly and gain a schema fingerprint + +- `token->cursor` contract: `nil` → `nil` (no cursor); raw map → pass-through (back-compat); a non-nil string that fails to decode, fails the `eacl1_` prefix, or is expired → `ex-info {:type :eacl/invalid-cursor, :reason :expired|:undecodable}`. +- TTL: default **no expiry**. `make-client` gains `:cursor-ttl-seconds`; when set, `cursor->token` embeds `:t` and decoding enforces it (the existing `ttl-seconds` plumbing is finally connected; today it is dead code and 300 s is hardcoded). Tokens without `:t` never expire. Rationale: the TTL protects nothing security-relevant (tokens are unauthenticated data), while the 5-minute default silently corrupts every batch job slower than 300 s/page. +- Fingerprint: cursors (v2 and v3 state) carry `:f {:s :p }` — the D8 schema-history digest at mint time plus a 128-bit digest (SHA-256 truncated, as in D8) of `pr-str` of the query's resolved paths (v2) or plan (v3). On resume: equal `:s` ⇒ proceed (identical schema history ⇒ identical paths — exact, no hashing involved in the common case); differing `:s` ⇒ recompute this query's paths and compare `:p` — equal ⇒ proceed (the schema change didn't touch this query), differing ⇒ throw `{:type :eacl/stale-cursor}`. Unrelated schema changes therefore do *not* invalidate in-flight cursors, and reordered `:p` path indices / stale v3 `:stack`s are caught. Residual false-negative requires a 128-bit collision *and* a mid-pagination schema change to the same permission — negligible. Cursors lacking `:f` (minted before this change) are accepted for one release with a `log/warn`. + +### D13. Protocol completeness and typed errors in the client layer + +- Implement `write-relationship!` (both arities) and `delete-relationship!` (both arities) as delegations to `spiceomic-write-relationships!`. +- Replace client-layer `assert`s (subject existence in `lookup-resources`/`count-resources`, the `{:pre …}` in `lookup-subjects`, the consistency assert) with the D9 behaviors and `ex-info {:type :eacl/unsupported-consistency}` respectively. Asserts are compile-time-removable and untyped; the audit found three different failure shapes for the same class of input. +- Delete the vacuous `(= (:type subject-ent) (:type subject))` assert in `count-resources`. +- `expand-permission-tree` throws `ex-info {:type :eacl/not-implemented}` instead of bare `Exception`. + +### D14. Housekeeping is part of the change, not spec'd + +Deletions (`eacl.datomic.rules`, `eacl.datomic.rules.optimized`, `rules/optimized_old.clj`, `eacl.datomic.impl.datalog`, `base/Relationship`, root-level commented scripts), README corrections (ID-configuration keys per D11; quickstart relationship example rewritten around `tx-relationship`/`create-relationships!`), the `eacl.datomic.parser_test` → `eacl.datomic.parser-test` ns rename, and the two test typos ride along as tasks without spec requirements. The commented-out namespaces reference v6 attrs absent from the installed schema — they cannot work and actively mislead. + +### D15. Verification: differential property test + per-fix regression tests + +Every fix lands with the audit's repro as a regression test. Additionally, a seeded, hand-rolled randomized differential test (no new deps — `test.check` not introduced) generates small graphs and asserts the audit's invariant: `lookup-resources` set == `can?`-derived ground truth == paginated union (several page sizes) == `count-resources`, and the reverse for `lookup-subjects`. All tests runnable via nREPL per repo convention. + +## Risks / Trade-offs + +- [Strict arrow validation (D5) may reject existing production schemas that relied on silent per-type no-ops] → The error lists exactly which subject types lack the target; migration is additive (define the missing permission). Ship note in README breaking-changes section. If a real schema cannot be made SpiceDB-valid, revisit with an explicit permissive flag rather than silent behavior. +- [Reads-return-empty (D9) can mask caller typos that previously blew up with `AssertionError`] → Typos on *writes* still throw; `can?` was already `false`-on-unknown, so the contract is now uniform and documented. Apps that want existence checks should check existence, not rely on authz-layer asserts. +- [Digest scan cost grows with all-time schema-edit history] → Identity-memoized per db value (one scan per `(d/db conn)` call); O(schema edits ever) is microseconds against hot segments for realistic churn. Pathological continuous schema churn is an ops smell in its own right — documented; and `d/excise` of ancient schema history, should anyone ever need it, is handled *correctly* by the digest (it invalidates). +- [128-bit digest collision could alias two schema states] → Non-adversarial input (your own schema), pairwise probability ~2⁻⁶⁴ — the same class as hardware bit-flip rates. Accepted and documented; no cheaper mechanism avoids it without reintroducing a writer contract. +- [`(.id db)` is not documented public API] → REPL-verified on plain/as-of/with dbs against peer 1.0.6733; access is wrapped so any failure falls back to a unique sentinel key (cache miss — correct, merely slower), and a pinned CI test makes a peer upgrade that changes it fail loudly. Note the digest includes entity ids, so even a cross-database `.id` collision could only share entries whose paths are identical. +- [A future path-relevant attribute added outside the digested tuples would escape invalidation] → Coverage invariant documented at the definition site; task 5.4's mutation-class regression tests (add/edit/retract × relation/permission each must change the digest) make the invariant executable. +- [No-expiry default for cursor TTL (D12) means tokens circulate indefinitely] → Tokens are basis-relative pagination state, not credentials; staleness is now caught by the `:f` fingerprint for schema changes, and data drift between pages was already the documented `(d/db conn)` caveat. Deployments wanting expiry set `:cursor-ttl-seconds`. +- [Deleting dead namespaces breaks any out-of-tree requires of them] → They reference storage attrs absent from the v7 schema; any such require was already broken at runtime. Noted in breaking changes. +- [`:f` fingerprint digests contain values derived from relation eids] → Not stable across DB rebuilds — acceptable: cursors are already documented as basis-bound; a false mismatch just forces a clean restart with a loud `:eacl/stale-cursor` error instead of silent corruption. + +## Migration Plan + +1. Land in dependency order (mirrors audit §19): D1–D4 (schema-write safety) → D7 (relation-datoms) → D9/D10/D11 (ID boundary) → D12 (cursors) → D8 (cache basis) → D5/D6 → D13 → D14/D15 throughout. +2. Storage: **no changes at all** — the schema-history digest is derived entirely from existing datoms; no new attributes, no migration transaction. No relationship/tuple/token format changes; cursors minted by old code remain decodable (`:f` absent → warn-and-accept for one release). +3. Consumers upgrade checklist (goes into README breaking-changes section): unknown-ID read behavior, `make-client` opts validation, cursor errors instead of silent restarts, `tx-relationship` tempid opt-in, strict arrow validation, dead-namespace deletion. +4. Rollback: `git revert` — no data migrations to unwind; nothing was written to storage that old code could even observe. + +## Open Questions + +- None blocking. Two deliberate deferrals recorded: (a) v3 recursive-cursor redesign (needs its own change; interacts with SpiceDB-compat and possible server-side cursor state), (b) transactional `:create` uniqueness (needs a transaction function; today documented as best-effort). diff --git a/openspec/changes/fix-audit-root-causes/proposal.md b/openspec/changes/fix-audit-root-causes/proposal.md new file mode 100644 index 00000000..a8aed7d4 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/proposal.md @@ -0,0 +1,38 @@ +# Proposal: fix-audit-root-causes + +## Why + +The 2026-07-06 full source audit ([docs/reports/2026-07-06-eacl-full-source-audit.md](../../../docs/reports/2026-07-06-eacl-full-source-audit.md)) verified 15 bugs against a live REPL, none covered by the (green) test suite. Two are critical data-loss/correctness bugs (`write-schema!` silently wipes the schema on parse failure; permissions silently fail for legal type names), and four are high severity (stale authorization from cache, a relationship data leak, silently ignored client config, unbounded cursors). The engines themselves verified sound — the defects are root-caused in five edge layers: schema-write pipeline trust, one bounded index scan, cache keying, nil-tolerant ID resolution, and nil-on-failure cursor decoding. This change fixes those root causes rather than patching symptoms. + +## What Changes + +- **Schema-write safety**: `write-schema!` rejects unparseable schema (instaparse failure objects currently flow through as an *empty* schema and retract everything), supports `//` and `/* */` comments, rejects duplicate definitions/relations instead of last-wins, validates arrows against *all* subject types of a relation (currently declaration-order-dependent), flattens parenthesized union expressions instead of crashing, and refuses full-schema retraction without explicit opt-in. +- **Permission-path resolution**: replace the `:a`–`:z` bounded `d/index-range` in `relation-datoms` with a prefix scan so relations with any legal subject-type keyword (uppercase, `z`-prefixed, namespaced) participate in permission evaluation; make cache invalidation *derived* instead of signaled — cache keys carry a digest of the schema's visible history read from the db value being queried, so every schema mutation (write-schema!, programmatic transaction, retraction, excision) invalidates automatically on every peer, for as-of views, and for speculative `d/with` dbs (currently the caches are keyed by the immutable `(.id db)` and only evicted by same-JVM `write-schema!` — revoked permissions keep granting access on other peers and after programmatic schema writes). +- **Strict object-ID resolution**: **BREAKING** — reads (`read-relationships`, `lookup-resources`, `lookup-subjects`, counts) with a nonexistent object ID return *empty* results (SpiceDB-compatible, consistent with `can?` → `false`) instead of today's mix of return-all-relationships (the leak), `AssertionError`s, and `false`; relationship writes validate both endpoints resolve and throw typed errors naming the missing object instead of raw Datomic `not-an-entity` errors; `impl/tx-relationship` requires explicit opt-in (`:allow-tempids? true`) to treat unresolvable strings as tempids instead of silently minting ghost entities; `make-client` accepts the README-documented `:entid->object-id` key and **BREAKING** — throws on unrecognized option keys (previously silently ignored, so typo'd ID config fell back to `:eacl/id`). +- **Cursor token handling**: **BREAKING** — expired or undecodable cursor tokens throw a typed error instead of decoding to `nil` and silently restarting pagination at page 1; cursor TTL becomes configurable via `make-client` opts (the existing `ttl-seconds` parameter is currently dead code); cursors embed a permission-path fingerprint so a schema change mid-pagination fails loudly instead of silently mis-skipping. +- **API error contract**: implement the declared-but-missing `write-relationship!`/`delete-relationship!` protocol methods (currently `AbstractMethodError`); replace `assert`-based input validation in the client layer with typed `ex-info` errors (asserts vanish under `*assert*` false); remove the vacuous type assertion in `count-resources`. +- **Housekeeping** (no spec impact): delete dead v6 namespaces (`eacl.datomic.rules*`, `eacl.datomic.impl.datalog`, `base/Relationship`) and commented-out root scripts; fix the README ID-configuration and relationship-transaction examples; rename `eacl.datomic.parser_test` → `eacl.datomic.parser-test` so it runs under `clj -X:test`; fix test typos (`[:eacl/id "user2"]`, empty `testing` block); add a differential property test (lookup-resources set == can? ground truth == paginated union == count) codifying the audit's cross-check. + +**Explicitly out of scope**: the v3 recursive-cursor growth/eid-leak redesign (audit §6). Its root cause is architectural (the cursor *is* the recursion state) and interacts with SpiceDB-compat goals; it needs its own design change. This change only adds the cursor-fingerprint guard and documents the growth characteristic. + +## Capabilities + +### New Capabilities + +- `schema-write-safety`: parsing, validating, and transacting SpiceDB schema strings without silent data loss — parse-failure rejection, comment support, duplicate rejection, order-independent arrow validation, paren-expression support, full-retraction guard. +- `permission-path-resolution`: resolving schema edges (relations/permissions) for permission evaluation — correct for all legal keyword type names, and cache-fresh across schema changes on every peer. +- `object-id-resolution`: the external-ID ↔ internal-eid boundary of the Datomic client — strict resolution with typed errors, opt-in tempids, validated client configuration. +- `cursor-token-handling`: opaque pagination cursor encoding/decoding — typed failures, configurable TTL, schema-change detection. +- `api-error-contract`: `IAuthorization` protocol completeness and typed error behavior for invalid inputs on the Datomic client. + +### Modified Capabilities + + + +## Impact + +- **Code**: `src/eacl/spicedb/parser.clj` (parse pipeline, grammar, validation), `src/eacl/datomic/schema.clj` (write-schema!, reference validation), `src/eacl/datomic/impl/indexed.clj` (relation-datoms, cache keys/eviction), `src/eacl/datomic/impl.clj` (tx-relationship, read-relationships), `src/eacl/datomic/core.clj` (make-client, cursor tokens, error contract, protocol methods), `src/eacl/core.clj` (protocol docstrings), deletions of `src/eacl/datomic/rules*.clj`, `src/eacl/datomic/impl/datalog.clj`, parts of `src/eacl/datomic/impl/base.clj`. +- **APIs**: behavioral changes marked **BREAKING** above are all silent-failure → loud-failure conversions; wire formats (cursor token prefix `eacl1_`, relationship tuples, schema entities) are unchanged. Existing valid configurations and schemas continue to work unmodified. +- **Consumers**: CloudAfrica production and side projects must review: (1) any code depending on `read-relationships` returning results for unknown IDs, (2) `make-client` opts maps for unknown/typo'd keys, (3) pagination loops for expired-cursor handling, (4) direct `impl/tx-relationship` callers relying on tempid pass-through (test fixtures in this repo are updated as part of the change). +- **Tests**: new coverage for every fixed path (parse failure, adversarial type names, cache invalidation, nonexistent IDs, cursor expiry/tamper), plus the differential property test; existing suite must stay green. +- **Docs**: README ID-configuration and quickstart sections corrected; audit report cross-referenced from fixes. diff --git a/openspec/changes/fix-audit-root-causes/specs/api-error-contract/spec.md b/openspec/changes/fix-audit-root-causes/specs/api-error-contract/spec.md new file mode 100644 index 00000000..1581acb2 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/specs/api-error-contract/spec.md @@ -0,0 +1,34 @@ +# api-error-contract + +`IAuthorization` protocol completeness and typed error behavior for the Datomic client. Covers the `Spiceomic` record in `eacl.datomic.core` and protocol declarations in `eacl.core`. + +## ADDED Requirements + +### Requirement: All declared protocol methods are implemented +Every method declared on `IAuthorization` SHALL have an implementation on the Datomic client. `write-relationship!` (operation arity and map arity) and `delete-relationship!` (positional and map arities) SHALL delegate to the relationship-write pipeline. No protocol method SHALL throw `AbstractMethodError`. + +#### Scenario: write-relationship! works +- **WHEN** `(write-relationship! client :touch subject :owner resource)` is called with resolvable objects +- **THEN** the relationship exists afterwards and the call returns a `:zed/token` + +#### Scenario: delete-relationship! works +- **WHEN** `(delete-relationship! client subject :owner resource)` is called for an existing relationship +- **THEN** the relationship no longer exists afterwards + +### Requirement: Unimplemented and unsupported features throw typed errors +`expand-permission-tree` SHALL throw `ex-info` with `:type :eacl/not-implemented`. Passing a consistency other than `fully-consistent` to `can?` SHALL throw `ex-info` with `:type :eacl/unsupported-consistency`. These SHALL be `ex-info` (catchable by `:type`), not bare `Exception`s or `assert`s. + +#### Scenario: Non-full consistency is a typed error +- **WHEN** `can?` is called with `(consistency/fresh token)` +- **THEN** an `ex-info` with `:type :eacl/unsupported-consistency` is thrown + +### Requirement: Client input validation does not rely on assertions +Input validation in the client layer (missing/invalid subjects, resources, cursors, configuration) SHALL be enforced with typed `ex-info` errors or the documented empty-result contract — never with `assert`/`{:pre …}`, whose behavior disappears when `*assert*` is disabled. Vacuous checks (assertions that can never fail, such as comparing a value to itself) SHALL be removed. + +#### Scenario: Validation survives disabled assertions +- **WHEN** the client namespaces are compiled with `*assert*` bound to `false` and an invalid input from the unknown-ID contract is supplied +- **THEN** the documented behavior (empty result or typed error) still occurs + +#### Scenario: No vacuous type assertion in count-resources +- **WHEN** `count-resources` is called with a valid subject +- **THEN** no self-comparing type assertion exists in the code path (verified by inspection/test of behavior with mismatched entity types) diff --git a/openspec/changes/fix-audit-root-causes/specs/cursor-token-handling/spec.md b/openspec/changes/fix-audit-root-causes/specs/cursor-token-handling/spec.md new file mode 100644 index 00000000..976d8f19 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/specs/cursor-token-handling/spec.md @@ -0,0 +1,42 @@ +# cursor-token-handling + +Opaque pagination cursor encoding/decoding for the Datomic client: loud typed failures, configurable TTL, and schema-change detection. Covers `cursor->token`, `token->cursor`, and cursor plumbing in `eacl.datomic.core` / `eacl.datomic.impl.indexed`. + +## ADDED Requirements + +### Requirement: Invalid cursor tokens throw typed errors +`token->cursor` SHALL return `nil` only for a `nil` input (meaning "first page") and SHALL pass raw cursor maps through unchanged (backward compatibility). Any non-nil string token that cannot be decoded — wrong prefix, corrupt base64/EDN — SHALL throw `ex-info` with `:type :eacl/invalid-cursor`. Pagination SHALL NOT silently restart from the first page on a bad cursor. + +#### Scenario: Garbage token fails loudly +- **WHEN** `lookup-resources` is called with `:cursor "eacl1_not-valid"` or `:cursor "garbage"` +- **THEN** an `ex-info` with `:type :eacl/invalid-cursor` is thrown, and the first page is not silently returned + +#### Scenario: nil cursor still means first page +- **WHEN** `lookup-resources` is called with `:cursor nil` +- **THEN** the first page is returned + +### Requirement: Cursor TTL is configurable and defaults to no expiry +Cursor expiry SHALL be off by default: tokens minted without a configured TTL SHALL carry no expiry and SHALL decode regardless of age. `make-client` SHALL accept `:cursor-ttl-seconds`; when set, minted tokens embed expiry and decoding an expired token SHALL throw `ex-info` with `:type :eacl/invalid-cursor` and `:reason :expired`. + +#### Scenario: Slow batch pagination does not restart +- **WHEN** a client without `:cursor-ttl-seconds` resumes pagination with a token minted more than 5 minutes ago +- **THEN** the next page is returned normally + +#### Scenario: Configured TTL is enforced loudly +- **WHEN** a client configured with `{:cursor-ttl-seconds 60}` decodes a token older than 60 seconds +- **THEN** an `:eacl/invalid-cursor` error with `:reason :expired` is thrown — not a silent first page + +### Requirement: Cursors detect permission-path changes between pages +Cursors SHALL embed a two-part fingerprint at mint time: the schema-history digest of the minting db value, and a content digest of the query's resolved permission paths (v2) or recursive query plan (v3). On resume: an identical schema digest SHALL proceed (identical schema history implies identical paths); a differing schema digest SHALL trigger recomputation of this query's paths — if their digest matches the cursor's, pagination proceeds (the schema change did not affect this query); if it differs, `ex-info` with `:type :eacl/stale-cursor` SHALL be thrown instead of silently mis-skipping results. Tokens minted before fingerprints existed (no fingerprint field) SHALL be accepted with a logged warning for one release. + +#### Scenario: Schema change affecting the query fails loudly +- **WHEN** page 1 is fetched, `write-schema!` changes the paths of the queried permission, and page 2 is requested with page 1's cursor +- **THEN** an `:eacl/stale-cursor` error is thrown + +#### Scenario: Unrelated schema change does not invalidate the cursor +- **WHEN** page 1 is fetched, a schema change lands that does not alter the queried permission's resolved paths, and page 2 is requested with page 1's cursor +- **THEN** pagination resumes normally + +#### Scenario: Unchanged schema resumes normally +- **WHEN** pages are fetched across an unchanged schema (including unrelated relationship writes in between) +- **THEN** pagination resumes exactly where it left off, with no duplicates or gaps diff --git a/openspec/changes/fix-audit-root-causes/specs/object-id-resolution/spec.md b/openspec/changes/fix-audit-root-causes/specs/object-id-resolution/spec.md new file mode 100644 index 00000000..aa4ab612 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/specs/object-id-resolution/spec.md @@ -0,0 +1,57 @@ +# object-id-resolution + +The external-ID ↔ internal-eid boundary of the Datomic client: one coherent contract for unknown IDs, opt-in tempids, and validated client configuration. Covers `make-client`, `spice-object->internal`, `spiceomic-read-relationships`, `spiceomic-write-relationships!`, and `impl/tx-relationship`. + +## ADDED Requirements + +### Requirement: Unknown object IDs on read operations return empty results +Read operations given a subject or resource ID that does not resolve to an entity SHALL return empty results — `read-relationships` → `[]`, `lookup-resources`/`lookup-subjects` → an empty page, `count-resources` → `{:count 0}`, `can?` → `false` — consistent with SpiceDB, where unknown object IDs simply match nothing. An unresolvable ID SHALL NOT be conflated with an absent filter. + +#### Scenario: read-relationships does not degrade to a global scan +- **WHEN** relationships exist for users `alice` and `bob`, and `read-relationships` is called with `{:subject/id "i-do-not-exist"}` +- **THEN** the result is `[]` — not the full relationship set + +#### Scenario: Lookups return empty pages instead of AssertionErrors +- **WHEN** `lookup-resources` is called with a nonexistent subject, or `lookup-subjects` with a nonexistent resource +- **THEN** an empty `:data` page is returned (no `AssertionError`, no exception) + +#### Scenario: can? remains false for unknown objects +- **WHEN** `can?` is called with a nonexistent subject or resource ID (including `nil`) +- **THEN** it returns `false` + +### Requirement: Unknown object IDs on write operations throw typed errors +`write-relationships!` (and the create/touch/delete wrappers) SHALL throw `ex-info` with `:type :eacl/unknown-object` identifying the offending `{:type … :id …}` when a relationship endpoint does not resolve to an **existing entity**, before any tx-data is built. The check SHALL verify entity existence (datom presence), not mere `d/entid` resolution — `d/entid` passes numeric inputs through unchanged, so plausible-but-unallocated eids "resolve" and would otherwise surface as raw transactor errors. Raw Datomic `:db.error/not-an-entity` / `:db.error/invalid-entity-id` errors SHALL NOT surface for this case, and `:delete`/`:touch` SHALL NOT silently no-op on unresolvable endpoints. + +#### Scenario: Create with nonexistent subject +- **WHEN** `create-relationships!` is called with subject `{:type :user :id "ghost-user"}` that has no entity +- **THEN** an `ex-info` with `:type :eacl/unknown-object` and the subject's type and id in `ex-data` is thrown + +#### Scenario: Create with unallocated numeric eid +- **WHEN** a relationship write is attempted (via the internal API) with a numeric ID in a valid eid range that was never allocated +- **THEN** the same `:eacl/unknown-object` error is thrown before transacting — not a raw `:db.error/invalid-entity-id` + +### Requirement: Tempid pass-through in tx-relationship is opt-in +`impl/tx-relationship` SHALL throw `:eacl/unknown-object` for string IDs that do not resolve, unless called with `{:allow-tempids? true}`, in which case unresolvable strings pass through as Datomic tempids (supporting same-transaction entity+relationship creation). Ghost entities SHALL NOT be minted by default. + +#### Scenario: Typo'd ID no longer mints a ghost entity +- **WHEN** `tx-relationship` is called (without the flag) with resource id `"acct-1x"` while only `"acct-1"` exists +- **THEN** it throws `:eacl/unknown-object` and no new entity is created + +#### Scenario: Fixtures-style same-transaction creation still works +- **WHEN** `tx-relationship` is called with `{:allow-tempids? true}` and string IDs matching `:db/id` tempids of entities in the same transaction +- **THEN** the transaction succeeds and the relationship tuples reference the newly created entities + +### Requirement: Client configuration is validated and matches documented keys +`make-client` SHALL accept `:entid->object-id` (`(fn [db eid] …)`) as the canonical ID-coercion option (as documented in the README), SHALL continue accepting `:entity->object-id` as a deprecated alias, SHALL throw `:eacl/invalid-config` when both are supplied, and SHALL throw `:eacl/invalid-config` listing unknown and known keys when an unrecognized option key is supplied. + +#### Scenario: README-documented key is honored +- **WHEN** a client is built with `{:entid->object-id (fn [db eid] (str "EXT-" (:eacl/id (d/entity db eid))))}` +- **THEN** `lookup-resources` returns IDs produced by that function (e.g. `"EXT-acct-1"`) + +#### Scenario: Typo'd option key fails fast +- **WHEN** a client is built with `{:entid->objectid …}` (misspelled) +- **THEN** `make-client` throws `:eacl/invalid-config` naming the unknown key and the set of known keys + +#### Scenario: Conflicting aliases are rejected +- **WHEN** both `:entid->object-id` and `:entity->object-id` are supplied +- **THEN** `make-client` throws `:eacl/invalid-config` diff --git a/openspec/changes/fix-audit-root-causes/specs/permission-path-resolution/spec.md b/openspec/changes/fix-audit-root-causes/specs/permission-path-resolution/spec.md new file mode 100644 index 00000000..e15714f5 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/specs/permission-path-resolution/spec.md @@ -0,0 +1,64 @@ +# permission-path-resolution + +Resolving schema edges (relations and permissions) for permission evaluation: correct for all legal keyword type names, and cache-fresh for every schema mutation, on every peer, for every db view — with no writer-side contract. Covers `relation-datoms`, `calc-permission-paths`, and the path/plan caches in `eacl.datomic.impl.indexed`. + +## ADDED Requirements + +### Requirement: Relation lookup supports all legal keyword type names +`relation-datoms` (and everything built on it: `calc-permission-paths`, `find-relation-def`, `resolve-self-relation`, the recursive query planner) SHALL return the relation entities for a `(resource-type, relation-name)` pair regardless of how the subject-type keyword collates — including uppercase-initial, `z`-prefixed, and namespaced keywords. Permission evaluation SHALL NOT silently ignore relations because of their subject-type name. + +#### Scenario: Subject type sorting after :z +- **WHEN** the schema defines `(Relation :zone :owner :zebra)` with `(Permission :zone :admin {:relation :owner})`, and a `:zebra` subject has an `:owner` relationship to a `:zone` resource +- **THEN** `can?` returns `true` and `lookup-resources` returns the zone + +#### Scenario: Uppercase and namespaced subject types +- **WHEN** relations exist with subject types `:Admin` and `:my.app/user` +- **THEN** `relation-datoms` returns their datoms and permission paths include them + +#### Scenario: Prefix scan does not leak into other relations or attributes +- **WHEN** relations exist for both `(:zone :owner)` and `(:zone :ownerx)` and other indexed attributes follow the relation tuple index +- **THEN** `relation-datoms` for `(:zone :owner)` returns only exact `(:zone :owner *)` datoms + +### Requirement: Cache keys are derived from the schema history of the queried db value +The permission-path and recursive-query-plan cache keys SHALL be derived from the db value being queried: the database id plus a content digest of the visible history of the relation and permission composite tuple attributes, filtered to the db's as-of point when present (`d/as-of-t`, not `d/basis-t`, which returns the underlying basis for as-of views). Any schema mutation — `write-schema!`, a programmatic transaction, entity retraction, or excision — SHALL change the digest and therefore the key. Correctness SHALL NOT require any writer-side signal, helper call, or eviction, on any peer. + +#### Scenario: Programmatic schema change is picked up with no signal +- **WHEN** a Permission entity is retracted via plain `d/transact` (no helper, no eviction, no write-schema!) and `get-permission-paths` / `can?` are called on a db basis that includes the retraction +- **THEN** the revoked permission no longer grants access + +#### Scenario: write-schema! invalidates on every peer +- **WHEN** peer A performs `write-schema!` removing a permission, and peer B (which has cached paths for it) queries a db basis that includes A's transaction +- **THEN** peer B's `can?` reflects the removal without any process-local eviction on B + +#### Scenario: as-of views resolve historical paths +- **WHEN** a permission existed at basis T1 and was removed at T2 +- **THEN** `get-permission-paths` against `(d/as-of db T1)` returns the path, and against the current db returns none — even when both are queried from the same process after caching + +#### Scenario: Speculative dbs cannot poison the shared cache +- **WHEN** paths are computed against `(:db-after (d/with db tx))` where `tx` speculatively alters schema entities, and a real transaction later lands at the same `t` +- **THEN** queries against the real db value never receive paths computed from the speculative view + +#### Scenario: Unchanged schema keeps hitting the cache +- **WHEN** relationship (non-schema) writes occur between queries +- **THEN** path lookups for the same `(resource-type, permission)` are served from cache without recomputation (observable via a `calc-permission-paths` call counter) + +### Requirement: Only positively classified db views share the cache +The digest SHALL be computed only for db values positively classified as plain or as-of views (`d/as-of-t`, `d/since-t`, `d/is-filtered`, `d/is-history` predicates). Any other view — `d/filter`, `d/since`, history dbs, or unrecognized types — SHALL receive a unique cache key, computing paths fresh from that view and never sharing entries with other views. In particular, a `d/filter` db that hides schema datoms SHALL NOT publish its paths under the plain db's key: filter predicates are arbitrary functions (possibly impure or time-dependent), so digests of filtered views are not trustworthy as shared keys. + +#### Scenario: Filtered db cannot poison the plain db's cache +- **WHEN** paths are computed against a `d/filter` db whose predicate hides some permission entities, and the plain db is queried afterwards +- **THEN** the plain db's paths include the hidden permissions (computed fresh or from its own entry — never from the filtered view's computation) + +### Requirement: Cache-key derivation failures degrade to misses, never staleness +If computing the schema digest fails for any reason (unsupported db view type, internal API change), the cache key SHALL be unique for that lookup, forcing a recomputation from the queried db value. No failure mode SHALL serve a previously cached entry for a db whose schema state cannot be established. + +#### Scenario: Digest failure forces recomputation +- **WHEN** the digest computation throws for a given db value +- **THEN** paths are computed directly from that db value and the result is correct for it (at worst uncached) + +### Requirement: Manual eviction remains available +`evict-permission-paths-cache!` SHALL remain public and SHALL clear both the permission-path and query-plan caches; `write-schema!` SHALL continue to invoke it (immediate local effect, though no longer required for correctness). + +#### Scenario: Manual eviction clears both caches +- **WHEN** `evict-permission-paths-cache!` is called +- **THEN** subsequent path and plan lookups recompute (observable via call counters) diff --git a/openspec/changes/fix-audit-root-causes/specs/schema-write-safety/spec.md b/openspec/changes/fix-audit-root-causes/specs/schema-write-safety/spec.md new file mode 100644 index 00000000..2c023306 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/specs/schema-write-safety/spec.md @@ -0,0 +1,75 @@ +# schema-write-safety + +Parsing, validating, and transacting SpiceDB schema strings without silent data loss. Covers `eacl.spicedb.parser` and `eacl.datomic.schema/write-schema!`. + +## ADDED Requirements + +### Requirement: Unparseable schema is rejected without side effects +`write-schema!` SHALL throw an `ex-info` with `:type :eacl.schema/parse-error` (including the instaparse failure detail) when the schema string does not parse, and SHALL NOT transact any changes. `->eacl-schema` SHALL throw when handed an instaparse failure object and SHALL never coerce a failed parse into an empty schema. + +#### Scenario: Syntax error leaves existing schema untouched +- **WHEN** a schema with relations and permissions is stored, and `write-schema!` is called with a schema string missing a closing brace +- **THEN** an `ex-info` with `:type :eacl.schema/parse-error` is thrown, and `read-schema` afterwards returns the same relations and permissions as before + +#### Scenario: Parse failure reports position detail +- **WHEN** `write-schema!` is called with `"definition user { relation owner user }"` (missing `:`) +- **THEN** the thrown error's `ex-data` contains the instaparse failure (line/column/expected information) + +### Requirement: Schema comments are supported +The parser SHALL accept `//` line comments and `/* */` block comments anywhere whitespace is legal, matching the SpiceDB DSL. + +#### Scenario: Line comment before a definition +- **WHEN** `write-schema!` is called with `"// users\ndefinition user {}"` +- **THEN** the schema is written successfully with the `user` definition + +#### Scenario: Comments inside a definition body +- **WHEN** a schema contains `/* block */` between relations and `// trailing` after a permission expression +- **THEN** parsing succeeds and the extracted relations and permissions are identical to the comment-free equivalent + +### Requirement: Duplicate declarations are rejected +Schema extraction SHALL throw a typed error when the same definition name appears twice, when the same relation name is declared twice within a definition, or when a permission shares a name with a relation on the same definition. Multi-type relations declared once with `|` (e.g. `relation owner: user | group`) SHALL NOT be treated as duplicates. + +#### Scenario: Duplicate definition blocks +- **WHEN** a schema contains two `definition account { ... }` blocks +- **THEN** `->eacl-schema` throws `ex-info` with `:type :eacl.schema/duplicate-definition` naming `account`, and no block is silently dropped + +#### Scenario: Duplicate relation declaration +- **WHEN** a definition contains `relation owner: user` twice +- **THEN** a typed duplicate-relation error names the definition and relation + +#### Scenario: Multi-type relation is not a duplicate +- **WHEN** a definition contains `relation owner: user | group` once +- **THEN** the schema is accepted and expands to two Relation entities + +### Requirement: Full-schema retraction requires explicit opt-in +`write-schema!` SHALL throw `ex-info` with `:type :eacl.schema/empty-schema-guard` when the new schema contains zero definitions while the stored schema is non-empty, unless called with `{:allow-empty-schema? true}`. + +#### Scenario: Empty parse result cannot wipe schema +- **WHEN** the stored schema is non-empty and `write-schema!` is called with a schema string yielding zero definitions +- **THEN** the guard error is thrown and no retractions are transacted + +#### Scenario: Explicit opt-in allows wiping +- **WHEN** the same call is made as `(write-schema! conn schema-string {:allow-empty-schema? true})` and no relationships would be orphaned +- **THEN** the retraction proceeds + +### Requirement: Arrow validation is order-independent and covers all subject types +`validate-schema-references` SHALL validate an arrow `rel->target` against **every** subject type of `rel`. The schema SHALL be rejected if any subject type lacks `target`, or if `target` resolves to a relation on some subject types and a permission on others. Acceptance SHALL NOT depend on the declaration order of subject types. + +#### Scenario: Order does not change the verdict +- **WHEN** `permission mgmt` exists on `user` but not on `group`, and two schemas differ only in `relation owner: user | group` vs `relation owner: group | user`, each with `permission admin = owner->mgmt` +- **THEN** both schemas are rejected with an error listing `group` as lacking `mgmt` + +#### Scenario: Target present on all subject types is accepted +- **WHEN** `mgmt` is defined on both `user` and `group` +- **THEN** the schema is accepted regardless of subject-type declaration order + +### Requirement: Parenthesized union expressions are supported +Permission expressions using parentheses around union operands (e.g. `permission manage = (owner + editor)`) SHALL be flattened to their union components. A parenthesized expression used as an arrow base or target SHALL be rejected with a typed validation error, not an assertion failure. + +#### Scenario: Parenthesized union flattens +- **WHEN** `write-schema!` is called with `permission manage = (owner + editor)` where both relations exist +- **THEN** the schema is accepted and `manage` behaves identically to `permission manage = owner + editor` + +#### Scenario: Parenthesized arrow base is rejected clearly +- **WHEN** a schema contains `permission p = (a + b)->c` +- **THEN** a typed validation error explains parenthesized arrow bases are unsupported, and no `AssertionError` escapes diff --git a/openspec/changes/fix-audit-root-causes/tasks.md b/openspec/changes/fix-audit-root-causes/tasks.md new file mode 100644 index 00000000..1b198095 --- /dev/null +++ b/openspec/changes/fix-audit-root-causes/tasks.md @@ -0,0 +1,63 @@ +# Tasks: fix-audit-root-causes + +Order follows the design's migration plan (risk-first). Every group ends with a verification gate; run tests via nREPL per repo convention (`clj-nrepl-eval`), never via cold `clojure -M:test`. Audit references (§N) point at [docs/reports/2026-07-06-eacl-full-source-audit.md](../../../docs/reports/2026-07-06-eacl-full-source-audit.md), which contains a REPL repro for every item — each repro becomes a regression test. + +## 1. Schema-write safety (parser + write-schema!) — D1–D6 + +- [x] 1.1 Make `->eacl-schema` throw `ex-info {:type :eacl.schema/parse-error :failure (insta/get-failure tree)}` on instaparse failure input, and make `transform-schema` throw on non-`:schema` input instead of returning `nil` (§1) +- [x] 1.2 Add `//` and `/* */` comment support via a custom whitespace parser passed to instaparse `:auto-whitespace` (D2); keep `parse-schema`'s return contract unchanged +- [x] 1.3 Reject duplicate `definition` blocks, duplicate relation declarations within a definition, and permission/relation name collisions during extraction with typed errors naming the duplicate (§9, D3); verify `relation x: a | b` still expands to two Relations +- [x] 1.4 Add the full-retraction guard to `write-schema!` (`:eacl.schema/empty-schema-guard` when new schema has zero definitions and stored schema is non-empty) plus the `{:allow-empty-schema? true}` opts arity on `schema/write-schema!` only (D4) +- [x] 1.5 Rewrite `validate-schema-references` to validate arrows against the full **set** of subject types per relation (reject if any type lacks the target or target kinds differ across types, with types named in the error), and rewrite `resolve-component`/`collect-schema-info` to consult the same set so parser and validator cannot diverge (§10, D5) +- [x] 1.6 Flatten parenthesized union operands in `flatten-expression`; reject parenthesized arrow bases/targets with a typed validation issue in `collect-parse-tree-issues` (§8, D6) +- [x] 1.7 Regression tests for 1.1–1.6: malformed schema throws and leaves stored schema untouched (§1 repro); commented schema round-trips; duplicate-definition schema throws; both subject-type orders of the §10 repro are rejected; `(owner + editor)` paren schema accepted and behaves as union; empty-schema write throws without opt-in +- [x] 1.8 Gate: `eacl.datomic.schema-test`, parser tests, and `eacl.spice-test` green via nREPL + +## 2. Relation lookup prefix scan — D7 + +- [x] 2.1 Replace `relation-datoms`'s `:a`–`:z` `d/index-range` with the verified `d/seek-datoms :avet` prefix scan, including the mandatory attr-eid guard in `take-while` (§2, D7 — exact code in design) +- [x] 2.2 Regression tests: end-to-end `can?`/`lookup-resources`/`lookup-subjects` for subject types `:zebra`, `:Admin`, `:my.app/user` (§2 repro); exact-prefix isolation test (`(:zone :owner)` does not match `(:zone :ownerx)` datoms or subsequent attributes) +- [x] 2.3 Gate: full `eacl.datomic.impl.indexed-test` green via nREPL + +## 3. Strict object-ID resolution — D9–D11 + +- [x] 3.1 Introduce a supplied-but-unresolvable sentinel in `spice-object->internal` and the `spiceomic-read-relationships` filter resolution; short-circuit reads to empty results (`read-relationships` → `[]`, `lookup-resources`/`lookup-subjects` → empty page, `count-resources` → `{:count 0}`) and delete the corresponding client-layer asserts and the `lookup-subjects` `{:pre …}` (§4, D9, D13) +- [x] 3.2 Validate both endpoints in `spice-relationship->internal` before building tx-data; throw `ex-info {:type :eacl/unknown-object :object {:type … :id …}}` for create/touch/delete; the check verifies entity existence via datom presence (`(seq (d/datoms db :eavt eid))`), not mere `d/entid` resolution, so unallocated numeric eids are caught too (§11, D9) +- [x] 3.3 Make `impl/tx-relationship` strict by default with an `{:allow-tempids? true}` opts arity threaded through `resolve-relationship`/`object-id->eid-or-tempid`; update `fixtures/relationship-fixtures`, `txes-additional-account3+server`, and tempid-dependent tests to pass the flag (§12, D10) +- [x] 3.4 `make-client` option validation: accept canonical `:entid->object-id`, keep `:entity->object-id` as deprecated alias, throw `:eacl/invalid-config` on both-supplied or any unknown key (§5, D11) +- [x] 3.5 Regression tests: §4 leak repro returns `[]`; §11 ghost-write repro throws `:eacl/unknown-object`; §12 typo repro throws and mints no entity while the `:allow-tempids?` fixtures path still works; §5 repro returns `EXT-`-prefixed IDs; misspelled opt key throws +- [x] 3.6 Update `eacl.datomic.config-test` (missing-subject lookup now returns an empty page, not `thrown?`) and any indexed tests relying on assert behavior +- [x] 3.7 Gate: full suite green via nREPL + +## 4. Cursor token handling — D12 + +- [x] 4.1 Rework `token->cursor`: `nil` → `nil`, raw map pass-through, any other failure (bad prefix, corrupt base64/EDN, expired) → `ex-info {:type :eacl/invalid-cursor :reason :undecodable|:expired}`; treat missing `:t` as no-expiry (§7) +- [x] 4.2 Add `:cursor-ttl-seconds` to `make-client` opts (default nil = no expiry) and thread it into every `cursor->token` call; delete the hardcoded 300 s default +- [x] 4.3 Add the two-part fingerprint `:f {:s :p }` in `build-v2-cursor` and the v3 recursive state (D12); on resume: equal `:s` proceeds; differing `:s` recomputes this query's paths and compares `:p` — equal proceeds, differing throws `ex-info {:type :eacl/stale-cursor}`; missing `:f` is accepted with a `log/warn`; document v3 cursor growth (§6) in `lookup-resources` docstring and README as a known limitation with the follow-up change noted (depends on 5.1's `schema-basis`; implement after group 5 or stub `:s` until then) +- [x] 4.4 Regression tests: garbage token throws; expired token throws when TTL configured; default-config token decodes with an old timestamp (§7 repro inverted); a schema change altering the queried permission's paths throws `:eacl/stale-cursor` on resume; an unrelated schema change resumes normally; unchanged schema resumes with no duplicates/gaps; raw-map cursors still work (back-compat test exists in `spice-test`) +- [x] 4.5 Gate: `eacl.spice-test` and `eacl.datomic.impl.indexed-test` green via nREPL + +## 5. Schema-digest cache keys — D8 + +- [x] 5.1 Implement `(schema-basis db)` in `impl.indexed`: positively classify the db view first — plain (`as-of-t` nil, `since-t` nil, `is-filtered` false, `is-history` false) or as-of (`as-of-t` non-nil, others clear) compute the digest; anything else (filter/since/history/unrecognized) returns a unique sentinel. Digest = 128 bits (SHA-256 truncated — FIPS-safe) folded in index order over the history datoms (`[e v t added?]`) of `:eacl.relation/resource-type+relation-name+subject-type` and `:eacl.permission/resource-type+source-relation-name+target-type+target-name+permission-name`, filtered to `t ≤ (or (d/as-of-t db) (d/basis-t db))`; wrap the whole computation (including `.id` access) so any exception also returns a unique sentinel (forced cache miss, never staleness) +- [x] 5.2 Memoize the digest in a synchronized `WeakHashMap` keyed by the db value (Datomic `Db` equality is value- and content-based, REPL-verified — see 5.3 pins); memoize only positively classified views, never sentinels; include the digest in both `permission-paths-cache-key` and `recursive-query-plan-cache-key`; keep `evict-permission-paths-cache!` public and called from `write-schema!`; document the coverage invariant (every attr the path/plan computation reads must be a component of a digested tuple) at the `schema-basis` definition site +- [x] 5.3 Pin the REPL-verified Datomic facts as regression tests so a peer upgrade fails loudly: single-component edit rewrites the composite tuple in the same tx; `retractEntity` retracts the tuple into history; `(d/history (d/as-of db t))` is filtered to ≤ t; `(d/basis-t (d/as-of db t))` returns the underlying basis (hence the `as-of-t` filter); `d/history` works on `d/with` db-afters and includes speculative datoms; `.id` returns the same UUID on plain/as-of/with dbs; `d/is-filtered` is true only for `d/filter` dbs; `as-of-t`/`since-t`/`is-history` identify their views with with-dbs reading as plain; `d/history` on a `d/filter` db respects the filter (pinned with a hide-everything predicate; filter views stay uncached because predicates are arbitrary functions); `d/history` on a `d/since` db is since-filtered; `Db.equals` is value-based and content-aware (same-basis `d/db` calls equal; different-content `d/with` db-afters at the same `t` NOT equal) +- [x] 5.4 Regression tests: §3 repro — a plain programmatic permission retraction (no helper, no eviction) is immediately visible to `get-permission-paths`/`can?`; `d/as-of` at the pre-change basis returns the historical paths; a `d/with` speculative schema edit does not poison the shared cache for the real db; a `d/filter` db hiding permission entities does not poison the plain db's cache (and vice versa); mutation-class coverage — each of add/edit/retract × relation/permission changes the digest; unchanged-schema relationship writes still hit the cache (assert `calc-permission-paths` call count via `with-redefs` counter, as in the existing caching test) +- [x] 5.5 Document in README: invalidation is fully automatic for all schema writes; digest cost is O(all-time schema edits) per db value (identity-memoized), with pathological-churn guidance; note post-`d/excise` behavior is correct by construction +- [x] 5.6 Gate: full suite green via nREPL + +## 6. API error contract — D13 + +- [x] 6.1 Implement `write-relationship!` (both arities) and `delete-relationship!` (both arities) on `Spiceomic`, delegating to `spiceomic-write-relationships!` (§13) +- [x] 6.2 Replace the consistency `assert` with `ex-info {:type :eacl/unsupported-consistency}` and `expand-permission-tree`'s bare `Exception` with `ex-info {:type :eacl/not-implemented}`; remove the vacuous type assert in `spiceomic-count-resources` (§16.1, §16.2, §16.11) +- [x] 6.3 Tests: `write-relationship!`/`delete-relationship!` round-trip; `(consistency/fresh token)` throws the typed error (existing `thrown? Throwable` assertion in `spice-test` keeps passing); validation behavior verified with `*assert*` bound false +- [x] 6.4 Gate: full suite green via nREPL + +## 7. Housekeeping, docs, and the differential test — D14–D15 + +- [x] 7.1 Delete dead namespaces and files: `src/eacl/datomic/rules.clj`, `src/eacl/datomic/rules/optimized.clj`, `src/eacl/datomic/rules/optimized_old.clj`, `src/eacl/datomic/impl/datalog.clj`, the `Relationship` fn in `impl/base.clj`, root-level `simple_test.clj` / `test_large_offset.clj` / `test_cursor_pagination.clj`, and the commented-out `performance_test.clj`/`benchmark_test.clj` bodies (or move to `docs/attic/`) (§16.3, §15) +- [x] 7.2 Fix README: ID-configuration section names the canonical `make-client` keys (§5); quickstart "transact relationships" example rewritten around `create-relationships!` / `tx-relationship` with `:allow-tempids?` (§15); add a Breaking Changes section covering the D9/D11/D12/D10/D5 behavior changes and dead-namespace removal; document cursor TTL option and the unknown-ID contract +- [x] 7.3 Rename ns `eacl.datomic.parser_test` → `eacl.datomic.parser-test` (file stays `parser_test.clj`) so the cognitect runner's `-test$` pattern matches (§14) +- [x] 7.4 Fix test typos: `[:eacl/id "user2"]` → `"user-2"` in `indexed_test.clj` (~line 542) and give the empty `(testing "…arrow_relation works")` block (~line 434) its intended body (§16.9) +- [x] 7.5 Add the seeded differential property test (no new deps): generate small random schemas/graphs (direct, arrow, self-permission, multi-path, recursive parent) and assert `lookup-resources` set == `can?` ground truth == paginated union at page sizes 1/3/7 == `count-resources`, and the reverse via `lookup-subjects` (D15, audit §18.6) +- [x] 7.6 Final gate: full suite green via nREPL, then one cold `clj -X:test` run to confirm the runner discovers all namespaces (including the renamed parser tests) and everything passes diff --git a/openspec/config.yaml b/openspec/config.yaml new file mode 100644 index 00000000..392946c6 --- /dev/null +++ b/openspec/config.yaml @@ -0,0 +1,20 @@ +schema: spec-driven + +# Project context (optional) +# This is shown to AI when creating artifacts. +# Add your tech stack, conventions, style guides, domain knowledge, etc. +# Example: +# context: | +# Tech stack: TypeScript, React, Node.js +# We use conventional commits +# Domain: e-commerce platform + +# Per-artifact rules (optional) +# Add custom rules for specific artifacts. +# Example: +# rules: +# proposal: +# - Keep proposals under 500 words +# - Always include a "Non-goals" section +# tasks: +# - Break tasks into chunks of max 2 hours diff --git a/src/eacl/datomic/core.clj b/src/eacl/datomic/core.clj index 44b2220d..90a6a345 100644 --- a/src/eacl/datomic/core.clj +++ b/src/eacl/datomic/core.clj @@ -15,40 +15,75 @@ [malli.core :as m])) (defn cursor->token - "Serializes an internal cursor map to an opaque string token with TTL." - [cursor & [{:keys [ttl-seconds] :or {ttl-seconds 300}}]] - (when cursor - (let [with-expiry (assoc cursor :t (+ (quot (System/currentTimeMillis) 1000) - ttl-seconds))] - (str "eacl1_" - (.encodeToString - (java.util.Base64/getEncoder) - (.getBytes (pr-str with-expiry) "UTF-8")))))) + "Serializes an internal cursor map to an opaque string token. + An expiry timestamp (:t, epoch seconds) is embedded only when the client is + configured with :cursor-ttl-seconds; by default tokens do not expire — + slow batch pagination must never silently restart (audit §7)." + ([cursor] (cursor->token cursor nil)) + ([cursor {:keys [cursor-ttl-seconds]}] + (when cursor + (let [cursor' (if cursor-ttl-seconds + (assoc cursor :t (+ (quot (System/currentTimeMillis) 1000) + cursor-ttl-seconds)) + cursor)] + (str "eacl1_" + (.encodeToString + (java.util.Base64/getEncoder) + (.getBytes (pr-str cursor') "UTF-8"))))))) (defn token->cursor - "Deserializes an opaque cursor token. Raw cursor maps are accepted for - backward compatibility." - [token-or-cursor] - (cond - (nil? token-or-cursor) nil - (map? token-or-cursor) token-or-cursor - (and (string? token-or-cursor) - (.startsWith ^String token-or-cursor "eacl1_")) - (try - (let [cursor (edn/read-string - (String. (.decode (java.util.Base64/getDecoder) - (.getBytes (subs token-or-cursor 6) "UTF-8")) - "UTF-8")) - now (quot (System/currentTimeMillis) 1000)] - (when (> (:t cursor now) now) - (dissoc cursor :t))) - (catch Exception _ nil)) - :else nil)) + "Deserializes an opaque cursor token. + + Contract: + - nil means \"first page\" and returns nil; + - raw cursor maps pass through (backward compatibility); + - any other non-nil input that fails to decode throws + ex-info {:type :eacl/invalid-cursor :reason :undecodable}; + - a token carrying an expiry (:t) throws {:reason :expired} when expired and + the client is configured with :cursor-ttl-seconds. Tokens without :t never + expire. A bad cursor must fail loudly — decoding to nil silently restarted + pagination from the first page (audit §7)." + ([token-or-cursor] (token->cursor token-or-cursor nil)) + ([token-or-cursor {:keys [cursor-ttl-seconds]}] + (cond + (nil? token-or-cursor) nil + (map? token-or-cursor) token-or-cursor + + (and (string? token-or-cursor) + (.startsWith ^String token-or-cursor "eacl1_")) + (let [cursor (try + (edn/read-string + (String. (.decode (java.util.Base64/getDecoder) + (.getBytes (subs token-or-cursor 6) "UTF-8")) + "UTF-8")) + (catch Exception e + (throw (ex-info "Invalid cursor token: cannot be decoded." + {:type :eacl/invalid-cursor + :reason :undecodable} + e))))] + (when-not (map? cursor) + (throw (ex-info "Invalid cursor token: does not decode to a cursor map." + {:type :eacl/invalid-cursor + :reason :undecodable}))) + (let [now (quot (System/currentTimeMillis) 1000)] + (if (and cursor-ttl-seconds (:t cursor) (> now (:t cursor))) + (throw (ex-info "Invalid cursor token: expired." + {:type :eacl/invalid-cursor + :reason :expired + :expired-at (:t cursor)})) + (dissoc cursor :t)))) + + :else + (throw (ex-info "Invalid cursor token: unrecognized format." + {:type :eacl/invalid-cursor + :reason :undecodable + :token token-or-cursor}))))) (defn default-internal-cursor->spice [db {:keys [entid->object-id]} cursor] (when cursor - (if (= 2 (:v cursor)) + (cond + (= 2 (:v cursor)) (cond-> cursor (:e cursor) (update :e #(entid->object-id db %)) (:p cursor) (update :p @@ -56,6 +91,11 @@ (into {} (map (fn [[k v]] [k (entid->object-id db v)])) p)))) + + (= 3 (:v cursor)) + cursor + + :else (cond (:resource cursor) (S/transform [:resource :id] #(entid->object-id db %) cursor) (:subject cursor) (S/transform [:subject :id] #(entid->object-id db %) cursor))))) @@ -63,7 +103,8 @@ (defn default-spice-cursor->internal [db {:keys [object-id->entid]} cursor] (when cursor - (if (= 2 (:v cursor)) + (cond + (= 2 (:v cursor)) (cond-> cursor (:e cursor) (update :e #(object-id->entid db %)) (:p cursor) (update :p @@ -71,6 +112,11 @@ (into {} (map (fn [[k v]] [k (object-id->entid db v)])) p)))) + + (= 3 (:v cursor)) + cursor + + :else (cond (:resource cursor) (S/transform [:resource :id] #(object-id->entid db %) cursor) (:subject cursor) (S/transform [:subject :id] #(object-id->entid db %) cursor))))) @@ -92,19 +138,41 @@ filters] (let [subject-id (:subject/id filters) resource-id (:resource/id filters) - subject-eid (when subject-id (object-id->entid db subject-id)) - resource-eid (when resource-id (object-id->entid db resource-id)) - filters' (cond-> filters + subject-eid (when (some? subject-id) (object-id->entid db subject-id)) + resource-eid (when (some? resource-id) (object-id->entid db resource-id))] + (if (or (and (some? subject-id) (nil? subject-eid)) + (and (some? resource-id) (nil? resource-eid))) + ;; A filter names an object that does not exist: nothing can match. + ;; A supplied-but-unresolvable ID must not be conflated with an absent + ;; filter — that conflation degraded this query to a global scan. + [] + (let [filters' (cond-> filters subject-id (assoc :subject/id subject-eid) resource-id (assoc :resource/id resource-eid))] - (->> (impl/read-relationships db filters') - (map #(relationship->spice db opts %))))) + (->> (impl/read-relationships db filters') + (map #(relationship->spice db opts %))))))) + +(defn- resolve-existing-object + "Resolves an external spice object to its internal eid, verifying the entity + actually exists. Existence is checked via datom presence because d/entid + passes unallocated numeric eids through unchanged. Throws :eacl/unknown-object + when the object cannot be resolved to an existing entity." + [db object-id->entid {:keys [type id] :as obj}] + (let [eid (when (some? id) (object-id->entid db id))] + (if (and eid (seq (d/datoms db :eavt eid))) + (assoc obj :id eid) + (throw (ex-info (str "Unknown object: " (pr-str type) " with id " (pr-str id) " does not exist.") + {:type :eacl/unknown-object + :object {:type type :id id}}))))) (defn spice-relationship->internal - [db {:keys [spice-object->internal]} {:keys [subject relation resource]}] - {:subject (spice-object->internal db subject) + "Resolves both relationship endpoints to existing internal eids. + Throws :eacl/unknown-object for either endpoint rather than letting nils or + ghost ids reach tx-data (raw :db.error/not-an-entity) or silently no-op." + [db {:keys [object-id->entid]} {:keys [subject relation resource]}] + {:subject (resolve-existing-object db object-id->entid subject) :relation relation - :resource (spice-object->internal db resource)}) + :resource (resolve-existing-object db object-id->entid resource)}) (defn spiceomic-write-relationships! [conn opts updates] @@ -119,19 +187,21 @@ {:zed/token (str basis)})) (defn spiceomic-can? - [db {:keys [object->entid]} subject permission resource consistency] - (assert (= consistency/fully-consistent consistency) - "EACL only supports consistency/fully-consistent at this time.") + [db {:keys [object->entid]} subject permission resource consistency max-depth] + (when-not (= consistency/fully-consistent consistency) + (throw (ex-info "EACL only supports consistency/fully-consistent at this time." + {:type :eacl/unsupported-consistency + :consistency consistency}))) (let [subject-type (:type subject) subject-eid (object->entid db subject) resource-type (:type resource) resource-eid (object->entid db resource)] (if-not (and subject-eid resource-eid) false - (impl/can? db - (spice-object subject-type subject-eid) - permission - (spice-object resource-type resource-eid))))) + (impl/can? db {:subject (spice-object subject-type subject-eid) + :permission permission + :resource (spice-object resource-type resource-eid) + :max-depth max-depth})))) (defn spiceomic-lookup-resources [db @@ -144,24 +214,23 @@ {:as query :keys [subject]}] (log/debug 'spiceomic-lookup-resources 'query query) (let [internal-subject (spice-object->internal db subject)] - (assert (:id internal-subject) - (str "subject " (pr-str subject) - " passed to lookup-resources does not exist with ident " - (object-id->ident (:id subject)))) - (->> query - (S/setval [:subject] internal-subject) - (S/transform [:cursor] - (fn [token-or-cursor] - (some->> (token->cursor token-or-cursor) - (spice-cursor->internal db opts)))) - (impl/lookup-resources db) - (S/transform [:data S/ALL] - (fn [{:keys [type id]}] - (spice-object type (entid->object-id db id)))) - (S/transform [:cursor] - (fn [internal-cursor] - (some->> (internal-cursor->spice db opts internal-cursor) - cursor->token)))))) + (if (nil? (:id internal-subject)) + ;; Unknown subjects match nothing (SpiceDB-consistent; can? is false). + {:data [] :cursor nil} + (->> query + (S/setval [:subject] internal-subject) + (S/transform [:cursor] + (fn [token-or-cursor] + (some->> (token->cursor token-or-cursor opts) + (spice-cursor->internal db opts)))) + (impl/lookup-resources db) + (S/transform [:data S/ALL] + (fn [{:keys [type id]}] + (spice-object type (entid->object-id db id)))) + (S/transform [:cursor] + (fn [internal-cursor] + (some-> (internal-cursor->spice db opts internal-cursor) + (cursor->token opts)))))))) (defn spiceomic-count-resources [db @@ -171,22 +240,20 @@ internal-cursor->spice]} {:as query :keys [subject]}] (let [subject-ent (spice-object->internal db subject)] - (assert (:id subject-ent) - (str "subject passed to count-resources does not exist: " (pr-str subject))) - (assert (= (:type subject-ent) (:type subject)) - (str "count-resources: subject type passed does not match entity: " - (pr-str subject))) - (->> query - (S/setval [:subject] subject-ent) - (S/transform [:cursor] - (fn [token-or-cursor] - (some->> (token->cursor token-or-cursor) - (spice-cursor->internal db opts)))) - (impl/count-resources db) - (S/transform [:cursor] - (fn [internal-cursor] - (some->> (internal-cursor->spice db opts internal-cursor) - cursor->token)))))) + (if (nil? (:id subject-ent)) + ;; Unknown subjects match nothing (SpiceDB-consistent; can? is false). + {:count 0 :limit (:limit query -1) :cursor nil} + (->> query + (S/setval [:subject] subject-ent) + (S/transform [:cursor] + (fn [token-or-cursor] + (some->> (token->cursor token-or-cursor opts) + (spice-cursor->internal db opts)))) + (impl/count-resources db) + (S/transform [:cursor] + (fn [internal-cursor] + (some-> (internal-cursor->spice db opts internal-cursor) + (cursor->token opts)))))))) (defn spiceomic-lookup-subjects [db @@ -196,32 +263,37 @@ spice-cursor->internal internal-cursor->spice]} query] - (->> query - (S/transform [:resource] #(spice-object->internal db %)) - (S/transform [:cursor] - (fn [token-or-cursor] - (some->> (token->cursor token-or-cursor) - (spice-cursor->internal db opts)))) - (impl/lookup-subjects db) - (S/transform [:data S/ALL] - (fn [{:keys [type id]}] - (spice-object type (entid->object-id db id)))) - (S/transform [:cursor] - (fn [internal-cursor] - (some->> (internal-cursor->spice db opts internal-cursor) - cursor->token))))) + (let [internal-resource (spice-object->internal db (:resource query))] + (if (nil? (:id internal-resource)) + ;; Unknown resources match nothing (SpiceDB-consistent). + {:data [] :cursor nil} + (->> query + (S/setval [:resource] internal-resource) + (S/transform [:cursor] + (fn [token-or-cursor] + (some->> (token->cursor token-or-cursor opts) + (spice-cursor->internal db opts)))) + (impl/lookup-subjects db) + (S/transform [:data S/ALL] + (fn [{:keys [type id]}] + (spice-object type (entid->object-id db id)))) + (S/transform [:cursor] + (fn [internal-cursor] + (some-> (internal-cursor->spice db opts internal-cursor) + (cursor->token opts)))))))) (defrecord Spiceomic [conn opts] IAuthorization (can? [_ subject permission resource] - (spiceomic-can? (d/db conn) opts subject permission resource consistency/fully-consistent)) + (spiceomic-can? (d/db conn) opts subject permission resource consistency/fully-consistent nil)) (can? [_ subject permission resource consistency] - (spiceomic-can? (d/db conn) opts subject permission resource consistency)) + (spiceomic-can? (d/db conn) opts subject permission resource consistency nil)) - (can? [_ {:keys [subject permission resource consistency]}] + (can? [_ {:keys [subject permission resource consistency max-depth]}] (spiceomic-can? (d/db conn) opts subject permission resource - (or consistency consistency/fully-consistent))) + (or consistency consistency/fully-consistent) + max-depth)) (read-schema [_] (schema/read-schema (d/db conn))) @@ -235,6 +307,14 @@ (write-relationships! [_ updates] (spiceomic-write-relationships! conn opts updates)) + (write-relationship! [_ operation subject relation resource] + (spiceomic-write-relationships! conn opts + [(->RelationshipUpdate operation (->Relationship subject relation resource))])) + + (write-relationship! [_ {:keys [operation subject relation resource]}] + (spiceomic-write-relationships! conn opts + [(->RelationshipUpdate operation (->Relationship subject relation resource))])) + (create-relationships! [_ relationships] (spiceomic-write-relationships! conn opts (for [rel relationships] @@ -253,6 +333,14 @@ (for [rel relationships] (->RelationshipUpdate :delete rel)))) + (delete-relationship! [_ subject relation resource] + (spiceomic-write-relationships! conn opts + [(->RelationshipUpdate :delete (->Relationship subject relation resource))])) + + (delete-relationship! [_ {:keys [subject relation resource]}] + (spiceomic-write-relationships! conn opts + [(->RelationshipUpdate :delete (->Relationship subject relation resource))])) + (lookup-resources [_ query] (spiceomic-lookup-resources (d/db conn) opts query)) @@ -263,34 +351,69 @@ (spiceomic-lookup-subjects (d/db conn) opts query)) (expand-permission-tree [_ _] - (throw (Exception. "not impl.")))) + (throw (ex-info "expand-permission-tree is not implemented yet." + {:type :eacl/not-implemented + :method 'expand-permission-tree})))) + +(def ^:private known-client-opt-keys + #{:entid->object-id + :entity->object-id + :object-id->ident + :internal-cursor->spice + :spice-cursor->internal + :cursor-ttl-seconds}) (defn make-client + "Builds an IAuthorization client over a Datomic conn. + + Options (unknown keys throw :eacl/invalid-config — a silently ignored key + means silently wrong ID coercion, audit §5): + - :entid->object-id (fn [db eid] external-id) — canonical, as documented in the README. + - :entity->object-id (fn [entity] external-id) — deprecated alias; do not combine with the above. + - :object-id->ident (fn [external-id] ident-resolvable-by-d-entid). Default: [:eacl/id id]. + - :cursor-ttl-seconds — optional cursor token expiry; default nil (tokens never expire). + - :internal-cursor->spice / :spice-cursor->internal — advanced cursor coercion overrides." [conn - {:keys [entity->object-id + {:as config-opts + :keys [entid->object-id + entity->object-id object-id->ident internal-cursor->spice - spice-cursor->internal] - :or {entity->object-id (fn [ent] (:eacl/id ent)) - object-id->ident (fn [obj-id] [:eacl/id obj-id]) + spice-cursor->internal + cursor-ttl-seconds] + :or {object-id->ident (fn [obj-id] [:eacl/id obj-id]) internal-cursor->spice default-internal-cursor->spice spice-cursor->internal default-spice-cursor->internal}}] - (assert (fn? object-id->ident) - "EACL Config Error: object-id->ident fn is required to coerce a Spice Object ID to a Datomic ident that can be resolved by d/entid.") - (let [object-id->entid (fn [db object-id] + (when-let [unknown-keys (seq (remove known-client-opt-keys (keys config-opts)))] + (throw (ex-info (str "EACL Config Error: unknown make-client option(s) " (pr-str (vec unknown-keys)) + ". Known options: " (pr-str (vec (sort known-client-opt-keys))) ".") + {:type :eacl/invalid-config + :unknown-keys (vec unknown-keys) + :known-keys known-client-opt-keys}))) + (when (and entid->object-id entity->object-id) + (throw (ex-info "EACL Config Error: supply only one of :entid->object-id (canonical) or :entity->object-id (deprecated alias)." + {:type :eacl/invalid-config + :conflicting-keys [:entid->object-id :entity->object-id]}))) + (when-not (fn? object-id->ident) + (throw (ex-info "EACL Config Error: object-id->ident must be a fn that coerces a Spice Object ID to a Datomic ident resolvable by d/entid." + {:type :eacl/invalid-config + :key :object-id->ident}))) + (let [entid->object-id (or entid->object-id + (when entity->object-id + (fn [db eid] (entity->object-id (d/entity db eid)))) + (fn [db eid] (:eacl/id (d/entity db eid)))) + object-id->entid (fn [db object-id] (d/entid db (object-id->ident object-id))) - entid->object-id (fn [db eid] - (entity->object-id (d/entity db eid))) opts {:object-id->ident object-id->ident :entid->object-id entid->object-id - :entity->object-id entity->object-id :object-id->entid object-id->entid :object->entid (fn [db {:keys [id]}] (object-id->entid db id)) :internal-object->spice (fn [db {:keys [type id]}] (spice-object type (entid->object-id db id))) :spice-object->internal (fn [db obj] - (update obj :id #(object-id->entid db %))) + (update obj :id #(when (some? %) (object-id->entid db %)))) :internal-cursor->spice internal-cursor->spice - :spice-cursor->internal spice-cursor->internal}] + :spice-cursor->internal spice-cursor->internal + :cursor-ttl-seconds cursor-ttl-seconds}] (->Spiceomic conn opts))) diff --git a/src/eacl/datomic/impl.clj b/src/eacl/datomic/impl.clj index 2c50bdfb..586dd8fb 100644 --- a/src/eacl/datomic/impl.clj +++ b/src/eacl/datomic/impl.clj @@ -15,10 +15,23 @@ [subject relation resource] (eacl/->Relationship subject relation resource)) -(def can? impl.indexed/can?) -(def lookup-subjects impl.indexed/lookup-subjects) -(def lookup-resources impl.indexed/lookup-resources) -(def count-resources impl.indexed/count-resources) +(defn can? + ([db subject permission resource] + (impl.indexed/can? db subject permission resource)) + ([db demand] + (impl.indexed/can? db demand))) + +(defn lookup-subjects + [db query] + (impl.indexed/lookup-subjects db query)) + +(defn lookup-resources + [db query] + (impl.indexed/lookup-resources db query)) + +(defn count-resources + [db query] + (impl.indexed/count-resources db query)) (def ^:private forward-relationship-attr :eacl.v7.relationship/subject-type+relation+resource-type+resource) @@ -33,12 +46,43 @@ true (throw (Exception. "Unauthorized")))) +(defn- unknown-object! + [object-id] + (throw (ex-info (str "Unknown object: " (pr-str object-id) " does not resolve to an existing entity." + " Pass {:allow-tempids? true} to tx-relationship for same-transaction tempids.") + {:type :eacl/unknown-object + :object-id object-id}))) + (defn- object-id->eid-or-tempid - [db object-id] + "Resolves an object id to an existing eid. Unresolvable ids throw + :eacl/unknown-object unless :allow-tempids? is set, in which case strings, + negative longs and datomic.db.DbId values pass through as tempids for + same-transaction entity+relationship creation. Silent tempid pass-through + minted ghost entities on typo'd ids (audit §12). Positive numeric eids are + verified via datom presence — the transactor rejects unallocated eids anyway, + but with a raw :db.error/invalid-entity-id." + [db object-id {:keys [allow-tempids?]}] (cond - (number? object-id) object-id - (string? object-id) (or (d/entid db [:eacl/id object-id]) object-id) - :else (or (d/entid db object-id) object-id))) + (number? object-id) + (cond + (seq (d/datoms db :eavt object-id)) object-id + (and allow-tempids? (neg? object-id)) object-id + :else (unknown-object! object-id)) + + (string? object-id) + (or (d/entid db [:eacl/id object-id]) + (if allow-tempids? + object-id + (unknown-object! object-id))) + + (instance? datomic.db.DbId object-id) + (if allow-tempids? + object-id + (unknown-object! object-id)) + + :else + (or (d/entid db object-id) + (unknown-object! object-id)))) (defn- find-relation-eid [db resource-type relation-name subject-type] @@ -51,11 +95,11 @@ db resource-type relation-name subject-type)) (defn- resolve-relationship - [db {:keys [subject relation resource]}] + [db {:keys [subject relation resource]} opts] (let [subject-type (:type subject) - subject-eid (object-id->eid-or-tempid db (:id subject)) + subject-eid (object-id->eid-or-tempid db (:id subject) opts) resource-type (:type resource) - resource-eid (object-id->eid-or-tempid db (:id resource)) + resource-eid (object-id->eid-or-tempid db (:id resource) opts) relation-eid (find-relation-eid db resource-type relation subject-type)] (when-not relation-eid (throw @@ -114,10 +158,15 @@ false)) (defn find-one-relationship-id - "Returns the resolved tuple identity for an existing relationship, or nil." + "Returns the resolved tuple identity for an existing relationship, or nil. + A read: unresolvable endpoints mean no such relationship can exist -> nil." [db relationship] - (let [resolved (resolve-relationship db relationship)] - (when (relationship-exists? db resolved) + (let [resolved (try + (resolve-relationship db relationship {}) + (catch clojure.lang.ExceptionInfo e + (when-not (= :eacl/unknown-object (:type (ex-data e))) + (throw e))))] + (when (and resolved (relationship-exists? db resolved)) resolved))) (defn- find-relations @@ -235,17 +284,24 @@ (filter #(relationship-matches-filters? normalized-filters %))))) (defn tx-relationship - "Translate relationship data into v7 tuple writes." + "Translate relationship data into v7 tuple writes. + + Strict by default: endpoints must resolve to existing entities or this + throws :eacl/unknown-object. Pass {:allow-tempids? true} to let unresolvable + string ids / tempids pass through for same-transaction entity+relationship + creation (fixtures-style)." ([db subject relation resource] - (tx-relationship db (eacl/->Relationship subject relation resource))) + (tx-relationship db (eacl/->Relationship subject relation resource) {})) ([db relationship] - (add-relationship-txes (resolve-relationship db relationship)))) + (tx-relationship db relationship {})) + ([db relationship opts] + (add-relationship-txes (resolve-relationship db relationship opts)))) (defn tx-update-relationship "Relationship writes are implemented against v7 forward/reverse tuple indexes. - :touch is idempotent." + :touch is idempotent. Endpoints must resolve to existing entities." [db {:keys [operation relationship]}] - (let [resolved (resolve-relationship db relationship) + (let [resolved (resolve-relationship db relationship {}) exists? (relationship-exists? db resolved)] (case operation :touch diff --git a/src/eacl/datomic/impl/base.clj b/src/eacl/datomic/impl/base.clj index d1ea6f89..33e0cd77 100644 --- a/src/eacl/datomic/impl/base.clj +++ b/src/eacl/datomic/impl/base.clj @@ -116,17 +116,8 @@ (throw (ex-info "Invalid Permission spec. Expected one of {:relation name}, {:permission name}, {:arrow source :permission target} or {:arrow source :relation target}" {:spec spec})))) -(defn Relationship - "A Relationship between a subject and a resource via Relation. Copied from core2." - [subject relation-name resource] - ; :pre can be expensive. - {:pre [(:id subject) - (:type subject) - (keyword? relation-name) - (:id resource) - (:type resource)]} - {:eacl.relationship/resource-type (:type resource) - :eacl.relationship/resource (:id resource) - :eacl.relationship/relation-name relation-name - :eacl.relationship/subject-type (:type subject) - :eacl.relationship/subject (:id subject)}) +;; NOTE: the v6-era `Relationship` fn (emitting :eacl.relationship/* entity +;; attrs) was removed: those attributes do not exist in the v7 schema, so any +;; transact of its output failed with :db.error/not-an-entity. Use +;; eacl.datomic.impl/Relationship (data) + eacl.datomic.impl/tx-relationship +;; (tx-data) instead. diff --git a/src/eacl/datomic/impl/datalog.clj b/src/eacl/datomic/impl/datalog.clj deleted file mode 100644 index 128fbcf1..00000000 --- a/src/eacl/datomic/impl/datalog.clj +++ /dev/null @@ -1,153 +0,0 @@ -(ns eacl.datomic.impl.datalog - "Optimized EACL implementation with performance improvements" - (:require - [datomic.api :as d] - [eacl.core :as proto :refer [spice-object]] - ;[eacl.datomic.impl.base :as base] - [eacl.datomic.rules.optimized :as rules])) - -(defn lookup-subjects - "Optimized version of lookup-subjects" - [db - {:as filters - resource :resource - permission :permission - subject-type :subject/type - _subject-relation :subject/relation ; not currently supported. - limit :limit - offset :offset}] - {:pre [(:type resource) (:id resource)]} - (let [{resource-type :type - resource-id :id} resource - - resource-eid (d/entid db resource-id)] - ; Q: can we support dynamic object type resolution? - (assert resource-eid (str "lookup-subjects (object->entid " (pr-str resource) ") must resolve to a valid Datomic entid.")) - ; todo configurable type resolution. - ;(assert (= resource-type (:eacl/type resource-ent)) (str "Resource type does not match " resource-type ".")) - (let [subject-types+eids (->> (d/q '[:find ?subject-type ?subject - :in $ % ?subject-type ?permission ?resource-type ?resource-eid - :where - (has-permission ?subject-type ?subject ?permission ?resource-type ?resource-eid) - [(not= ?subject ?resource-eid)]] - db - rules/rules-lookup-subjects - subject-type - permission - resource-type - resource-eid)) - paginated-types+eids (cond->> subject-types+eids ; better name. - offset (drop offset) - limit (take limit)) - formatted (->> paginated-types+eids - (map (fn [[type id]] - (spice-object type id))))] - ;(prn 'paginated-types+eids paginated-types+eids) - ;(prn 'formatted formatted) - ; todo: subjects cursor is WIP. We still support offset & limit. - {:data formatted - :cursor nil}))) - -(defn can? - "can? uses recursive Datalog rules. Seems to be fast enough." - [db subject permission resource] - {:pre [subject - resource - (keyword? permission) - (:id subject) - (:type subject) - - (:id resource) - (:type resource)]} - (let [; todo hoist. - {subject-type :type - subject-ident :id} subject - - {resource-type :type - resource-ident :id} resource - - ; todo assert resource-ident - - ; do we need the d/entid here? - subject-eid (d/entid db subject-ident) - resource-eid (d/entid db resource-ident)] - (if-not (and subject-eid resource-eid) ; duplicated in Spiceomic. - false - (->> (d/q '[:find ?subject . - :in $ % ?subject-type ?subject ?perm ?resource-type ?resource - :where - (has-permission ?subject-type ?subject ?perm ?resource-type ?resource)] - db - rules/check-permission-rules - subject-type - subject-eid - permission - resource-type - resource-eid) - (boolean))))) - -(defn lookup-resources - ; outdated. - "Slow version of lookup-resources that reuses check-permission-rules. - Does not support cursor, only limit & offset." - [db {:as _query - subject :subject - permission :permission - resource-type :resource/type - limit :limit - cursor :cursor}] - {:pre [(:type subject) (:id subject)]} - (let [{subject-type :type - subject-ident :id} subject - - {cursor-path :path-index - cursor-resource :resource} cursor - - {cursor-resource-type :type - cursor-resource-eid :id} cursor-resource - - subject-eid (d/entid db subject-ident)] - - (assert subject-eid (pr-str "lookup-resources requires a valid subject :id that resolves to an eid via d/entid: " subject-ident ".")) - (let [resource-types+eids (->> (d/q '[:find ?resource-type ?resource - :in $ % ?subject-type ?subject-eid ?permission ?resource-type - :where - ;(has-permission ?subject-type ?subject-eid ?permission ?resource-type ?resource) - (has-permission ?subject-type ?subject-eid ?permission ?resource-type ?resource) - [(not= ?resource ?subject-eid)]] ; do we still need this? - db - rules/check-permission-rules ; rules-lookup-resources - subject-type - subject-eid - permission - resource-type)) - sorted-by-type+eid (sort resource-types+eids) - offsetted-results (if (and cursor-resource-type cursor-resource-eid) - (->> sorted-by-type+eid - (drop-while (fn [[resource-type resource-eid]] - ; design decision on <= vs < is to skip the matching value until next cursor is smarter. - (<= resource-eid cursor-resource-eid)))) - sorted-by-type+eid) - - paginated-types+eids (cond->> offsetted-results ; resource-types+eids - ; offset (drop offset) ; cursor should take care of this. - limit (take limit)) - ;sorted-by-type+eid (sort paginated-types+eids) - formatted (->> paginated-types+eids ; sorted-by-type+eid - (map (fn [[type eid]] (spice-object type eid)))) - last-result (when (seq formatted) (last formatted))] ; Fix: only get last when there are results - {:cursor (when last-result {:resource last-result}) ; Fix: only create cursor when there's a last result - :limit limit - :data formatted}))) - -(defn count-resources - "Temporary. Just calls lookup-resources. - Super inefficient due to the sort in lookup-resources, which is not required. - Any complete count will need to materialize full index. - Note that count-resources supports cursor, so if you count from a :cursor, - it will only return the results after the cursor." - [db query] - (->> (assoc query :limit Long/MAX_VALUE) - (lookup-resources db) - (:data) - (count))) \ No newline at end of file diff --git a/src/eacl/datomic/impl/indexed.clj b/src/eacl/datomic/impl/indexed.clj index 849b8c01..371d865c 100644 --- a/src/eacl/datomic/impl/indexed.clj +++ b/src/eacl/datomic/impl/indexed.clj @@ -51,15 +51,26 @@ (map (fn [[_ _ v]] (nth v 3)))))) (defn relation-datoms - "Returns relation datoms for the exact resource/relation name pair." + "Returns relation datoms for the exact resource/relation name pair, + for ANY subject-type keyword. + + Implemented as a seek + prefix take-while rather than a bounded + d/index-range: a keyword-sentinel range like [.. :a]..[.. :z] silently + misses subject types that collate outside it (uppercase-initial, + z-prefixed, and all namespaced keywords), which made those relations + invisible to permission evaluation. The attr-eid guard is mandatory — + seek-datoms iterates past the end of the attribute's index segment." [db resource-type relation-name] (if (and resource-type relation-name) - (let [start-tuple [resource-type relation-name :a] - end-tuple [resource-type relation-name :z]] - (d/index-range db - :eacl.relation/resource-type+relation-name+subject-type - start-tuple - end-tuple)) + (let [attr-eid (d/entid db :eacl.relation/resource-type+relation-name+subject-type)] + (->> (d/seek-datoms db :avet + :eacl.relation/resource-type+relation-name+subject-type + [resource-type relation-name]) + (take-while (fn [datom] + (and (= attr-eid (:a datom)) + (let [v (:v datom)] + (and (= resource-type (nth v 0)) + (= relation-name (nth v 1))))))))) [])) (defn find-relation-def @@ -101,12 +112,140 @@ (def permission-paths-cache (atom (cache/lru-cache-factory {} :threshold 1000))) -(defn evict-permission-paths-cache! [] - (reset! permission-paths-cache (cache/lru-cache-factory {} :threshold 1000))) +(def recursive-query-plan-cache + (atom (cache/lru-cache-factory {} :threshold 256))) + +(defn evict-permission-paths-cache! + "Manual override that clears both the permission-path and query-plan caches. + write-schema! calls it for immediate local hygiene; cross-peer and as-of + correctness rely on the :eacl/schema-version cache key, not on this. It is + also the recovery hatch after unsupported programmatic schema edits." + [] + (reset! permission-paths-cache (cache/lru-cache-factory {} :threshold 1000)) + (reset! recursive-query-plan-cache (cache/lru-cache-factory {} :threshold 256))) + +;; --- Schema-version cache scope (issue #74) --------------------------------- +;; +;; The path/plan caches are invalidated ONLY by eacl.datomic.schema/write-schema!, +;; which asserts a fresh :eacl/schema-version squuid on the schema singleton in +;; the same transaction as any definition change. Reading the stamp is a single +;; AVET lookup — O(log N), no history scans — and unrelated d/transact calls +;; leave it (and therefore every cache key) untouched, so relationship write +;; load can never evict or recompute paths. (The previous design derived a +;; digest from schema history per db basis, i.e. recomputed after every +;; transact — issue #74.) +;; +;; Contract: permission schema mutations MUST go through write-schema!. +;; Programmatic edits of relation/permission datoms (raw d/transact, d/with, +;; excision) do not bump the version, so caches and cursors may serve paths +;; computed from the pre-edit schema until the next write-schema! or a manual +;; evict-permission-paths-cache!. That trade-off is by design. +;; +;; The stamp lives in the database, not in peer memory, so: +;; - write-schema! on any peer invalidates every peer (the stored value changes); +;; - d/as-of views read the version that was current at that basis, giving +;; historic views their own cache slots (audit §3 fix, preserved); +;; - a squuid can never be re-asserted to an unchanged value by a concurrent +;; writer (no counter-elision race); +;; - cursor fingerprints survive process restarts. + +(def schema-version-attr + "Installed by eacl.datomic.schema/v7-schema; asserted by write-schema!." + :eacl/schema-version) + +(defn schema-version + "The schema-version squuid asserted by the most recent write-schema! visible + in this db value, or nil for databases that predate versioned schema writes. + A single AVET lookup on a one-datom attribute." + [db] + (when (d/entid db schema-version-attr) + (:v (first (d/datoms db :avet schema-version-attr))))) + +(def ^:private uncached-scope [::uncached]) + +(defn- classified-view + "Positively classifies a db value: :plain, :as-of, or nil for any view the + version stamp cannot be trusted on. d/filter views are excluded because their + predicates are arbitrary functions that may hide definition datoms without + hiding the version datom; d/since views hide old schema and are pointless to + cache; history views are not queryable schema states." + [db] + (cond + (d/is-history db) nil + (d/is-filtered db) nil + (some? (d/since-t db)) nil + (some? (d/as-of-t db)) :as-of + :else :plain)) + +(defn- schema-cache-scope + "Returns [database-id schema-version-string] for positively classified + plain/as-of db values, or a sentinel scope for anything else (filter, since, + history, unrecognized views) and for ANY failure. Sentinel scopes bypass the + caches entirely: every failure mode degrades to recomputation from the + queried db value — never to serving a stale entry." + [db] + (or (try + (when (classified-view db) + [(str (.id db)) (some-> (schema-version db) str)]) + (catch Throwable _ nil)) + uncached-scope)) + +(defn- uncached-scope? [scope] + (identical? uncached-scope scope)) + +(defn schema-version-stamp + "String form of the schema version for this db value, or nil when no version + has been written yet or the view cannot be classified (see schema-cache-scope). + Used for cursor fingerprints; nil forces the :p paths-digest comparison." + [db] + (let [scope (schema-cache-scope db)] + (when-not (uncached-scope? scope) + (second scope)))) + +(defn- fingerprint-digest + "128-bit hex digest of a value's printed representation (SHA-256 truncated)." + [x] + (let [md (java.security.MessageDigest/getInstance "SHA-256")] + (.update md (.getBytes (pr-str x) "UTF-8")) + (format "%032x" (java.math.BigInteger. 1 (java.util.Arrays/copyOf (.digest md) 16))))) + +(defn- cursor-fingerprint + "Two-part cursor fingerprint: :s is the schema-version stamp at mint time + (nil on unclassifiable views or pre-version databases), :p digests this + query's resolved paths/plan." + [db paths-or-plan] + {:s (schema-version-stamp db) + :p (fingerprint-digest paths-or-plan)}) + +(defn- check-cursor-fingerprint! + "Guards cursor resumption against schema changes. An equal schema-version + stamp means no write-schema! ran between mint and resume (programmatic datom + edits bypass the stamp — see the schema-version contract above), so the + cursor resumes on the fast path. A differing (or unavailable) stamp falls + back to comparing this query's resolved paths/plan: equal means the schema + change did not affect this query; different throws :eacl/stale-cursor instead + of silently mis-skipping via reordered :p path indices or replaying a stale + v3 :stack. Cursors minted before fingerprints existed are accepted with a + warning." + [db cursor paths-or-plan] + (when (map? cursor) + (if-let [f (:f cursor)] + (let [current-s (schema-version-stamp db)] + (when-not (and (:s f) current-s (= (:s f) current-s)) + (when-not (= (:p f) (fingerprint-digest paths-or-plan)) + (throw (ex-info "Stale cursor: the permission paths for this query changed since the cursor was minted. Restart pagination." + {:type :eacl/stale-cursor}))))) + (log/warn "Cursor without a schema fingerprint accepted (minted before fingerprints existed).")))) (defn- permission-paths-cache-key - [db resource-type permission-name] - [(.id db) resource-type permission-name]) + [scope resource-type permission-name] + (conj scope resource-type permission-name)) + +(defn- recursive-query-plan-cache-key + [scope root-node] + (conj scope root-node)) + +(def ^:private default-max-depth 50) (defn calc-permission-paths "Returns path maps with resolved relation eids. @@ -165,15 +304,20 @@ (defn get-permission-paths [db resource-type permission-name] - (let [cache @permission-paths-cache - cache-key (permission-paths-cache-key db resource-type permission-name)] - (if (cache/has? cache cache-key) - (do - (swap! permission-paths-cache cache/hit cache-key) - (cache/lookup cache cache-key)) - (let [paths (calc-permission-paths db resource-type permission-name)] - (swap! permission-paths-cache cache/miss cache-key paths) - paths)))) + (let [scope (schema-cache-scope db)] + (if (uncached-scope? scope) + ;; Unclassifiable view (filter/since/history) or digest failure: + ;; compute fresh from this db value; never share cache entries. + (calc-permission-paths db resource-type permission-name) + (let [cache @permission-paths-cache + cache-key (permission-paths-cache-key scope resource-type permission-name)] + (if (cache/has? cache cache-key) + (do + (swap! permission-paths-cache cache/hit cache-key) + (cache/lookup cache cache-key)) + (let [paths (calc-permission-paths db resource-type permission-name)] + (swap! permission-paths-cache cache/miss cache-key paths) + paths)))))) (defn- permission-query-node [resource-type permission-name] @@ -191,38 +335,101 @@ distinct vec)) +(defn- build-recursive-query-plan + [db root-node] + (let [ordered (volatile! []) + seen (volatile! #{}) + visiting (volatile! #{}) + recursive? (volatile! false)] + (letfn [(visit [node] + (when-not (contains? @seen node) + (vswap! seen conj node) + (vswap! ordered conj node) + (vswap! visiting conj node) + (doseq [dep (permission-query-dependencies db node)] + (when (contains? @visiting dep) + (vreset! recursive? true)) + (when-not (contains? @seen dep) + (visit dep))) + (vswap! visiting disj node)))] + (visit root-node) + (let [nodes @ordered + node-paths (into {} + (map (fn [[resource-type permission-name :as node]] + [node (vec (get-permission-paths db resource-type permission-name))])) + nodes) + seed-sources (into {} + (map (fn [[resource-type _ :as node]] + [node (->> (get node-paths node) + (map-indexed + (fn [path-idx path] + (case (:type path) + :relation {:kind :relation + :path-idx path-idx + :subject-type (:subject-type path) + :relation-eid (:relation-eid path)} + :arrow (when (:target-relation path) + {:kind :arrow-relation + :path-idx path-idx + :target-type (:target-type path) + :via-relation-eid (:via-relation-eid path) + :sub-paths (vec (:sub-paths path))}) + nil))) + (remove nil?) + vec)])) + nodes) + dependents (reduce + (fn [acc [resource-type _ :as node]] + (reduce + (fn [acc path] + (case (:type path) + :self-permission + (update acc + (permission-query-node resource-type (:target-permission path)) + (fnil conj []) + {:kind :copy + :node node}) + + :arrow + (if-let [target-permission (:target-permission path)] + (update acc + (permission-query-node (:target-type path) target-permission) + (fnil conj []) + {:kind :via + :node node + :intermediate-type (:target-type path) + :via-relation-eid (:via-relation-eid path)}) + acc) + + acc)) + acc + (get node-paths node))) + {} + nodes)] + {:root-node root-node + :nodes nodes + :recursive? @recursive? + :seed-sources seed-sources + :dependents dependents})))) + +(defn- recursive-query-plan + [db root-node] + (let [scope (schema-cache-scope db)] + (if (uncached-scope? scope) + (build-recursive-query-plan db root-node) + (let [cache-key (recursive-query-plan-cache-key scope root-node) + cache @recursive-query-plan-cache] + (if (cache/has? cache cache-key) + (do + (swap! recursive-query-plan-cache cache/hit cache-key) + (cache/lookup cache cache-key)) + (let [plan (build-recursive-query-plan db root-node)] + (swap! recursive-query-plan-cache cache/miss cache-key plan) + plan)))))) + (defn- recursive-permission-query? [db resource-type permission-name] - (let [root (permission-query-node resource-type permission-name)] - (loop [stack [{:node root - :deps (seq (permission-query-dependencies db root))}] - visited #{}] - (if-let [{:keys [node deps]} (peek stack)] - (if-let [dep (first deps)] - (cond - (= dep node) true - (some #(= dep (:node %)) stack) true - (contains? visited dep) (recur (conj (pop stack) {:node node - :deps (next deps)}) - visited) - :else (recur (conj (conj (pop stack) {:node node - :deps (next deps)}) - {:node dep - :deps (seq (permission-query-dependencies db dep))}) - visited)) - (recur (pop stack) (conj visited node))) - false)))) - -(defn- reachable-permission-query-nodes - [db root-node] - (loop [stack [root-node] - seen #{}] - (if-let [node (peek stack)] - (if (contains? seen node) - (recur (pop stack) seen) - (recur (into (pop stack) (permission-query-dependencies db node)) - (conj seen node))) - (vec seen)))) + (:recursive? (recursive-query-plan db (permission-query-node resource-type permission-name)))) (defn- extract-cursor-eid [cursor v1-key] @@ -269,112 +476,231 @@ (= subject-type (:subject-type %))) sub-paths)) -(defn- collect-subject-resources - [db subject-type subject-eid relation-eid resource-type] - (into #{} (subject->resources db - subject-type - subject-eid - relation-eid - resource-type - nil))) - -(defn- collect-resources-via-intermediates - [db intermediate-type intermediate-eids via-relation-eid resource-type] - (reduce (fn [acc intermediate-eid] - (into acc (subject->resources db - intermediate-type - intermediate-eid - via-relation-eid - resource-type - nil))) - #{} - intermediate-eids)) - -(defn- eval-recursive-permission-node - [db subject-type subject-eid [resource-type permission-name] current-results] - (reduce - (fn [acc path] - (case (:type path) - :relation - (if (= subject-type (:subject-type path)) - (into acc (collect-subject-resources db - subject-type - subject-eid - (:relation-eid path) - resource-type)) - acc) - - :self-permission - (into acc (get current-results - (permission-query-node resource-type (:target-permission path)) - #{})) - - :arrow - (let [intermediate-type (:target-type path) - intermediate-eids (if (:target-relation path) - (reduce (fn [intermediate-acc sub-path] - (into intermediate-acc - (subject->resources db - subject-type - subject-eid - (:relation-eid sub-path) - intermediate-type - nil))) - #{} - (matching-relation-sub-paths (:sub-paths path) subject-type)) - (get current-results - (permission-query-node intermediate-type (:target-permission path)) - #{}))] - (into acc (collect-resources-via-intermediates db - intermediate-type - intermediate-eids - (:via-relation-eid path) - resource-type))) - - acc)) - #{} - (get-permission-paths db resource-type permission-name))) - -(defn- solve-recursive-permission-results - [db subject-type subject-eid root-node] - (let [nodes (reachable-permission-query-nodes db root-node) - initial (zipmap nodes (repeat #{}))] - (loop [results initial] - (let [next-results (reduce (fn [acc node] - (assoc acc node - (eval-recursive-permission-node db - subject-type - subject-eid - node - results))) - {} - nodes)] - (if (= results next-results) - next-results - (recur next-results)))))) - -(defn- recursive-resource-eids - [db subject-type subject-eid permission resource-type] - (get (solve-recursive-permission-results db - subject-type - subject-eid - (permission-query-node resource-type permission)) - (permission-query-node resource-type permission) - #{})) - -(defn- slice-sorted-results - [sorted-eids cursor-eid limit] - (let [after-cursor (if cursor-eid - (drop-while #(<= % cursor-eid) sorted-eids) - sorted-eids)] - (if (>= limit 0) - (take limit after-cursor) - after-cursor))) +(defn- query-max-depth + [{:keys [max-depth]}] + (or max-depth default-max-depth)) + +(defn- max-depth-exceeded! + [state node resource-eid] + (throw + (ex-info (str "recursive permission query exceeded max depth " (:max-depth state)) + {:type ::max-depth-exceeded + :max-depth (:max-depth state) + :node node + :resource-eid resource-eid}))) + +(defn- push-tasks + [stack tasks] + (reduce conj stack (reverse (remove nil? tasks)))) + +(defn- matching-sub-path-descriptors + [subject-type sub-paths] + (filter #(= subject-type (:subject-type %)) sub-paths)) + +(defn- initial-recursive-tasks + [plan subject-type max-depth] + (->> (:nodes plan) + (mapcat + (fn [node] + (mapcat + (fn [seed] + (case (:kind seed) + :relation + (when (= subject-type (:subject-type seed)) + [{:kind :direct-stream + :node node + :relation-eid (:relation-eid seed) + :cursor nil + :depth max-depth}]) + + :arrow-relation + (->> (matching-sub-path-descriptors subject-type (:sub-paths seed)) + (map (fn [sub-path] + {:kind :subject-intermediate-stream + :node node + :intermediate-type (:target-type seed) + :subject-relation-eid (:relation-eid sub-path) + :via-relation-eid (:via-relation-eid seed) + :cursor nil + :depth max-depth})) + vec) + + [])) + (get-in plan [:seed-sources node])))) + vec)) + +(defn- init-recursive-state + [plan subject-type max-depth] + {:v 3 + :mode :recursive-forward + :max-depth max-depth + :stack (push-tasks [] (initial-recursive-tasks plan subject-type max-depth)) + :best-depth {} + :emitted #{} + :last nil}) + +(defn- recursive-state-for-query + [plan query] + (let [max-depth (query-max-depth query) + cursor (:cursor query) + subject-type (:type (:subject query))] + (cond + (nil? cursor) + (init-recursive-state plan subject-type max-depth) + + (= 3 (:v cursor)) + (do + (when (and (:max-depth cursor) (not= (:max-depth cursor) max-depth)) + (throw (ex-info "recursive query cursor max-depth does not match query max-depth" + {:cursor-max-depth (:max-depth cursor) + :query-max-depth max-depth}))) + cursor) + + :else + (throw (ex-info "unsupported cursor version for recursive lookup" + {:cursor-version (:v cursor)}))))) + +(defn- dependent-recursive-tasks + [plan node resource-eid depth] + (let [next-depth (dec depth)] + (mapv + (fn [dep] + (case (:kind dep) + :copy {:kind :copy-fact + :node (:node dep) + :resource-eid resource-eid + :depth next-depth} + :via {:kind :via-stream + :node (:node dep) + :intermediate-type (:intermediate-type dep) + :intermediate-eid resource-eid + :via-relation-eid (:via-relation-eid dep) + :cursor nil + :depth next-depth})) + (get-in plan [:dependents node] [])))) + +(defn- accept-recursive-fact + [plan state node resource-eid depth] + (let [prev-depth (get-in state [:best-depth node resource-eid] Long/MIN_VALUE)] + (cond + (<= depth prev-depth) + {:state state + :emit nil + :tasks []} + + (neg? depth) + (max-depth-exceeded! state node resource-eid) + + :else + (let [state' (assoc-in state [:best-depth node resource-eid] depth) + root-node? (= node (:root-node plan)) + already-out? (contains? (:emitted state') resource-eid) + [state'' emit] + (if (and root-node? (not already-out?)) + [(-> state' + (update :emitted (fnil conj #{}) resource-eid) + (assoc :last resource-eid)) + resource-eid] + [state' nil])] + {:state state'' + :emit emit + :tasks (dependent-recursive-tasks plan node resource-eid depth)})))) + +(defn- recursive-next-result + [db plan subject-type subject-eid state] + (loop [state state] + (if-let [task (peek (:stack state))] + (let [state' (update state :stack pop)] + (case (:kind task) + :copy-fact + (let [{:keys [state emit tasks]} + (accept-recursive-fact plan state' (:node task) (:resource-eid task) (:depth task)) + next-state (update state :stack push-tasks tasks)] + (if emit + {:state next-state + :emit emit} + (recur next-state))) + + :direct-stream + (let [resource-type (first (:node task)) + next-eid (first (subject->resources db + subject-type + subject-eid + (:relation-eid task) + resource-type + (:cursor task)))] + (if next-eid + (let [updated-task (assoc task :cursor next-eid) + {:keys [state emit tasks]} + (accept-recursive-fact plan state' (:node task) next-eid (:depth task)) + next-state (update state :stack push-tasks (concat tasks [updated-task]))] + (if emit + {:state next-state + :emit emit} + (recur next-state))) + (recur state'))) + + :subject-intermediate-stream + (let [next-intermediate-eid (first (subject->resources db + subject-type + subject-eid + (:subject-relation-eid task) + (:intermediate-type task) + (:cursor task)))] + (if next-intermediate-eid + (let [updated-task (assoc task :cursor next-intermediate-eid) + via-task {:kind :via-stream + :node (:node task) + :intermediate-type (:intermediate-type task) + :intermediate-eid next-intermediate-eid + :via-relation-eid (:via-relation-eid task) + :cursor nil + :depth (:depth task)} + next-state (update state' :stack push-tasks [via-task updated-task])] + (recur next-state)) + (recur state'))) + + :via-stream + (let [resource-type (first (:node task)) + next-eid (first (subject->resources db + (:intermediate-type task) + (:intermediate-eid task) + (:via-relation-eid task) + resource-type + (:cursor task)))] + (if next-eid + (let [updated-task (assoc task :cursor next-eid) + {:keys [state emit tasks]} + (accept-recursive-fact plan state' (:node task) next-eid (:depth task)) + next-state (update state :stack push-tasks (concat tasks [updated-task]))] + (if emit + {:state next-state + :emit emit} + (recur next-state))) + (recur state'))))) + {:state state + :emit nil + :done? true}))) + +(defn- recursive-page + [db plan subject-type subject-eid state limit] + (loop [state state + results []] + (if (and (>= limit 0) + (>= (count results) limit)) + {:state state + :results results} + (let [{:keys [state emit done?]} (recursive-next-result db plan subject-type subject-eid state)] + (cond + emit (recur state (conj results emit)) + done? {:state state + :results results} + :else (recur state results)))))) (declare traverse-permission-path lookup-subject-eids* can*) (defn traverse-permission-path-via-subject - [db subject-type subject-eid path resource-type cursor-eid intermediate-cursor-eid visited-paths] + [db subject-type subject-eid path resource-type cursor-eid intermediate-cursor-eid visited-paths _depth-left _max-depth] (case (:type path) :relation {:results (when (= subject-type (:subject-type path)) @@ -453,21 +779,23 @@ path-seqs (->> paths (map (fn [path] (:results - (traverse-permission-path-via-subject db + (traverse-permission-path-via-subject db subject-type subject-eid path resource-type cursor-eid nil - next-visited)))) + next-visited + default-max-depth + default-max-depth)))) (filter seq))] (if (seq path-seqs) (lazy-sort/lazy-fold2-merge-dedupe-sorted-by identity path-seqs) [])))))) (defn traverse-permission-path-reverse - [db resource-type resource-eid path subject-type cursor-eid intermediate-cursor-eid visited-paths] + [db resource-type resource-eid path subject-type cursor-eid intermediate-cursor-eid visited-paths depth-left max-depth] (case (:type path) :relation {:results (when (= subject-type (:subject-type path)) @@ -486,7 +814,9 @@ (:target-permission path) subject-type cursor-eid - (or visited-paths #{})) + (or visited-paths #{}) + (dec depth-left) + max-depth) :!state nil} :arrow @@ -524,13 +854,23 @@ target-permission subject-type cursor-eid - (or visited-paths #{}))))))))) + (or visited-paths #{}) + (dec depth-left) + max-depth)))))))) (defn- lookup-subject-eids* - [db resource-type resource-eid permission-name subject-type cursor-eid visited-states] + [db resource-type resource-eid permission-name subject-type cursor-eid visited-states depth-left max-depth] (let [state [resource-type resource-eid permission-name subject-type]] - (if (contains? visited-states state) + (cond + (contains? visited-states state) [] + + (neg? depth-left) + (max-depth-exceeded! {:max-depth max-depth} + (permission-query-node resource-type permission-name) + resource-eid) + + :else (let [next-visited (conj visited-states state) paths (get-permission-paths db resource-type permission-name) path-seqs (->> paths @@ -543,18 +883,28 @@ subject-type cursor-eid nil - next-visited)))) + next-visited + depth-left + max-depth)))) (filter seq))] (if (seq path-seqs) (lazy-sort/lazy-fold2-merge-dedupe-sorted-by identity path-seqs) []))))) (defn- can* - [db subject-type subject-eid permission resource-type resource-eid visited-states] + [db subject-type subject-eid permission resource-type resource-eid visited-states depth-left max-depth] (let [state [subject-type subject-eid permission resource-type resource-eid] paths (get-permission-paths db resource-type permission)] - (if (contains? visited-states state) + (cond + (contains? visited-states state) false + + (neg? depth-left) + (max-depth-exceeded! {:max-depth max-depth} + (permission-query-node resource-type permission) + resource-eid) + + :else (let [next-visited (conj visited-states state)] (boolean (some @@ -577,7 +927,9 @@ (:target-permission path) resource-type resource-eid - next-visited) + next-visited + (dec depth-left) + max-depth) :arrow (let [intermediate-type (:target-type path) @@ -609,19 +961,26 @@ (:target-permission path) intermediate-type intermediate-eid - next-visited)) + next-visited + (dec depth-left) + max-depth)) intermediate-eids))))) paths)))))) (defn can? - [db subject permission resource] - (let [subject-type (:type subject) - subject-eid (d/entid db (:id subject)) - resource-type (:type resource) - resource-eid (d/entid db (:id resource))] - (if (or (nil? subject-eid) (nil? resource-eid)) - false - (can* db subject-type subject-eid permission resource-type resource-eid #{})))) + ([db subject permission resource] + (can? db {:subject subject + :permission permission + :resource resource})) + ([db {:keys [subject permission resource max-depth]}] + (let [subject-type (:type subject) + subject-eid (when (some? (:id subject)) (d/entid db (:id subject))) + resource-type (:type resource) + resource-eid (when (some? (:id resource)) (d/entid db (:id resource))) + max-depth (or max-depth default-max-depth)] + (if (or (nil? subject-eid) (nil? resource-eid)) + false + (can* db subject-type subject-eid permission resource-type resource-eid #{} max-depth max-depth))))) (def ^:private forward-direction {:anchor-key :subject @@ -642,9 +1001,10 @@ (let [{:keys [anchor-key traverse-fn v1-cursor-key perm-type-fn]} direction anchor (get query anchor-key) anchor-type (:type anchor) - anchor-eid (d/entid db (:id anchor)) + anchor-eid (when (some? (:id anchor)) (d/entid db (:id anchor))) cursor (:cursor query) cursor-eid (extract-cursor-eid cursor v1-cursor-key) + max-depth (query-max-depth query) permission (:permission query) perm-type (perm-type-fn query) result-type-key (if (= anchor-key :subject) :resource/type :subject/type) @@ -664,7 +1024,9 @@ result-type cursor-eid intermediate-cursor-eid - #{}) + #{} + max-depth + max-depth) {:results [] :!state nil})] {:idx idx :results results @@ -676,53 +1038,47 @@ :path-results path-results})) (defn- recursive-forward-lookup - [db query] - (let [{:keys [subject permission cursor limit] :or {limit 1000}} query - subject-type (:type subject) - subject-eid (d/entid db (:id subject)) + [db {:as query :keys [subject permission limit] :or {limit 1000}}] + (let [subject-type (:type subject) + subject-eid (when (some? (:id subject)) (d/entid db (:id subject))) resource-type (:resource/type query) - cursor-eid (extract-cursor-eid cursor :resource)] + root-node (permission-query-node resource-type permission) + plan (recursive-query-plan db root-node) + _ (check-cursor-fingerprint! db (:cursor query) plan) + state (recursive-state-for-query plan query)] (if subject-eid - (let [sorted-eids (sort (recursive-resource-eids db - subject-type - subject-eid - permission - resource-type)) - limited-eids (doall (slice-sorted-results sorted-eids cursor-eid limit)) - items (mapv #(spice-object resource-type %) limited-eids) - last-eid (:id (last items))] - {:data items - :cursor (build-v2-cursor cursor last-eid [] :resource)}) + (let [{:keys [state results]} + (recursive-page db plan subject-type subject-eid state limit)] + {:data (mapv #(spice-object resource-type %) results) + :cursor (assoc state :f (cursor-fingerprint db plan))}) {:data [] - :cursor (build-v2-cursor cursor nil [] :resource)}))) + :cursor (assoc state :f (cursor-fingerprint db plan))}))) (defn- recursive-forward-count - [db {:as query :keys [limit cursor] :or {limit -1}}] - (let [subject (:subject query) - subject-type (:type subject) - subject-eid (d/entid db (:id subject)) + [db {:as query :keys [limit subject permission] :or {limit -1}}] + (let [subject-type (:type subject) + subject-eid (when (some? (:id subject)) (d/entid db (:id subject))) resource-type (:resource/type query) - permission (:permission query) - cursor-eid (extract-cursor-eid cursor :resource)] + root-node (permission-query-node resource-type permission) + plan (recursive-query-plan db root-node) + _ (check-cursor-fingerprint! db (:cursor query) plan) + state (recursive-state-for-query plan query)] (if subject-eid - (let [sorted-eids (sort (recursive-resource-eids db - subject-type - subject-eid - permission - resource-type)) - counted-eids (doall (slice-sorted-results sorted-eids cursor-eid limit)) - last-eid (last counted-eids)] - {:count (count counted-eids) + (let [{:keys [state results]} + (recursive-page db plan subject-type subject-eid state limit)] + {:count (count results) :limit limit - :cursor (build-v2-cursor cursor last-eid [] :resource)}) + :cursor (assoc state :f (cursor-fingerprint db plan))}) {:count 0 :limit limit - :cursor (build-v2-cursor cursor nil [] :resource)}))) + :cursor (assoc state :f (cursor-fingerprint db plan))}))) (defn- lookup [db direction query] - (let [{:keys [result-type-fn v1-cursor-key]} direction + (let [{:keys [result-type-fn v1-cursor-key perm-type-fn]} direction {:keys [limit cursor] :or {limit 1000}} query + paths (get-permission-paths db (perm-type-fn query) (:permission query)) + _ (check-cursor-fingerprint! db cursor paths) {:keys [results path-results]} (lazy-merged-lookup db direction query) limited-results (if (>= limit 0) (take limit results) @@ -731,24 +1087,37 @@ items (doall (map #(spice-object result-type %) limited-results)) last-eid (:id (last items))] {:data items - :cursor (build-v2-cursor cursor last-eid path-results v1-cursor-key)})) + :cursor (assoc (build-v2-cursor cursor last-eid path-results v1-cursor-key) + :f (cursor-fingerprint db paths))})) (defn lookup-resources + "Enumerates resources of :resource/type that :subject holds :permission on. + + Non-recursive schemas return in ascending internal-eid order with v2 cursors. + Recursive schemas use a stable-discovery-order engine whose v3 cursor IS the + recursion state: it grows O(results-emitted-so-far) (~48 bytes/resource) and + embeds internal state, because exact cross-page deduplication requires the + emitted set. Known limitation (audit §6) pending a cursor-state redesign — + prefer bounded pagination sessions on recursive schemas." [db query] (if (recursive-permission-query? db (:resource/type query) (:permission query)) (recursive-forward-lookup db query) (lookup db forward-direction query))) (defn lookup-subjects + "Unknown/missing resources return an empty page (SpiceDB-consistent), + matching can? -> false; assertion-based rejection disappears under + *assert* false and crashed with an untyped AssertionError." [db query] - {:pre [(:type (:resource query)) (:id (:resource query))]} (lookup db reverse-direction query)) (defn count-resources [db {:as query :keys [limit cursor] :or {limit -1}}] (if (recursive-permission-query? db (:resource/type query) (:permission query)) (recursive-forward-count db query) - (let [{:keys [results path-results]} (lazy-merged-lookup db forward-direction query) + (let [paths (get-permission-paths db (:resource/type query) (:permission query)) + _ (check-cursor-fingerprint! db cursor paths) + {:keys [results path-results]} (lazy-merged-lookup db forward-direction query) limited-results (if (>= limit 0) (take limit results) results) @@ -756,11 +1125,14 @@ last-eid (last counted)] {:count (count counted) :limit limit - :cursor (build-v2-cursor cursor last-eid path-results :resource)}))) + :cursor (assoc (build-v2-cursor cursor last-eid path-results :resource) + :f (cursor-fingerprint db paths))}))) (defn count-subjects [db {:as query :keys [limit cursor] :or {limit -1}}] - (let [{:keys [results path-results]} (lazy-merged-lookup db reverse-direction query) + (let [paths (get-permission-paths db (:type (:resource query)) (:permission query)) + _ (check-cursor-fingerprint! db cursor paths) + {:keys [results path-results]} (lazy-merged-lookup db reverse-direction query) limited-results (if (>= limit 0) (take limit results) results) @@ -768,4 +1140,5 @@ last-eid (last counted)] {:count (count counted) :limit limit - :cursor (build-v2-cursor cursor last-eid path-results :subject)})) + :cursor (assoc (build-v2-cursor cursor last-eid path-results :subject) + :f (cursor-fingerprint db paths))})) diff --git a/src/eacl/datomic/rules.clj b/src/eacl/datomic/rules.clj deleted file mode 100644 index 93cdd573..00000000 --- a/src/eacl/datomic/rules.clj +++ /dev/null @@ -1,551 +0,0 @@ -(ns eacl.datomic.rules) - -;(def check-permission-rules -; "Can only be used for can? where type + id of both subject & resource object are provided." -; '[;; Reachability rules to traverse relationships: -; [(reachable ?resource ?subject) -; [(tuple ?resource ?subject) ?resource+subject] -; [?relationship :eacl.relationship/resource+subject ?resource+subject] -; -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?subject]] -; [(reachable ?resource ?subject) -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?mid] -; (reachable ?mid ?subject)] ; note inversion to traverse. -; -; ;; Direct permission check (copied and adapted from core2) -; [(has-permission ?subject ?permission-name ?resource) -; -; [(tuple ?resource ?relation-name-in-tuple ?subject) ?resource+rel-name+subject] -; [?relationship :eacl.relationship/resource+relation-name+subject ?resource+rel-name+subject] -; -; [?relationship :eacl.relationship/resource ?resource] ; subject has some relationship TO the resource -; [?relationship :eacl.relationship/relation-name ?relation-name-in-tuple] -; [?relationship :eacl.relationship/subject ?subject] ; subject of the relationship tuple -; -; ;; Permission definition: ?relation-name-in-perm-def grants ?permission-name on ?resource-type -; [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; THIS IS THE DIRECT GRANT -; -; ;; Match the relation name from the relationship tuple with the one in permission definition -; [(= ?relation-name-in-tuple ?relation-name-in-perm-def)] -; [(not= ?subject ?resource)]] -; -; ;; Indirect permission inheritance (copied from core2 - may need review/replacement with arrows) -; ;; This rule means: ?subject gets ?permission-name on ?resource if: -; ;; 1. A permission definition exists: for ?resource-type, ?relation-name-in-perm-def grants ?permission-name. -; ;; 2. ?resource has a relationship (as a subject of the tuple) via ?relation-name-in-perm-def to some ?target. -; ;; (e.g. doc D is "subject" of relation "group" to group G: D --group--> G) -; ;; 3. ?subject can "reach" that ?target (e.g. user U is member of group G). -; -; [(has-permission ?subject ?permission-name ?resource) -; ;; Permission definition -; [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; ; can these move down for speed? -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; Direct relation specified in perm -; -; ;; Structural relationship: ?resource is linked to ?target via ?relation-name-in-perm-def -; [(tuple ?target ?relation-name-in-perm-def ?resource) ?target+relation+resource] -; [?structural-rel :eacl.relationship/resource+relation-name+subject ?target+relation+resource] -; -; [?structural-rel :eacl.relationship/subject ?resource] -; [?structural-rel :eacl.relationship/relation-name ?relation-name-in-perm-def] -; [?structural-rel :eacl.relationship/resource ?target] -; -; (reachable ?target ?subject) ; User must be able to reach the target of the structural relationship -; [(not= ?subject ?resource)]] -; -; ;; Arrow permission rule: ?subject gets ?perm-name-on-this-resource if it has ?perm-name-on-related on an intermediate resource -; ;; Example: User U gets :admin on VPC_X if VPC_X --:account--> ACC_Y and User U has :admin on ACC_Y. -; ;; MODIFIED based on user feedback: Rule now expects intermediate --via-relation-name--> this-resource -; ;; Example: User U gets :view on SERVER_X if ACC_Y --:account--> SERVER_X and User U has :admin on ACC_Y. -; [(has-permission ?subject ?perm-name-on-this-resource ?this-resource) -; ;; 1. Find an arrow permission definition for this-resource-type and perm-name-on-this-resource -; [(tuple ?this-resource-type -; ?via-relation-name -; ?perm-on-related -; ?perm-name-on-this-resource) ?res-type+relation+related-perm+permission] -; [?arrow-perm-def -; :eacl.arrow-permission/resource-type+source-relation-name+target-permission-name+permission-name -; ?res-type+relation+related-perm+permission] -; -; [?arrow-perm-def :eacl.arrow-permission/resource-type ?this-resource-type] -; [?arrow-perm-def :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] -; [?arrow-perm-def :eacl.arrow-permission/source-relation-name ?via-relation-name] ; e.g., :account (the relation name specified in Permission) -; [?arrow-perm-def :eacl.arrow-permission/target-permission-name ?perm-on-related] ; e.g., :admin (on the intermediate/account) -; -; ;; 2. Find intermediate resource: ?intermediate-resource --via-relation-name--> ?this-resource -; [(tuple ?this-resource ?via-relation-name ?intermediate-resource) ?resource+relation+mid-resource] -; [?rel-linking-resources :eacl.relationship/resource+relation-name+subject ?resource+relation+mid-resource] -; [?rel-linking-resources :eacl.relationship/subject ?intermediate-resource] ; e.g., account is subject of tuple -; [?rel-linking-resources :eacl.relationship/relation-name ?via-relation-name] ; relation is :account -; [?rel-linking-resources :eacl.relationship/resource ?this-resource] ; e.g., server/vpc is resource of tuple -; -; ;; 3. Subject must have the target permission on the intermediate resource (recursive call) -; (has-permission ?subject ?perm-on-related ?intermediate-resource) -; [(not= ?subject ?this-resource)] ; Exclude self-references for safety -; ;; Ensure the intermediate resource is not the same as the subject to prevent some loops, -; ;; though main cycle prevention relies on data structure or more complex rule logic if needed. -; [(not= ?subject ?intermediate-resource)] -; ;; Ensure this-resource is not the same as intermediate for simple arrows like A -> B -; [(not= ?this-resource ?intermediate-resource)]]]) - -;(defn build-slow-rules [resource-type-attr] -; [;; Reachability rules to traverse relationships: -; '[(reachable ?resource ?subject) -; [(tuple ?resource ?subject) ?resource+subject] -; [?relationship :eacl.relationship/resource+subject ?resource+subject]] -; -; ;[?relationship :eacl.relationship/resource ?resource] -; ;[?relationship :eacl.relationship/subject ?subject]] -; '[(reachable ?resource ?subject) -; -; [(tuple ?resource ?mid) ?resource+mid] -; [?relationship :eacl.relationship/resource+subject ?resource+mid] ; range query? -; -; ; todo can we use tuple here? -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?mid] -; (reachable ?mid ?subject)] -; -; ;; Direct permission check (copied and adapted from core2) -; '[(has-permission ?subject ?permission-name ?resource) -; -; [(tuple ?resource ?relation-name-in-tuple ?subject) ?resource+rel-name+subject] -; [?relationship :eacl.relationship/resource+relation-name+subject ?resource+rel-name+subject] -; -; [?relationship :eacl.relationship/resource ?resource] ; subject has some relationship TO the resource -; [?relationship :eacl.relationship/relation-name ?relation-name-in-tuple] -; [?relationship :eacl.relationship/subject ?subject] ; subject of the relationship tuple -; -; ;; Permission definition: ?relation-name-in-perm-def grants ?permission-name on ?resource-type -; [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; THIS IS THE DIRECT GRANT -; -; ;; Match the relation name from the relationship tuple with the one in permission definition -; [(= ?relation-name-in-tuple ?relation-name-in-perm-def)] -; [(not= ?subject ?resource)] ; can we avoid this? -; [?resource :eacl/type ?resource-type]] ; this is super slow. different rules WIP. -; -; ;; Indirect permission inheritance (copied from core2 - may need review/replacement with arrows) -; ;; This rule means: ?subject gets ?permission-name on ?resource if: -; ;; 1. A permission definition exists: for ?resource-type, ?relation-name-in-perm-def grants ?permission-name. -; ;; 2. ?resource has a relationship (as a subject of the tuple) via ?relation-name-in-perm-def to some ?target. -; ;; (e.g. doc D is "subject" of relation "group" to group G: D --group--> G) -; ;; 3. ?subject can "reach" that ?target (e.g. user U is member of group G). -; (into -; ['(has-permission ?subject ?permission-name ?resource)] -; -; '[;; Permission definition -; [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; Direct relation specified in perm -; -; ;; Structural relationship: ?resource is linked to ?target via ?relation-name-in-perm-def -; [(tuple ?target ?relation-name-in-perm-def ?resource) ?target+relation+resource] -; [?structural-rel :eacl.relationship/resource+relation-name+subject ?target+relation+resource] -; -; [?structural-rel :eacl.relationship/subject ?resource] -; [?structural-rel :eacl.relationship/relation-name ?relation-name-in-perm-def] -; [?structural-rel :eacl.relationship/resource ?target] -; -; (reachable ?target ?subject) ; User must be able to reach the target of the structural relationship -; [(not= ?subject ?resource)] -; [?resource :eacl/type ?resource-type]]) -; -; ;; Arrow permission rule: ?subject gets ?perm-name-on-this-resource if it has ?perm-name-on-related on an intermediate resource -; ;; Example: User U gets :admin on VPC_X if VPC_X --:account--> ACC_Y and User U has :admin on ACC_Y. -; ;; MODIFIED based on user feedback: Rule now expects intermediate --via-relation-name--> this-resource -; ;; Example: User U gets :view on SERVER_X if ACC_Y --:account--> SERVER_X and User U has :admin on ACC_Y. -; '[(has-permission ?subject ?perm-name-on-this-resource ?this-resource) -; -; ;; 1. Find an arrow permission definition for this-resource-type and perm-name-on-this-resource -; [(tuple ?this-resource-type -; ?via-relation-name -; ?perm-on-related -; ?perm-name-on-this-resource) ?res-type+relation+related-perm+permission] -; [?arrow-perm-def -; :eacl.arrow-permission/resource-type+source-relation-name+target-permission-name+permission-name -; ?res-type+relation+related-perm+permission] -; -; ; can these move down for speed, or be decoupled in a 2nd phase? -; [?arrow-perm-def :eacl.arrow-permission/resource-type ?this-resource-type] -; [?arrow-perm-def :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] -; [?arrow-perm-def :eacl.arrow-permission/source-relation-name ?via-relation-name] ; e.g., :account (the relation name specified in Permission) -; [?arrow-perm-def :eacl.arrow-permission/target-permission-name ?perm-on-related] ; e.g., :admin (on the intermediate/account) -; -; ;; 2. Find intermediate resource: ?intermediate-resource --via-relation-name--> ?this-resource -; [(tuple ?this-resource ?via-relation-name ?intermediate-resource) ?resource+relation+mid-resource] -; [?rel-linking-resources :eacl.relationship/resource+relation-name+subject ?resource+relation+mid-resource] -; -; [?rel-linking-resources :eacl.relationship/subject ?intermediate-resource] ; e.g., account is subject of tuple -; [?rel-linking-resources :eacl.relationship/relation-name ?via-relation-name] ; relation is :account -; [?rel-linking-resources :eacl.relationship/resource ?this-resource] ; e.g., server/vpc is resource of tuple -; -; ;; 3. Subject must have the target permission on the intermediate resource (recursive call) -; (has-permission ?subject ?perm-on-related ?intermediate-resource) -; [(not= ?subject ?this-resource)] ; Exclude self-references for safety -; ;; Ensure the intermediate resource is not the same as the subject to prevent some loops, -; ;; though main cycle prevention relies on data structure or more complex rule logic if needed. -; [(not= ?subject ?intermediate-resource)] -; ;; Ensure this-resource is not the same as intermediate for simple arrows like A -> B -; [(not= ?this-resource ?intermediate-resource)] -; [?this-resource :eacl/type ?this-resource-type]]]) ; this is super slow. different rules WIP.]]) - -(def slow-lookup-rules (build-slow-rules :eacl/type)) - -(def rules-lookup-subjects - '[;; Reachability rules to traverse relationships: - [(reachable ?resource ?subject) ; I think we need types here for speed. - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject] - - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?subject]] - - [(reachable ?resource ?subject) - - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] ; range query? - - ; todo can we use tuple here? - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?mid] - (reachable ?mid ?subject)] - - ;; Direct permission check (copied and adapted from core2) - [(has-permission ?subject-type ?subject ?permission-name ?resource) - - [(tuple ?resource ?relation-name-in-tuple ?subject) ?resource+rel-name+subject] - [?relationship :eacl.relationship/resource+relation-name+subject ?resource+rel-name+subject] - - [?relationship :eacl.relationship/resource ?resource] ; subject has some relationship TO the resource - [?relationship :eacl.relationship/relation-name ?relation-name-in-tuple] - [?relationship :eacl.relationship/subject ?subject] ; subject of the relationship tuple - - ;; Permission definition: ?relation-name-in-perm-def grants ?permission-name on ?resource-type - [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] - - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; THIS IS THE DIRECT GRANT - - ;; Match the relation name from the relationship tuple with the one in permission definition - [(= ?relation-name-in-tuple ?relation-name-in-perm-def)] - [(not= ?subject ?resource)] ; can we avoid this? - [?subject :eacl/type ?subject-type]] ; this is super slow. different rules WIP. - - ;; Indirect permission inheritance (copied from core2 - may need review/replacement with arrows) - ;; This rule means: ?subject gets ?permission-name on ?resource if: - ;; 1. A permission definition exists: for ?resource-type, ?relation-name-in-perm-def grants ?permission-name. - ;; 2. ?resource has a relationship (as a subject of the tuple) via ?relation-name-in-perm-def to some ?target. - ;; (e.g. doc D is "subject" of relation "group" to group G: D --group--> G) - ;; 3. ?subject can "reach" that ?target (e.g. user U is member of group G). - - [(has-permission ?subject-type ?subject ?permission-name ?resource) - ;; Permission definition - [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] - - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; Direct relation specified in perm - - ;; Structural relationship: ?resource is linked to ?target via ?relation-name-in-perm-def - [(tuple ?target ?relation-name-in-perm-def ?resource) ?target+relation+resource] - [?structural-rel :eacl.relationship/resource+relation-name+subject ?target+relation+resource] - - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/relation-name ?relation-name-in-perm-def] - [?structural-rel :eacl.relationship/resource ?target] - - (reachable ?target ?subject) ; User must be able to reach the target of the structural relationship - [(not= ?subject ?resource)] - [?subject :eacl/type ?subject-type]] - - ;; Arrow permission rule: ?subject gets ?perm-name-on-this-resource if it has ?perm-name-on-related on an intermediate resource - ;; Example: User U gets :admin on VPC_X if VPC_X --:account--> ACC_Y and User U has :admin on ACC_Y. - ;; MODIFIED based on user feedback: Rule now expects intermediate --via-relation-name--> this-resource - ;; Example: User U gets :view on SERVER_X if ACC_Y --:account--> SERVER_X and User U has :admin on ACC_Y. - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource) - - ; this order looks wrong. - ;; 1. Find an arrow permission definition for this-resource-type and perm-name-on-this-resource - [(tuple ?this-resource-type - ?via-relation-name - ?perm-on-related - ?perm-name-on-this-resource) ?res-type+relation+related-perm+permission] - [?arrow-perm-def - :eacl.arrow-permission/resource-type+source-relation-name+target-permission-name+permission-name - ?res-type+relation+related-perm+permission] - - ; can these move down for speed, or be decoupled in a 2nd phase? - [?arrow-perm-def :eacl.arrow-permission/resource-type ?this-resource-type] - [?arrow-perm-def :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm-def :eacl.arrow-permission/source-relation-name ?via-relation-name] ; e.g., :account (the relation name specified in Permission) - [?arrow-perm-def :eacl.arrow-permission/target-permission-name ?perm-on-related] ; e.g., :admin (on the intermediate/account) - - ;; 2. Find intermediate resource: ?intermediate-resource --via-relation-name--> ?this-resource - [(tuple ?this-resource ?via-relation-name ?intermediate-resource) ?resource+relation+mid-resource] - [?rel-linking-resources :eacl.relationship/resource+relation-name+subject ?resource+relation+mid-resource] - - [?rel-linking-resources :eacl.relationship/subject ?intermediate-resource] ; e.g., account is subject of tuple - [?rel-linking-resources :eacl.relationship/relation-name ?via-relation-name] ; relation is :account - [?rel-linking-resources :eacl.relationship/resource ?this-resource] ; e.g., server/vpc is resource of tuple - - ;; 3. Subject must have the target permission on the intermediate resource (recursive call) - (has-permission ?subject-type ?subject ?perm-on-related ?intermediate-resource) - [(not= ?subject ?this-resource)] ; Exclude self-references for safety - ;; Ensure the intermediate resource is not the same as the subject to prevent some loops, - ;; though main cycle prevention relies on data structure or more complex rule logic if needed. - [(not= ?subject ?intermediate-resource)] - ;; Ensure this-resource is not the same as intermediate for simple arrows like A -> B - [(not= ?this-resource ?intermediate-resource)] - ; TODO: this-resource looks dubious here. do we need it? - [?this-resource :eacl/type ?this-resource-type]]]) - -(def rules-lookup-resources - ; resource look has known subject Type + ID, and known resource type. - '[;; Reachability rules to traverse relationships: - [(reachable ?resource ?subject) ; I think we need types here for speed. - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject] - - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?subject]] - - [(reachable ?resource ?subject) - - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] ; range query? - - ; todo can we use tuple here? - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?mid] - (reachable ?mid ?subject)] - - ;; Direct permission check (copied and adapted from core2) - [(has-permission ?subject ?permission-name ?resource-type ?resource) - - [(tuple ?resource ?relation-name-in-tuple ?subject) ?resource+rel-name+subject] - [?relationship :eacl.relationship/resource+relation-name+subject ?resource+rel-name+subject] - - [?relationship :eacl.relationship/resource ?resource] ; subject has some relationship TO the resource - [?relationship :eacl.relationship/relation-name ?relation-name-in-tuple] - [?relationship :eacl.relationship/subject ?subject] ; subject of the relationship tuple - - ;; Permission definition: ?relation-name-in-perm-def grants ?permission-name on ?resource-type - [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] - - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; THIS IS THE DIRECT GRANT - - ;; Match the relation name from the relationship tuple with the one in permission definition - [(= ?relation-name-in-tuple ?relation-name-in-perm-def)] - [(not= ?subject ?resource)] ; can we avoid this? - [?resource :eacl/type ?resource-type]] - - ;; Indirect permission inheritance (copied from core2 - may need review/replacement with arrows) - ;; This rule means: ?subject gets ?permission-name on ?resource if: - ;; 1. A permission definition exists: for ?resource-type, ?relation-name-in-perm-def grants ?permission-name. - ;; 2. ?resource has a relationship (as a subject of the tuple) via ?relation-name-in-perm-def to some ?target. - ;; (e.g. doc D is "subject" of relation "group" to group G: D --group--> G) - ;; 3. ?subject can "reach" that ?target (e.g. user U is member of group G). - - [(has-permission ?subject ?permission-name ?resource-type ?resource) - ;; Permission definition - ;; order looks dubious - [(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] - - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; Direct relation specified in perm - - ;; Structural relationship: ?resource is linked to ?target via ?relation-name-in-perm-def - [(tuple ?target ?relation-name-in-perm-def ?resource) ?target+relation+resource] - [?structural-rel :eacl.relationship/resource+relation-name+subject ?target+relation+resource] - - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/relation-name ?relation-name-in-perm-def] - [?structural-rel :eacl.relationship/resource ?target] - - (reachable ?target ?subject) ; User must be able to reach the target of the structural relationship - [(not= ?subject ?resource)] - [?resource :eacl/type ?this-resource-type]] ; is this correct? - - ;; Arrow permission rule: ?subject gets ?perm-name-on-this-resource if it has ?perm-name-on-related on an intermediate resource - ;; Example: User U gets :admin on VPC_X if VPC_X --:account--> ACC_Y and User U has :admin on ACC_Y. - ;; MODIFIED based on user feedback: Rule now expects intermediate --via-relation-name--> this-resource - ;; Example: User U gets :view on SERVER_X if ACC_Y --:account--> SERVER_X and User U has :admin on ACC_Y. - [(has-permission ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) - - ;; 1. Find an arrow permission definition for this-resource-type and perm-name-on-this-resource - [(tuple ?this-resource-type - ?via-relation-name - ?perm-on-related - ?perm-name-on-this-resource) ?res-type+relation+related-perm+permission] - [?arrow-perm-def - :eacl.arrow-permission/resource-type+source-relation-name+target-permission-name+permission-name - ?res-type+relation+related-perm+permission] - - ; can these move down for speed, or be decoupled in a 2nd phase? - [?arrow-perm-def :eacl.arrow-permission/resource-type ?this-resource-type] - [?arrow-perm-def :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm-def :eacl.arrow-permission/source-relation-name ?via-relation-name] ; e.g., :account (the relation name specified in Permission) - [?arrow-perm-def :eacl.arrow-permission/target-permission-name ?perm-on-related] ; e.g., :admin (on the intermediate/account) - - ;; 2. Find intermediate resource: ?intermediate-resource --via-relation-name--> ?this-resource - [(tuple ?this-resource ?via-relation-name ?intermediate-resource) ?resource+relation+mid-resource] - [?rel-linking-resources :eacl.relationship/resource+relation-name+subject ?resource+relation+mid-resource] - - [?rel-linking-resources :eacl.relationship/subject ?intermediate-resource] ; e.g., account is subject of tuple - [?rel-linking-resources :eacl.relationship/relation-name ?via-relation-name] ; relation is :account - [?rel-linking-resources :eacl.relationship/resource ?this-resource] ; e.g., server/vpc is resource of tuple - - [?intermediate-resource :eacl/type ?intermediate-resource-type] ; do we need this? - - ;; 3. Subject must have the target permission on the intermediate resource (recursive call) - (has-permission ?subject ?perm-on-related ?intermediate-resource-type ?intermediate-resource) - [(not= ?subject ?this-resource)] ; Exclude self-references for safety - ;; Ensure the intermediate resource is not the same as the subject to prevent some loops, - ;; though main cycle prevention relies on data structure or more complex rule logic if needed. - [(not= ?subject ?intermediate-resource)] - ;; Ensure this-resource is not the same as intermediate for simple arrows like A -> B - [(not= ?this-resource ?intermediate-resource)] - [?this-resource :eacl/type ?this-resource-type]]]) - -;(def rules-lookup-subjects -; ; lookup-subjects knows resource & subject type which implies knowing resource type. -; '[;; Reachability rules to traverse relationships: -; [(reachable ?resource ?subject) -; ; todo use reachable tuple -; [(tuple ?resource ?subject) ?resource+subject] -; [?relationship :eacl.relationship/resource+subject ?resource+subject] -; -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?subject]] -; -; [(reachable ?resource ?subject) -; ; don't think this will work... -; [(tuple ?resource ?mid) ?resource+mid] -; [?relationship :eacl.relationship/resource+subject ?resource+mid] -; -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?mid] -; (reachable ?mid ?subject)] -; -; ;; Direct permission check -; [(has-permission ?resource ?permission-name ?subject-type ?subject) -; -; [(tuple ?resource ?relation-name-in-tuple ?subject) ?resource+rel-name+subject] -; [?relationship :eacl.relationship/resource+relation-name+subject ?resource+rel-name+subject] -; -; [?relationship :eacl.relationship/resource ?resource] ; subject has some relationship TO the resource -; [?relationship :eacl.relationship/relation-name ?relation-name-in-tuple] -; [?relationship :eacl.relationship/subject ?subject] ; subject of the relationship tuple -; -; ;; Permission definition: ?relation-name-in-perm-def grants ?permission-name on ?resource-type -; [(tuple ?resource-type ?relation-name-in-tuple ?permission-name) ?res-type+relation+permission] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-tuple] ; THIS IS THE DIRECT GRANT -; -; ;; Match the relation name from the relationship tuple with the one in permission definition -; ;[(= ?relation-name-in-tuple ?relation-name-in-perm-def)] ; can this be unified better? should this be higher up? -; -; [(not= ?subject ?resource)]] -; -; ;; Indirect permission inheritance (copied from core2 - may need review/replacement with arrows) -; ;; This rule means: ?subject gets ?permission-name on ?resource if: -; ;; 1. A permission definition exists: for ?resource-type, ?relation-name-in-perm-def grants ?permission-name. -; ;; 2. ?resource has a relationship (as a subject of the tuple) via ?relation-name-in-perm-def to some ?target. -; ;; (e.g. doc D is "subject" of relation "group" to group G: D --group--> G) -; ;; 3. ?subject can "reach" that ?target (e.g. user U is member of group G). -; -; [(has-permission ?resource ?permission-name ?subject-type ?subject) -; ;; Permission definition -; ;; I don't think tuple makes sense here... -; ;[(tuple ?resource-type ?relation-name-in-perm-def ?permission-name) ?res-type+relation+permission] -; ;[?perm-def :eacl.permission/resource-type+relation-name+permission-name ?res-type+relation+permission] -; -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name-in-perm-def] ; Direct relation specified in perm -; -; ;; Structural relationship: ?resource is linked to ?target via ?relation-name-in-perm-def -; [(tuple ?target ?relation-name-in-perm-def ?resource) ?target+relation+resource] -; [?structural-rel :eacl.relationship/resource+relation-name+subject ?target+relation+resource] -; -; [?structural-rel :eacl.relationship/subject ?resource] -; [?structural-rel :eacl.relationship/relation-name ?relation-name-in-perm-def] -; [?structural-rel :eacl.relationship/resource ?target] -; -; (reachable ?target ?subject) ; User must be able to reach the target of the structural relationship -; [?resource :eacl/type ?resource-type] ; super slow. different rules WIP. -; [?subject :eacl/type ?subject-type] -; [(not= ?subject ?resource)]] -; -; ;; Arrow permission rule: ?subject gets ?perm-name-on-this-resource if it has ?perm-name-on-related on an intermediate resource -; ;; Example: User U gets :admin on VPC_X if VPC_X --:account--> ACC_Y and User U has :admin on ACC_Y. -; ;; MODIFIED based on user feedback: Rule now expects intermediate --via-relation-name--> this-resource -; ;; Example: User U gets :view on SERVER_X if ACC_Y --:account--> SERVER_X and User U has :admin on ACC_Y. -; [(has-permission ?this-resource ?perm-name-on-this-resource ?subject-type ?subject) -; -; ;; 1. Find an arrow permission definition for this-resource-type and perm-name-on-this-resource -; [(tuple ?this-resource-type -; ?via-relation-name -; ?perm-on-related -; ?perm-name-on-this-resource) ?res-type+relation+related-perm+permission] -; [?arrow-perm-def -; :eacl.arrow-permission/resource-type+source-relation-name+target-permission-name+permission-name -; ?res-type+relation+related-perm+permission] -; -; ; can these move down for speed, or be decoupled in a 2nd phase? -; [?arrow-perm-def :eacl.arrow-permission/resource-type ?this-resource-type] -; [?arrow-perm-def :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] -; [?arrow-perm-def :eacl.arrow-permission/source-relation-name ?via-relation-name] ; e.g., :account (the relation name specified in Permission) -; [?arrow-perm-def :eacl.arrow-permission/target-permission-name ?perm-on-related] ; e.g., :admin (on the intermediate/account) -; -; ;; 2. Find intermediate resource: ?intermediate-resource --via-relation-name--> ?this-resource -; [(tuple ?this-resource ?via-relation-name ?intermediate-resource) ?resource+relation+mid-resource] -; [?rel-linking-resources :eacl.relationship/resource+relation-name+subject ?resource+relation+mid-resource] -; -; [?rel-linking-resources :eacl.relationship/subject ?intermediate-resource] ; e.g., account is subject of tuple -; [?rel-linking-resources :eacl.relationship/relation-name ?via-relation-name] ; relation is :account -; [?rel-linking-resources :eacl.relationship/resource ?this-resource] ; e.g., server/vpc is resource of tuple -; -; ;[?this-resource :eacl/type ?this-resource-type] -; ;[?subject :eacl/type ?subject-type] -; [?intermediate-resource :resource/type ?intermediate-resource-type] -; -; ;; 3. Subject must have the target permission on the intermediate resource (recursive call) -; (has-permission ?subject ?perm-on-related ?intermediate-resource-type ?intermediate-resource) -; [(not= ?subject ?this-resource)] ; Exclude self-references for safety -; ;; Ensure the intermediate resource is not the same as the subject to prevent some loops, -; ;; though main cycle prevention relies on data structure or more complex rule logic if needed. -; [(not= ?subject ?intermediate-resource)] -; ;; Ensure this-resource is not the same as intermediate for simple arrows like A -> B -; [(not= ?this-resource ?intermediate-resource)]]]) -; this is super slow. different rules WIP.]]) diff --git a/src/eacl/datomic/rules/optimized.clj b/src/eacl/datomic/rules/optimized.clj deleted file mode 100644 index 3cd59d68..00000000 --- a/src/eacl/datomic/rules/optimized.clj +++ /dev/null @@ -1,427 +0,0 @@ -(ns eacl.datomic.rules.optimized - "Optimized Datalog rules for EACL performance improvements") - -(def check-permission-rules - "Recursive Datalog rules for can? & lookup-resources using unified permission schema." - '[;; Optimized reachability using tuples - [(reachable ?resource ?subject) - ;; Direct relationship - most common case - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject]] - - [(reachable ?resource ?subject) - ;; Indirect relationship - use tuple for first hop - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] - ;; Only traverse if needed - (reachable ?mid ?subject)] - - ;; Direct permission - unified schema (no source-relation-name) - [(has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) - - ;; Find relationships for this resource - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/resource-type ?resource-type] - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/subject-type ?subject-type] - [?relationship :eacl.relationship/relation-name ?relation-name] - - ;; Find direct permission (no source-relation-name means direct) - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/target-type :relation] - [?perm-def :eacl.permission/target-name ?relation-name] - [?perm-def :eacl.permission/source-relation-name :self] - - ;; Exclude self-references - [(not= ?subject ?resource)]] - - ;; Indirect permission inheritance via direct permissions - [(has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) - - ;; Find direct permission definitions for this resource type - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/target-type :relation] - [?perm-def :eacl.permission/target-name ?relation-name] - [?perm-def :eacl.permission/source-relation-name :self] - - ;; Find structural relationships where resource is the subject - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/subject-type ?resource-type] - [?structural-rel :eacl.relationship/relation-name ?relation-name] - [?structural-rel :eacl.relationship/resource ?target] - [?structural-rel :eacl.relationship/resource-type ?target-type] - - ;; Check reachability last - (reachable ?target ?subject) - [(not= ?subject ?resource)]] - - ;; Arrow permission to permission - unified schema - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) - - ;; Find arrow permission definitions that target permissions - [?arrow-perm :eacl.permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.permission/target-type :permission] - [?arrow-perm :eacl.permission/target-name ?perm-on-related] - - ;; Find intermediate resource - [?rel-linking :eacl.relationship/resource-type ?this-resource-type] - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - [?rel-linking :eacl.relationship/subject-type ?intermediate-resource-type] - - ;; Recursive permission check - (has-permission ?subject-type ?subject ?perm-on-related ?intermediate-resource-type ?intermediate-resource) - - ;; Safety checks - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]] - - ;; Arrow permission to relation - unified schema - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) - - ;; Find arrow permission definitions that target relations - [?arrow-perm :eacl.permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.permission/target-type :relation] - [?arrow-perm :eacl.permission/target-name ?target-relation] - - ;; Find intermediate resource - [?rel-linking :eacl.relationship/resource-type ?this-resource-type] - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - [?rel-linking :eacl.relationship/subject-type ?intermediate-resource-type] - - ;; Check if subject has the target relation on intermediate resource - [?target-rel :eacl.relationship/resource ?intermediate-resource] - [?target-rel :eacl.relationship/resource-type ?intermediate-resource-type] - [?target-rel :eacl.relationship/subject ?subject] - [?target-rel :eacl.relationship/subject-type ?subject-type] - [?target-rel :eacl.relationship/relation-name ?target-relation] - - ;; Safety checks - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]]]) - -;(def check-permission-rules-broken -; '[(reachable ?resource ?subject) -; [?structural-rel :eacl.relationship/subject ?subject] -; [?structural-rel :eacl.relationship/resource ?resource] -; -; (reachable ?resource ?subject) -; [?structural-rel :eacl.relationship/subject ?mid] -; [?structural-rel :eacl.relationship/resource ?resource] -; (reachable ?mid ?subject) -; -; (has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/subject ?subject] -; [?relationship :eacl.relationship/relation-name ?relation-name] -; [?relationship :eacl.relationship/resource-type ?resource-type] -; [?relationship :eacl.relationship/subject-type ?subject-type] -; [(tuple ?resource-type ?relation-name ?permission-name) ?perm-tuple] -; [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?perm-tuple] -; [(not= ?subject ?resource)] -; -; (has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission-name] -; [?perm-def :eacl.permission/relation-name ?relation-name] -; [?structural-rel :eacl.relationship/subject ?resource] -; [?structural-rel :eacl.relationship/relation-name ?relation-name] -; [?structural-rel :eacl.relationship/resource ?target] -; [?structural-rel :eacl.relationship/subject-type ?resource-type] -; [?structural-rel :eacl.relationship/resource-type ?target-type] -; (reachable ?target ?subject) -; [(not= ?subject ?resource)] -; -; (has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) -; [?arrow-perm :eacl.arrow-permission/resource-type ?this-resource-type] -; [?arrow-perm :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] -; [?arrow-perm :eacl.arrow-permission/source-relation-name ?via-relation] -; [?arrow-perm :eacl.arrow-permission/target-permission-name ?perm-on-related] -; [?rel-linking :eacl.relationship/resource ?this-resource] -; [?rel-linking :eacl.relationship/resource-type ?this-resource-type] -; [?rel-linking :eacl.relationship/relation-name ?via-relation] -; [?rel-linking :eacl.relationship/subject ?intermediate-resource] -; [?rel-linking :eacl.relationship/subject-type ?intermediate-resource-type] -; (has-permission ?subject-type ?subject ?perm-on-related ?intermediate-resource-type ?intermediate-resource) -; [(not= ?subject ?this-resource)] -; [(not= ?subject ?intermediate-resource)] -; [(not= ?this-resource ?intermediate-resource)]]) - -(def rules-lookup-subjects - "Optimized rules for lookup-subjects using unified permission schema" - '[;; Reachability rules remain the same - ; Note: resource is known. subject is unknown. - [(reachable ?resource ?subject) - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject]] - - [(reachable ?resource ?subject) - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] - (reachable ?mid ?subject)] - - ;; Direct permission check - unified schema - [(has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) - - ;; Find relationships for this resource - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/relation-name ?relation-name] - [?relationship :eacl.relationship/resource-type ?resource-type] - [?relationship :eacl.relationship/subject-type ?subject-type] - - ;; Find direct permission (no source-relation-name means direct) - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/target-type :relation] - [?perm-def :eacl.permission/target-name ?relation-name] - [?perm-def :eacl.permission/source-relation-name :self] - - [(not= ?subject ?resource)]] - - ;; Indirect permission inheritance via direct permissions - [(has-permission ?subject-type ?subject ?permission-name ?resource-type ?resource) - ;; Find direct permission definitions - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/target-type :relation] - [?perm-def :eacl.permission/target-name ?relation-name] - [?perm-def :eacl.permission/source-relation-name :self] - - ;; Find structural relationships - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/relation-name ?relation-name] - [?structural-rel :eacl.relationship/resource ?target] - [?structural-rel :eacl.relationship/subject-type ?resource-type] - [?structural-rel :eacl.relationship/resource-type ?target-type] - - (reachable ?target ?subject) - [(not= ?subject ?resource)]] - - ;; Arrow permission to permission - unified schema - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) - - ;; Find arrow permissions that target permissions - [?arrow-perm :eacl.permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.permission/target-type :permission] - [?arrow-perm :eacl.permission/target-name ?perm-on-related] - - ;; Find intermediate - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - [?rel-linking :eacl.relationship/subject-type ?intermediate-resource-type] - [?rel-linking :eacl.relationship/resource-type ?this-resource-type] - - (has-permission ?subject-type ?subject ?perm-on-related ?intermediate-resource-type ?intermediate-resource) - - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]] - - ;; Arrow permission to relation - unified schema - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource-type ?this-resource) - - ;; Find arrow permissions that target relations - [?arrow-perm :eacl.permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.permission/target-type :relation] - [?arrow-perm :eacl.permission/target-name ?target-relation] - - ;; Find intermediate - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - [?rel-linking :eacl.relationship/subject-type ?intermediate-resource-type] - [?rel-linking :eacl.relationship/resource-type ?this-resource-type] - - ;; Check if subject has the target relation on intermediate resource - [?target-rel :eacl.relationship/resource ?intermediate-resource] - [?target-rel :eacl.relationship/resource-type ?intermediate-resource-type] - [?target-rel :eacl.relationship/subject ?subject] - [?target-rel :eacl.relationship/subject-type ?subject-type] - [?target-rel :eacl.relationship/relation-name ?target-relation] - - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]]]) - -;(def rules-lookup-resources -; "Optimized rules for lookup-resources - subject-centric approach" -; '[;; Helper rule: find relationships from subject -; -; [(relations-between-subject-resource ?subject ?relation ?resource-type ?resource) -; [(tuple ?subject ?resource-type) ?subject+resource-type] -; [?relationship :eacl.relationship/subject+resource-type ?subject+resource-type] -; -; [?relationship :eacl.relationship/relation-name ?relation] -; [?relationship :eacl.relationship/subject ?subject] -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/resource-type ?resource-type]] -; -; [(subject-has-relationships ?subject-type ?subject ?relation ?resource-type ?resource) -; ;; Start from subject -; -; [(tuple ?subject ?relation ?resource) ?subject+relation+resource] -; [?relationship :eacl.relationship/subject+relation-name+resource ?subject+relation+resource] -; -; [?relationship :eacl.relationship/subject ?subject] -; [?relationship :eacl.relationship/resource ?resource] -; [?relationship :eacl.relationship/relation-name ?relation] -; [?relationship :eacl.relationship/subject-type ?subject-type] -; [?relationship :eacl.relationship/resource-type ?resource-type]] -; -; ;; Direct permission check - subject-centric -; [(has-permission ?subject-type ?subject ?permission ?resource-type ?resource) -; ;; Check if resource is of correct type -; ;[?resource :eacl/type ?resource-type] -; -; [(tuple ?resource-type ?permission) ?rtype+permission] -; [?perm :eacl.permission/resource-type+permission-name ?rtype+permission] -; -; ;; Check if relation grants permission -; ;[?perm :eacl.permission/resource-type ?resource-type] -; [?perm :eacl.permission/relation-name ?relation] -; ;[?perm :eacl.permission/permission-name ?permission] -; -; (subject-has-relationships ?subject-type ?subject ?relation ?resource-type ?resource)] -; -; ;; Indirect permission via intermediate resources -; [(has-permission ?subject-type ?subject ?permission ?resource-type ?resource) -; ;; Find relationships where subject can reach an intermediate -; -; ; Find relations that grant this permission: -; [(tuple ?resource-type ?permission) ?rtype+permission] -; [?perm-def :eacl.permission/resource-type+permission-name ?rtype+permission] -; [?perm-def :eacl.permission/resource-type ?resource-type] -; [?perm-def :eacl.permission/permission-name ?permission] -; [?perm-def :eacl.permission/relation-name ?rel2] -; -; (relations-between-subject-resource ?subject ?rel1 ?intermediate-type ?intermediate) -; ;(subject-has-relationships ?subject-type ?subject ?rel1 ?intermediate-type ?intermediate) -; -; ;[(tuple ?intermediate ?rel2 ?resource-type) ?intermediate+rel2+rtype] -; ;[?relationship2 :eacl.relationship/subject+relation-name+resource-type ?intermediate+rel2+rtype] -; -; ;; Find resources connected to intermediate via rel2 -; [?relationship2 :eacl.relationship/subject-type ?intermediate-type] -; [?relationship2 :eacl.relationship/subject ?intermediate] -; [?relationship2 :eacl.relationship/relation-name ?rel2] -; [?relationship2 :eacl.relationship/resource ?resource] -; [?relationship2 :eacl.relationship/resource-type ?resource-type]] -; -; ;; Verify resource type -; ;[?resource :eacl/type ?resource-type]] -; -; ;; Arrow permission - optimized for subject-centric lookup -; [(has-permission ?subject-type ?subject ?permission ?resource-type ?resource) -; ; known: subject-type ,subject, permission, resource-type. -; ;; Find arrow permissions for the target resource type and permission -; -; [(tuple ?resource-type ?permission) ?rtype+perm] -; [?arrow :eacl.arrow-permission/resource-type+permission-name ?rtype+perm] -; -; ;[?arrow :eacl.arrow-permission/resource-type ?resource-type] -; ;[?arrow :eacl.arrow-permission/permission-name ?permission] -; [?arrow :eacl.arrow-permission/source-relation-name ?via-rel] -; [?arrow :eacl.arrow-permission/target-permission-name ?target-perm] -; -; ;; Find intermediate resources linked to target resource (resource is not known here) -; ; the problem is we know via-rel, but not ?intermediate or ?resource. -; ; how to cull ?resource here. -; ; we should be able to narrow down relationships here to find the relevant ?intermediate subjects for traversal -; -; ;[(tuple ?resource-type ?via-rel) ?rtype+via-rel] -; ;[?link :eacl.relationship/resource-type+relation-name ?rtype+via-rel] -; ;[?link :eacl.relationship/resource-type+relation-name ?subject+via-rel+rtype] -; -; [?link :eacl.relationship/resource ?resource] ; this is unknown and expands. -; [?link :eacl.relationship/resource-type ?resource-type] ; this is known via arg. -; [?link :eacl.relationship/relation-name ?via-rel] ; this is known via arrow. -; -; [?link :eacl.relationship/subject ?intermediate] ; this is looked up here -; [?link :eacl.relationship/subject-type ?intermediate-type] ; this is looked up here -; -; ;; Get intermediate resource type -; ;[?intermediate :eacl/type ?intermediate-type] -; -; ;; Check if subject has target permission on intermediate (recursive) -; (has-permission ?subject-type ?subject ?target-perm ?intermediate-type ?intermediate)]]) - -(def rules-lookup-resources - ; not currently used. lookup-resources currently uses the check-permission rules, which are slow. - "Too slow. Superseded by direct index impl. - Was optimized rules for lookup-resources - subject-centric approach" - '[;; Helper rule: find relationships from subject - [(subject-has-relationships ?subject ?relation ?resource) - ;; Start from subject - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/relation-name ?relation] - [?relationship :eacl.relationship/resource ?resource]] - - ;; Direct permission check - subject-centric - [(has-permission ?subject-type ?subject ?permission ?resource-type ?resource) - ;; Start from subject's relationships - (subject-has-relationships ?subject ?relation ?resource) - - ;; Check if resource is of correct type - [?resource :eacl/type ?resource-type] - - ;; Check if relation grants permission - [?perm :eacl.permission/resource-type ?resource-type] - [?perm :eacl.permission/relation-name ?relation] - [?perm :eacl.permission/permission-name ?permission]] - - ;; Indirect permission via intermediate resources - [(has-permission ?subject ?permission ?resource-type ?resource) - ;; Find relationships where subject can reach an intermediate - (subject-has-relationships ?subject ?rel1 ?intermediate) - - ;; Find permission definition that grants access via relation - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission] - [?perm-def :eacl.permission/relation-name ?rel2] - - ;; Find resources connected to intermediate via rel2 - [?relationship2 :eacl.relationship/subject ?intermediate] - [?relationship2 :eacl.relationship/relation-name ?rel2] - [?relationship2 :eacl.relationship/resource ?resource] - - ;; Verify resource type - [?resource :eacl/type ?resource-type]] - - ;; Arrow permission - optimized for subject-centric lookup - [(has-permission ?subject ?permission ?resource-type ?resource) - ;; Find arrow permissions for the target resource type and permission - [?arrow :eacl.arrow-permission/resource-type ?resource-type] - [?arrow :eacl.arrow-permission/permission-name ?permission] - [?arrow :eacl.arrow-permission/source-relation-name ?via-rel] - [?arrow :eacl.arrow-permission/target-permission-name ?target-perm] - - ;; Find resources of target type - [?resource :eacl/type ?resource-type] - - ;; Find intermediate resources linked to target resource - [?link :eacl.relationship/resource ?resource] - [?link :eacl.relationship/relation-name ?via-rel] - [?link :eacl.relationship/subject ?intermediate] - - ;; Get intermediate resource type - [?intermediate :eacl/type ?intermediate-type] - - ;; Check if subject has target permission on intermediate (recursive) - (has-permission ?subject ?target-perm ?intermediate-type ?intermediate)]]) \ No newline at end of file diff --git a/src/eacl/datomic/rules/optimized_old.clj b/src/eacl/datomic/rules/optimized_old.clj deleted file mode 100644 index 26ffe2d0..00000000 --- a/src/eacl/datomic/rules/optimized_old.clj +++ /dev/null @@ -1,225 +0,0 @@ -(ns eacl.datomic.rules.optimized-old - "Optimized Datalog rules for EACL performance improvements") - -(def check-permission-rules - "Optimized rules for can? - reordered clauses and better tuple usage" - '[;; Optimized reachability using tuples - [(reachable ?resource ?subject) - ;; Direct relationship - most common case - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject]] - - [(reachable ?resource ?subject) - ;; Indirect relationship - use tuple for first hop - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] - ;; Only traverse if needed - (reachable ?mid ?subject)] - - ;; Direct permission - optimized clause ordering - [(has-permission ?subject ?permission-name ?resource) - ;; Get resource type first - [?resource :resource/type ?resource-type] - - ;; Find relationships for this resource - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/relation-name ?relation-name] - - ;; Check permission using tuple - [(tuple ?resource-type ?relation-name ?permission-name) ?perm-tuple] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?perm-tuple] - - ;; Exclude self-references - [(not= ?subject ?resource)]] - - ;; Indirect permission inheritance - optimized - [(has-permission ?subject ?permission-name ?resource) - ;; Get resource type first (we already have the resource) - [?resource :resource/type ?resource-type] - - ;; Find permission definitions for this resource type - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name] - - ;; Find structural relationships where resource is the subject - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/relation-name ?relation-name] - [?structural-rel :eacl.relationship/resource ?target] - - ;; Check reachability last - (reachable ?target ?subject) - [(not= ?subject ?resource)]] - - ;; Arrow permission - optimized - [(has-permission ?subject ?perm-name-on-this-resource ?this-resource) - ;; Get resource type from the resource we already have - [?this-resource :resource/type ?this-resource-type] - - ;; Find arrow permission definitions - [?arrow-perm :eacl.arrow-permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.arrow-permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.arrow-permission/target-permission-name ?perm-on-related] - - ;; Find intermediate resource - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - - ;; Recursive permission check - (has-permission ?subject ?perm-on-related ?intermediate-resource) - - ;; Safety checks - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]]]) - -(def rules-lookup-subjects - "Optimized rules for lookup-subjects" - '[;; Reachability rules remain the same - [(reachable ?resource ?subject) - [(tuple ?resource ?subject) ?resource+subject] - [?relationship :eacl.relationship/resource+subject ?resource+subject]] - - [(reachable ?resource ?subject) - [(tuple ?resource ?mid) ?resource+mid] - [?relationship :eacl.relationship/resource+subject ?resource+mid] - (reachable ?mid ?subject)] - - ;; Direct permission check - optimized for known resource - [(has-permission ?subject-type ?subject ?permission-name ?resource) - ;; Get resource type (we already have resource entity) - [?resource :resource/type ?resource-type] - - ;; Find relationships for this resource - [?relationship :eacl.relationship/resource ?resource] - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/relation-name ?relation-name] - - ;; Check subject type - [?subject :resource/type ?subject-type] - - ;; Check permission using tuple - [(tuple ?resource-type ?relation-name ?permission-name) ?perm-tuple] - [?perm-def :eacl.permission/resource-type+relation-name+permission-name ?perm-tuple] - - [(not= ?subject ?resource)]] - - ;; Indirect permission inheritance - [(has-permission ?subject-type ?subject ?permission-name ?resource) - [?resource :resource/type ?resource-type] - - ;; Find permission definitions - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission-name] - [?perm-def :eacl.permission/relation-name ?relation-name] - - ;; Find structural relationships - [?structural-rel :eacl.relationship/subject ?resource] - [?structural-rel :eacl.relationship/relation-name ?relation-name] - [?structural-rel :eacl.relationship/resource ?target] - - (reachable ?target ?subject) - [?subject :resource/type ?subject-type] - [(not= ?subject ?resource)]] - - ;; Arrow permission - [(has-permission ?subject-type ?subject ?perm-name-on-this-resource ?this-resource) - [?this-resource :resource/type ?this-resource-type] - - ;; Find arrow permissions - [?arrow-perm :eacl.arrow-permission/resource-type ?this-resource-type] - [?arrow-perm :eacl.arrow-permission/permission-name ?perm-name-on-this-resource] - [?arrow-perm :eacl.arrow-permission/source-relation-name ?via-relation] - [?arrow-perm :eacl.arrow-permission/target-permission-name ?perm-on-related] - - ;; Find intermediate - [?rel-linking :eacl.relationship/resource ?this-resource] - [?rel-linking :eacl.relationship/relation-name ?via-relation] - [?rel-linking :eacl.relationship/subject ?intermediate-resource] - - (has-permission ?subject-type ?subject ?perm-on-related ?intermediate-resource) - - [(not= ?subject ?this-resource)] - [(not= ?subject ?intermediate-resource)] - [(not= ?this-resource ?intermediate-resource)]]]) - -(def rules-lookup-resources - "Optimized rules for lookup-resources - subject-centric approach" - '[;; Helper rule: find relationships from subject - [(subject-has-relationships ?subject ?relation ?resource) - ;; Start from subject - - ;[(tuple ?subject ?relation ?resource) ?subject+relation+resource] - ;[?relationship :eacl.relationship/] - - [?relationship :eacl.relationship/subject ?subject] - [?relationship :eacl.relationship/relation-name ?relation] - [?relationship :eacl.relationship/resource ?resource]] - - ;; Direct permission check - subject-centric - [(has-permission ?subject ?permission ?resource-type ?resource) - ;; Start from subject's relationships - (subject-has-relationships ?subject ?relation ?resource) - - ;; Check if resource is of correct type - [?resource :resource/type ?resource-type] - - ;; Check if relation grants permission - [?perm :eacl.permission/resource-type ?resource-type] - [?perm :eacl.permission/relation-name ?relation] - [?perm :eacl.permission/permission-name ?permission]] - - ;; Indirect permission via intermediate resources - [(has-permission ?subject ?permission ?resource-type ?resource) - ;; Find relationships where subject can reach an intermediate - (subject-has-relationships ?subject ?rel1 ?intermediate) - - ;; Find permission definition that grants access via relation - [?perm-def :eacl.permission/resource-type ?resource-type] - [?perm-def :eacl.permission/permission-name ?permission] - [?perm-def :eacl.permission/relation-name ?rel2] - - ;; Find resources connected to intermediate via rel2 - [?relationship2 :eacl.relationship/subject ?intermediate] - [?relationship2 :eacl.relationship/relation-name ?rel2] - [?relationship2 :eacl.relationship/resource ?resource] - - ;; Verify resource type - [?resource :resource/type ?resource-type]] - - ;; Arrow permission - optimized for subject-centric lookup - [(has-permission ?subject ?permission ?resource-type ?resource) - ; known: subject, permission, resource-type. - ;; Find arrow permissions for the target resource type and permission - - [(tuple ?resource-type ?permission) ?rtype+perm] - [?arrow :eacl.arrow-permission/resource-type+permission-name ?rtype+perm] - - ;[?arrow :eacl.arrow-permission/resource-type ?resource-type] - ;[?arrow :eacl.arrow-permission/permission-name ?permission] - [?arrow :eacl.arrow-permission/source-relation-name ?via-rel] - [?arrow :eacl.arrow-permission/target-permission-name ?target-perm] - - ;; Find resources of target type - [?resource :resource/type ?resource-type] ; this looks slow. - - ;; Find intermediate resources linked to target resource (resource is not known here) - ; the problem is we know via-rel, but not ?intermediate or ?resource. - ; how to cull ?resource here. - ; we should be able to narrow down relationships here to find the relevant ?intermediate subjects for traversal - - ;[(tuple ?subject ?via-rel) ?subject+via-rel] - ;[?link :eacl.relationship/subject+relation-name ?subject+via-rel] - - [?link :eacl.relationship/relation-name ?via-rel] - [?link :eacl.relationship/resource ?resource] - [?link :eacl.relationship/subject ?intermediate] - - ;; Get intermediate resource type - [?intermediate :resource/type ?intermediate-type] - - ;; Check if subject has target permission on intermediate (recursive) - (has-permission ?subject ?target-perm ?intermediate-type ?intermediate)]]) \ No newline at end of file diff --git a/src/eacl/datomic/schema.clj b/src/eacl/datomic/schema.clj index cf36a0e3..581e1695 100644 --- a/src/eacl/datomic/schema.clj +++ b/src/eacl/datomic/schema.clj @@ -36,6 +36,19 @@ ;(def Permission ; [:or DirectPermission ArrowPermission]) +(def schema-version-attr-definition + "Cache-invalidation stamp. write-schema! asserts a fresh squuid here in the + same transaction as any definition change; EACL's permission-path caches and + cursor fingerprints key on it, so ONLY write-schema! invalidates them (#74). + A squuid (not a counter) so two concurrent writers can never assert the same + value and elide each other's invalidation. Do not edit EACL definitions + outside write-schema! — the stamp will not change and caches will be stale." + {:db/ident :eacl/schema-version + :db/doc "Squuid bumped by write-schema! whenever definitions change. Path caches and cursor fingerprints key on it." + :db/valueType :db.type/uuid + :db/cardinality :db.cardinality/one + :db/index true}) + (def v7-schema [; :eacl/id is now optional. {:db/ident :eacl/id ; todo: figure out how to support :id, :object/id or :spice/id of different types. @@ -49,6 +62,8 @@ :db/valueType :db.type/string :db/cardinality :db.cardinality/one} + schema-version-attr-definition + ;; Relations {:db/ident :eacl.relation/resource-type :db/doc "EACL Relation: Resource Type" @@ -249,12 +264,18 @@ (into {} (for [[rt perms] permissions-by-type] [rt (set (map :eacl.permission/permission-name perms))])) - ;; Get subject types for each relation (for arrow target validation) + ;; Subject types for each relation as full SETS (for arrow target validation). + ;; Multi-type relations (relation owner: user | group) expand to one entry per + ;; type; keeping a set makes validation independent of declaration order. relation-subject-types - (into {} (for [rel relations] - [[(:eacl.relation/resource-type rel) + (reduce (fn [acc rel] + (update acc + [(:eacl.relation/resource-type rel) (:eacl.relation/relation-name rel)] - (:eacl.relation/subject-type rel)])) + (fnil conj #{}) + (:eacl.relation/subject-type rel))) + {} + relations) errors (atom [])] @@ -274,16 +295,16 @@ {:type :invalid-self-relation :permission (str (name res-type) "/" (name perm-name)) :target target-name - :message (str "Permission " (name res-type) "/" (name perm-name) - " references non-existent relation: " (name target-name))})) + :message (str "Permission " (name res-type) "/" (name perm-name)) + " references non-existent relation: " (name target-name)})) ;; Self -> permission: validate permission exists on this resource type (when-not (contains? (get permission-names-by-type res-type) target-name) (swap! errors conj {:type :invalid-self-permission :permission (str (name res-type) "/" (name perm-name)) :target target-name - :message (str "Permission " (name res-type) "/" (name perm-name) - " references non-existent permission: " (name target-name))}))) + :message (str "Permission " (name res-type) "/" (name perm-name)) + " references non-existent permission: " (name target-name)}))) ;; For arrow permissions (source-rel != :self) (do @@ -293,36 +314,37 @@ {:type :missing-source-relation :permission (str (name res-type) "/" (name perm-name)) :relation source-rel - :message (str "Permission " (name res-type) "/" (name perm-name) - " references non-existent relation: " (name source-rel))})) + :message (str "Permission " (name res-type) "/" (name perm-name)) + " references non-existent relation: " (name source-rel)})) - ;; If source relation exists, validate target exists on target resource type + ;; If source relation exists, validate the target exists on EVERY subject + ;; type of the source relation. Anything else is declaration-order-dependent, + ;; and SpiceDB requires arrow targets on all possible subject types. (when (contains? (get relation-names-by-type res-type) source-rel) - (let [target-res-type (get relation-subject-types [res-type source-rel])] - (when target-res-type - (if (= target-type :relation) - ;; Arrow to relation: validate relation exists on target type - (when-not (contains? (get relation-names-by-type target-res-type) target-name) - (swap! errors conj - {:type :invalid-arrow-target-relation - :permission (str (name res-type) "/" (name perm-name)) - :arrow-via source-rel - :target-type target-res-type - :target target-name - :message (str "Permission " (name res-type) "/" (name perm-name) - " arrow via " (name source-rel) "->" (name target-name) - " - relation '" (name target-name) "' does not exist on " (name target-res-type))})) - ;; Arrow to permission: validate permission exists on target type - (when-not (contains? (get permission-names-by-type target-res-type) target-name) - (swap! errors conj - {:type :invalid-arrow-target-permission - :permission (str (name res-type) "/" (name perm-name)) - :arrow-via source-rel - :target-type target-res-type - :target target-name - :message (str "Permission " (name res-type) "/" (name perm-name) - " arrow via " (name source-rel) "->" (name target-name) - " - permission '" (name target-name) "' does not exist on " (name target-res-type))})))))))))) + (doseq [target-res-type (get relation-subject-types [res-type source-rel])] + (if (= target-type :relation) + ;; Arrow to relation: validate relation exists on target type + (when-not (contains? (get relation-names-by-type target-res-type) target-name) + (swap! errors conj + {:type :invalid-arrow-target-relation + :permission (str (name res-type) "/" (name perm-name)) + :arrow-via source-rel + :target-type target-res-type + :target target-name + :message (str "Permission " (name res-type) "/" (name perm-name) + " arrow via " (name source-rel) "->" (name target-name) + " - relation '" (name target-name) "' does not exist on " (name target-res-type))})) + ;; Arrow to permission: validate permission exists on target type + (when-not (contains? (get permission-names-by-type target-res-type) target-name) + (swap! errors conj + {:type :invalid-arrow-target-permission + :permission (str (name res-type) "/" (name perm-name)) + :arrow-via source-rel + :target-type target-res-type + :target target-name + :message (str "Permission " (name res-type) "/" (name perm-name) + " arrow via " (name source-rel) "->" (name target-name) + " - permission '" (name target-name) "' does not exist on " (name target-res-type))}))))))))) (when (seq @errors) (throw (ex-info "Invalid schema: reference validation failed" @@ -370,18 +392,36 @@ "Computes delta between existing schema and new schema, checks for any orphaned relationships on retracted schema, produces tx-ops and applies. - - Throws if schema is invalid (operator validation, reference validation, orphan check)." - [conn schema-string] - (let [new-schema-map (parser/->eacl-schema (parser/parse-schema schema-string)) - ;; Validate schema references before proceeding (ADR 012 requirement) - _ (validate-schema-references new-schema-map) - db (d/db conn) - existing-schema (read-schema db) - deltas (compare-schema existing-schema new-schema-map) - {:keys [relations permissions]} deltas - relation-retractions (:retractions relations) - permission-retractions (:retractions permissions)] + + Throws if schema is invalid (parse failure, operator validation, reference + validation, orphan check), or if the new schema contains zero definitions + while a non-empty schema is stored (belt-and-braces against parser gaps — + a malformed input must never be able to retract the whole schema). Pass + {:allow-empty-schema? true} to explicitly wipe the stored schema." + ([conn schema-string] + (write-schema! conn schema-string {})) + ([conn schema-string {:keys [allow-empty-schema?]}] + ;; Upgrade path: databases installed before :eacl/schema-version existed. + (when-not (d/entid (d/db conn) :eacl/schema-version) + @(d/transact conn [schema-version-attr-definition])) + (let [new-schema-map (parser/->eacl-schema (parser/parse-schema schema-string)) + ;; Validate schema references before proceeding (ADR 012 requirement) + _ (validate-schema-references new-schema-map) + db (d/db conn) + existing-schema (read-schema db) + _ (when (and (empty? (:definitions new-schema-map)) + (not allow-empty-schema?) + (or (seq (:relations existing-schema)) + (seq (:permissions existing-schema)))) + (throw (ex-info (str "Refusing to replace a non-empty schema with zero definitions." + " Pass {:allow-empty-schema? true} to write-schema! if this is intentional.") + {:type :eacl.schema/empty-schema-guard + :existing {:relations (count (:relations existing-schema)) + :permissions (count (:permissions existing-schema))}}))) + deltas (compare-schema existing-schema new-schema-map) + {:keys [relations permissions]} deltas + relation-retractions (:retractions relations) + permission-retractions (:retractions permissions)] ;; Check for orphaned relationships (doseq [rel relation-retractions] @@ -392,7 +432,11 @@ {:relation rel :count cnt}))))) ;; Transact changes - (let [tx-data (concat + (let [schema-changed? (boolean (or (seq (:additions relations)) + (seq relation-retractions) + (seq (:additions permissions)) + (seq permission-retractions))) + tx-data (concat ;; Additions (:additions relations) (:additions permissions) @@ -401,9 +445,13 @@ [:db.fn/retractEntity [:eacl/id (:eacl/id rel)]]) (for [perm permission-retractions] [:db.fn/retractEntity [:eacl/id (:eacl/id perm)]]) - ;; Store schema string - [{:eacl/id "schema-string" - :eacl/schema-string schema-string}])] + ;; Store schema string + bump the version stamp when + ;; definitions changed. The stamp is what invalidates the + ;; path caches and cursor fingerprints — on every peer, + ;; and correctly for d/as-of views (issue #74). + [(cond-> {:eacl/id "schema-string" + :eacl/schema-string schema-string} + schema-changed? (assoc :eacl/schema-version (d/squuid)))])] @(d/transact conn tx-data) (impl.indexed/evict-permission-paths-cache!) - deltas))) + deltas)))) diff --git a/src/eacl/spicedb/parser.clj b/src/eacl/spicedb/parser.clj index 160a4a5b..e63b16fa 100644 --- a/src/eacl/spicedb/parser.clj +++ b/src/eacl/spicedb/parser.clj @@ -7,6 +7,21 @@ [eacl.datomic.impl :as impl])) ; primary-expr = identifier | <'('> permission-expr <')'> +;; Whitespace parser used by the comment-aware whitespace parser below. +(def ^:private whitespace + (insta/parser "whitespace = #'\\s+'")) + +;; SpiceDB schemas may contain // line comments and /* */ block comments +;; anywhere whitespace is legal. Modelled as auto-whitespace per the +;; instaparse whitespace-or-comments idiom. +(def ^:private whitespace-or-comments + (insta/parser + "ws-or-comments = ws | comments + comments = comment+ + comment = #'//[^\\n\\r]*' | #'/\\*(?s).*?\\*/' + ws = #'\\s+'" + :auto-whitespace whitespace)) + ;; Define the SpiceDB grammar with auto-whitespace ;; Full SpiceDB grammar - parses the complete official syntax. ;; EACL-specific restrictions are enforced during validation, not parsing. @@ -58,7 +73,7 @@ (* Identifiers - must not match keywords *) identifier = !('nil' | 'self' | 'definition' | 'relation' | 'permission' | 'with' | 'any' | 'all') #'[a-zA-Z_][a-zA-Z0-9_]*'" - :auto-whitespace :standard)) + :auto-whitespace whitespace-or-comments)) ;; Example SpiceDB schema (def example-schema @@ -127,7 +142,8 @@ (defn extract-relations "Extract relations from definition body. - Returns a map where each key is a relation name and value is a vector of type refs." + Returns a map where each key is a relation name and value is a vector of type refs. + Throws on duplicate relation declarations (multi-type via `|` is a single declaration)." [definition-body] (if (and (vector? definition-body) (= :definition-body (first definition-body))) (->> (rest definition-body) @@ -136,7 +152,15 @@ (let [rel-name (extract-identifier (second rel-name-node)) type-refs (extract-relation-type-expr type-expr-node)] [rel-name type-refs]))) - (into {})) + (reduce (fn [acc [rel-name type-refs]] + (if (contains? acc rel-name) + (throw (ex-info (str "Duplicate relation declaration: '" rel-name "'." + " Declare multiple subject types once with `|`," + " e.g. `relation " rel-name ": a | b`.") + {:type :eacl.schema/duplicate-relation + :relation rel-name})) + (assoc acc rel-name type-refs))) + {})) {})) (defn extract-permissions @@ -153,22 +177,45 @@ (defn extract-definitions "Extract definitions from parse tree. - Returns map of {type-path {:relations {...}, :permissions [...]}}" + Returns map of {type-path {:relations {...}, :permissions [...]}}. + Throws on duplicate definition blocks and on a permission sharing a name + with a relation on the same definition (SpiceDB rejects both; silently + letting the last one win produces destructive write-schema! deltas)." [parse-tree] (->> parse-tree (filter #(and (vector? %) (= :definition (first %)))) (map (fn [[_ type-path-node definition-body]] - (let [type-path (extract-type-path type-path-node)] + (let [type-path (extract-type-path type-path-node) + relations (extract-relations definition-body) + permissions (extract-permissions definition-body) + collisions (filter (set (keys relations)) (map :name permissions))] + (when (seq collisions) + (throw (ex-info (str "Permission and relation share a name on definition '" type-path + "': " (pr-str (vec collisions))) + {:type :eacl.schema/name-collision + :definition type-path + :names (vec collisions)}))) [type-path - {:relations (extract-relations definition-body) - :permissions (extract-permissions definition-body)}]))) - (into {}))) + {:relations relations + :permissions permissions}]))) + (reduce (fn [acc [type-path spec]] + (if (contains? acc type-path) + (throw (ex-info (str "Duplicate definition: '" type-path "'." + " Each type may be defined once; merge the blocks.") + {:type :eacl.schema/duplicate-definition + :definition type-path})) + (assoc acc type-path spec))) + {}))) (defn transform-schema - "Transform parse tree to intermediate representation." + "Transform parse tree to intermediate representation. + Throws on unexpected input; a failed parse must never coerce to an empty schema." [parse-tree] - (when (and (vector? parse-tree) (= :schema (first parse-tree))) - {:definitions (extract-definitions (rest parse-tree))})) + (if (and (vector? parse-tree) (= :schema (first parse-tree))) + {:definitions (extract-definitions (rest parse-tree))} + (throw (ex-info "Unexpected schema parse tree; refusing to interpret as an empty schema." + {:type :eacl.schema/parse-error + :parse-tree parse-tree})))) ;; Helper to parse expressions (defn parse-permission-expression [expr-str] @@ -320,12 +367,18 @@ :operator "-" :message "Unsupported operator: Exclusion (-). EACL only supports Union (+) at this time."})) - ;; Check for multi-level arrows + ;; Check for multi-level arrows and parenthesized arrow bases/targets :simple-arrow-expr - (when (> (count (filter #(and (vector? %) (= :base-expr (first %))) (rest node))) 2) - (swap! issues conj - {:type :multi-level-arrow - :message "Unsupported feature: Multi-level arrows (e.g., a->b->c). EACL only supports single-level arrows like rel->perm."})) + (let [base-exprs (filter #(and (vector? %) (= :base-expr (first %))) (rest node))] + (when (> (count base-exprs) 2) + (swap! issues conj + {:type :multi-level-arrow + :message "Unsupported feature: Multi-level arrows (e.g., a->b->c). EACL only supports single-level arrows like rel->perm."})) + (when (and (> (count base-exprs) 1) + (some #(and (vector? (second %)) (= :paren-expr (first (second %)))) base-exprs)) + (swap! issues conj + {:type :paren-arrow + :message "Unsupported feature: Parenthesized expressions as arrow bases or targets (e.g., (a + b)->c). Arrows take a single relation base."}))) ;; Check for .all() function (only .any() is implicitly supported via arrow) :arrow-func-expr @@ -452,6 +505,8 @@ ;; Converts new grammar parse tree to component list for EACL ;; ============================================================================ +(declare transform-union-expr) + (defn- extract-base-expr-identifier "Extract identifier string from a base-expr node." [node] @@ -460,9 +515,20 @@ (when (and (vector? child) (= :identifier (first child))) (second child))))) +(defn- base-expr-paren-child + "Returns the inner permission-expr node when a base-expr wraps a paren-expr, else nil." + [node] + (when (and (vector? node) (= :base-expr (first node))) + (let [child (second node)] + (when (and (vector? child) (= :paren-expr (first child))) + (second child))))) + (defn- transform-arrow-expr "Transform an arrow expression to component maps. - Returns vector of {:type :identifier/:arrow, ...} maps." + Returns vector of {:type :identifier/:arrow, ...} maps. + Parenthesized union operands flatten (EACL is union-only, so `(a + b)` == `a + b`); + parens as arrow bases/targets are rejected during validation, with a defensive + throw here in case transform is called directly." [node] (cond ;; Arrow function expression: rel.any(perm) or rel.all(perm) @@ -475,15 +541,23 @@ ;; .any() is equivalent to arrow, .all() should have been rejected by validation [{:type :arrow :base {:type :identifier :name base-id} :path [target-id]}]) - ;; Simple arrow expression: rel->perm or rel->perm->perm2 + ;; Simple arrow expression: identifier, (paren union), or rel->perm chains (and (vector? node) (= :simple-arrow-expr (first node))) - (let [base-exprs (filter #(and (vector? %) (= :base-expr (first %))) (rest node)) - ids (map extract-base-expr-identifier base-exprs)] - (if (= 1 (count ids)) - ;; Single identifier - direct permission/relation reference - [{:type :identifier :name (first ids)}] - ;; Arrow expression - [{:type :arrow :base {:type :identifier :name (first ids)} :path (vec (rest ids))}])) + (let [base-exprs (filter #(and (vector? %) (= :base-expr (first %))) (rest node))] + (if (= 1 (count base-exprs)) + (let [base-expr (first base-exprs)] + (if-let [inner-permission-expr (base-expr-paren-child base-expr)] + ;; Parenthesized union operand: flatten to its components. + (vec (transform-union-expr (second inner-permission-expr))) + ;; Single identifier - direct permission/relation reference + [{:type :identifier :name (extract-base-expr-identifier base-expr)}])) + ;; Arrow chain: every element must be a plain identifier. + (let [ids (map extract-base-expr-identifier base-exprs)] + (when (some nil? ids) + (throw (ex-info "Parenthesized expressions are not supported as arrow bases or targets." + {:type :eacl.schema/paren-arrow + :node node}))) + [{:type :arrow :base {:type :identifier :name (first ids)} :path (vec (rest ids))}]))) ;; Wrapped arrow expr (and (vector? node) (= :arrow-expr (first node))) @@ -518,23 +592,18 @@ ;; ============================================================================ (defn- collect-schema-info - "Build lookup tables from transformed schema for arrow resolution." + "Build lookup tables from transformed schema for arrow resolution. + Relation subject types are kept as full sets so arrow resolution and + validation can never depend on declaration order." [definitions] (reduce-kv (fn [acc res-type {:keys [relations permissions]}] - (let [;; Get simple type names (first type ref for each relation) - relation-names (set (keys relations)) - ;; Map relation name to target type (first type in list) - relation-types (into {} - (for [[rel-name type-refs] relations - :let [first-ref (first type-refs)] - :when first-ref] - [rel-name (:type first-ref)]))] - (assoc acc res-type - {:relations relation-names - :relation-types relation-types - :relation-all-types relations - :permissions (set (map :name permissions))}))) + (assoc acc res-type + {:relations (set (keys relations)) + :relation-subject-types (into {} + (for [[rel-name type-refs] relations] + [rel-name (set (keep :type type-refs))])) + :permissions (set (map :name permissions))})) {} definitions)) @@ -559,14 +628,38 @@ path-elements (:path component) path (first path-elements) info (get schema-info resource-type) - target-type (get-in info [:relation-types base-name])] - (if-not target-type + subject-types (get-in info [:relation-subject-types base-name])] + (if (empty? subject-types) (throw (ex-info (str "Unknown relation for arrow base: " base-name " on " resource-type) {:component component :resource-type resource-type})) - (let [target-info (get schema-info target-type) - target-is-relation (contains? (:relations target-info) path)] - {:arrow (keyword base-name) - (if target-is-relation :relation :permission) (keyword path)}))) + ;; The target kind must be resolved against ALL subject types of the base + ;; relation, never just the first/last declared one — otherwise resolution + ;; and validation become declaration-order-dependent. + (let [kinds (set (map (fn [subject-type] + (let [target-info (get schema-info subject-type)] + (cond + (contains? (:relations target-info) path) :relation + (contains? (:permissions target-info) path) :permission + :else :missing))) + subject-types)) + present (disj kinds :missing)] + (cond + (= present #{:relation :permission}) + (throw (ex-info (str "Arrow target '" path "' resolves to a relation on some subject types of '" + base-name "' and a permission on others: " (pr-str subject-types)) + {:type :eacl.schema/mixed-arrow-target + :component component + :resource-type resource-type + :subject-types subject-types})) + + (= present #{:relation}) + {:arrow (keyword base-name) :relation (keyword path)} + + ;; :permission on all types that have it, or missing everywhere — + ;; construct a permission target and let validate-schema-references + ;; produce the per-type missing-target errors. + :else + {:arrow (keyword base-name) :permission (keyword path)})))) (throw (ex-info "Unsupported component type" {:component component})))) @@ -578,19 +671,28 @@ "Convert parsed SpiceDB schema to EACL internal representation. Steps: - 1. Transform parse tree to intermediate representation - 2. Validate EACL restrictions (throws on unsupported features) - 3. Convert to EACL Relations and Permissions + 1. Reject instaparse failures (a failed parse must never become an empty schema — + write-schema! diffs against the existing schema, so an empty result retracts everything) + 2. Transform parse tree to intermediate representation + 3. Validate EACL restrictions (throws on unsupported features) + 4. Convert to EACL Relations and Permissions - Returns {:relations [...] :permissions [...]}" + Returns {:definitions [...] :relations [...] :permissions [...]}" [parse-tree] + (when (insta/failure? parse-tree) + (let [failure (insta/get-failure parse-tree)] + (throw (ex-info (str "Schema parse error: " (pr-str failure)) + {:type :eacl.schema/parse-error + :failure failure})))) (let [transformed (transform-schema parse-tree)] ;; Validate EACL restrictions (parsing allows full SpiceDB, validation enforces limits) (validate-eacl-restrictions parse-tree transformed) (let [definitions (:definitions transformed) schema-info (collect-schema-info definitions)] - {:relations + {:definitions (vec (keys definitions)) + + :relations (vec ;; Expand multi-type relations into multiple Relation entities (for [[res-type {:keys [relations]}] definitions diff --git a/test/eacl/bench/pagination_test.clj b/test/eacl/bench/pagination_test.clj index b9dc645f..bd9ff166 100644 --- a/test/eacl/bench/pagination_test.clj +++ b/test/eacl/bench/pagination_test.clj @@ -62,7 +62,8 @@ (defn- tx-relationships [db relationships] - (mapcat #(impl/tx-relationship db %) relationships)) + ; :allow-tempids? because entities are created in the same transaction. + (mapcat #(impl/tx-relationship db % {:allow-tempids? true}) relationships)) ;; --- Seeding --- diff --git a/test/eacl/bench/recursive_pagination_test.clj b/test/eacl/bench/recursive_pagination_test.clj new file mode 100644 index 00000000..ff6e19f3 --- /dev/null +++ b/test/eacl/bench/recursive_pagination_test.clj @@ -0,0 +1,147 @@ +(ns eacl.bench.recursive-pagination-test + "Recursive pagination benchmarks for the stable frontier iterator. + + Baseline on eacl/v7 before this change (depth-1000 parent chain, internal API): + - lookup-resources limit=50: median ~1312ms + - count-resources limit=50: median ~1342ms + + This benchmark exercises the public API and ensures the recursive path + remains materially faster than the old full-set closure solver." + (:require [clojure.test :refer [deftest testing is]] + [datomic.api :as d] + [eacl.core :as eacl] + [eacl.datomic.core :as spiceomic] + [eacl.datomic.impl :as impl :refer [Relationship]] + [eacl.datomic.schema :as schema] + [eacl.datomic.datomic-helpers :refer [with-mem-conn]] + [eacl.datomic.fixtures :refer [->user ->account]])) + +(def recursive-parent-schema-dsl + "definition user {} + + definition account { + relation parent: account + relation reader: user + + permission read = reader + parent->read + }") + +(defn- tx-relationships + [db relationships] + (mapcat #(impl/tx-relationship db %) relationships)) + +(defn- seed-recursive-parent! + [conn depth] + (let [acl (spiceomic/make-client conn {})] + @(d/transact conn schema/v6-schema) + (eacl/write-schema! acl recursive-parent-schema-dsl) + @(d/transact conn + (into [{:db/id "user-1" :eacl/id "user-1"}] + (map (fn [i] + {:db/id (str "acc-" i) + :eacl/id (str "acc-" i)})) + (range depth))) + @(d/transact conn + (tx-relationships (d/db conn) + (concat + [(Relationship (->user "user-1") :reader (->account "acc-0"))] + (for [i (range (dec depth))] + (Relationship (->account (str "acc-" i)) + :parent + (->account (str "acc-" (inc i)))))))) + acl)) + +(defn- run-timed + [n f] + (mapv (fn [_] + (let [start (System/nanoTime) + _ (f) + end (System/nanoTime)] + (/ (double (- end start)) 1e6))) + (range n))) + +(defn- median + [coll] + (let [sorted (sort coll) + n (count sorted) + mid (quot n 2)] + (if (odd? n) + (nth sorted mid) + (/ (+ (nth sorted (dec mid)) (nth sorted mid)) 2.0)))) + +(defn- percentile + [coll p] + (let [sorted (sort coll) + idx (min (dec (count sorted)) + (int (Math/ceil (* (/ p 100.0) (count sorted)))))] + (nth sorted idx))) + +(def ^:private chain-depth 1000) +(def ^:private warmup-iterations 10) +(def ^:private bench-iterations 20) +(def ^:private pagination-iterations 10) +(def ^:private pages-per-run 20) +(def ^:private first-page-threshold-ms 50) +(def ^:private count-threshold-ms 50) +(def ^:private per-page-threshold-ms 10) + +(deftest ^:benchmark recursive-parent-pagination-benchmark + (testing "Recursive parent pagination performance" + (with-mem-conn [conn []] + (let [acl (seed-recursive-parent! conn chain-depth) + base-query {:subject (->user "user-1") + :permission :read + :resource/type :account + :limit 50 + :max-depth 2000}] + + (testing "first page lookup (limit=50)" + (run-timed warmup-iterations #(eacl/lookup-resources acl base-query)) + (let [times (run-timed bench-iterations #(eacl/lookup-resources acl base-query)) + med (median times) + p95 (percentile times 95)] + (println (format "Recursive first page (limit=50): median=%.2fms, p95=%.2fms, min=%.2fms, max=%.2fms" + med p95 (apply min times) (apply max times))) + (is (< med first-page-threshold-ms) + (format "REGRESSION: recursive first page median %.2fms exceeds %dms threshold" + med first-page-threshold-ms)))) + + (testing "count from the first page frontier (limit=50)" + (run-timed warmup-iterations #(eacl/count-resources acl base-query)) + (let [times (run-timed bench-iterations #(eacl/count-resources acl base-query)) + med (median times) + p95 (percentile times 95)] + (println (format "Recursive count (limit=50): median=%.2fms, p95=%.2fms, min=%.2fms, max=%.2fms" + med p95 (apply min times) (apply max times))) + (is (< med count-threshold-ms) + (format "REGRESSION: recursive count median %.2fms exceeds %dms threshold" + med count-threshold-ms)))) + + (testing "multi-page pagination" + (let [times (run-timed pagination-iterations + (fn [] + (loop [cursor nil + page 0] + (when (< page pages-per-run) + (let [result (eacl/lookup-resources acl (cond-> base-query + cursor (assoc :cursor cursor)))] + (recur (:cursor result) (inc page))))))) + med (median times) + per-page (/ med pages-per-run)] + (println (format "Recursive pagination (%d pages): median=%.2fms total (%.2fms/page), min=%.2fms, max=%.2fms" + pages-per-run med per-page (apply min times) (apply max times))) + (is (< per-page per-page-threshold-ms) + (format "REGRESSION: recursive per-page median %.2fms exceeds %dms threshold" + per-page per-page-threshold-ms)))) + + (testing "pagination set correctness" + (let [all-results (loop [cursor nil + acc []] + (let [{:keys [data cursor]} (eacl/lookup-resources acl (cond-> base-query + cursor (assoc :cursor cursor))) + acc' (into acc data)] + (if (and cursor (seq data)) + (recur cursor acc') + acc')))] + (is (= chain-depth (count all-results))) + (is (= chain-depth (count (distinct (map :id all-results))))))))))) diff --git a/test/eacl/benchmark_test.clj b/test/eacl/benchmark_test.clj deleted file mode 100644 index 63cbdf0b..00000000 --- a/test/eacl/benchmark_test.clj +++ /dev/null @@ -1,403 +0,0 @@ -;(ns eacl.benchmark-test -; (:require [criterium.core :as crit] -; [eacl.core :as eacl] -; [eacl.datomic.core :as spiceomic] -; ; [eacl.datomic.impl-base :as base :refer [Relation Relationship Permission]] -; [eacl.datomic.impl :as impl :refer [Relation Relationship Permission]] -; [eacl.datomic.schema :as schema] -; [datomic.api :as d] -; [eacl.datomic.fixtures :as fixtures :refer [->platform ->account ->user ->server]] -; [clojure.test :as t :refer [deftest testing is]] -; [clojure.tools.logging :as log])) -; -;;(defn rand-subject []) -; -;(defn tx! [conn tx-data] -; ;(prn tx-data) -; @(d/transact conn tx-data)) -; -;(defn ids->tempid-map [uuid-coll] -; (->> uuid-coll -; (reduce (fn [acc uuid] -; (assoc acc uuid (d/tempid :db.part/user))) -; {}))) -; -;(defn make-account-user-txes [account-tempid n] -; (let [user-uuids (repeatedly n d/squuid) -; user-uuid->tempid (ids->tempid-map user-uuids)] -; (for [user-uuid user-uuids] -; (let [user-tempid (user-uuid->tempid user-uuid)] -; [{:db/id user-tempid -; :eacl/id (str user-uuid) -; :user/account account-tempid} ; only to police permission checks. -; (Relationship (->user user-tempid) -; :owner -; (->account account-tempid))])))) -; -;(defn make-account-server-txes [account-tempid n] -; (let [server-uuids (repeatedly n d/squuid) -; server-uuid->tempid (ids->tempid-map server-uuids)] -; -; (for [server-uuid server-uuids] -; (let [server-tempid (server-uuid->tempid server-uuid)] -; [{:db/id server-tempid -; :eacl/id (str server-uuid) -; :server/account account-tempid ; only to police permission checks. -; :server/name (str "Servers " server-uuid)} -; (Relationship (->account account-tempid) :account (->server server-tempid))])))) -; -;(defn make-account-txes [{:keys [num-users num-servers]} account-uuid] -; (let [account-tempid (d/tempid :db.part/user) -; account-txes [{:db/id account-tempid -; :eacl/id (str account-uuid)} -; (Relationship (->platform [:eacl/id "platform"]) :platform (->account account-tempid))] -; user-txes (make-account-user-txes account-tempid num-users) -; server-txes (make-account-server-txes account-tempid num-servers)] -; (concat account-txes (flatten user-txes) (flatten server-txes)))) -; -;(defn server->user-ids [db server-id] -; (d/q '[:find [?user-id ...] -; :in $ ?server-id -; :where -; [?server :eacl/id ?server-id] -; [?server :server/account ?account] -; [?user :user/account ?account] -; [?user :eacl/id ?user-id]] -; db server-id)) -; -;(defn setup-benchmark [db] -; {:accounts (d/q '[:find [?account-id ...] -; :where -; [?server :server/account ?account] -; [?account :eacl/id ?account-id]] -; db) -; :users (d/q '[:find [?user-id ...] -; :where -; [?user :user/account ?account] -; [?user :eacl/id ?user-id]] -; db) -; :servers (d/q '[:find [?server-id ...] -; :where -; [?server :server/account ?account] -; [?server :eacl/id ?server-id]] -; db)}) -; -;(defn run-benchmark [!counter client {:keys [accounts users servers]}] -; ; now we cross-check each user for server and police that the value is correct -; ; do we need to shuffle these for accurate test? -; (doall (for [server-id servers -; user-id users] -; (do -; (swap! !counter inc) -; [user-id server-id (eacl/can? client (->user user-id) :view (->server server-id))])))) -; -;(defn check-results [server->user-set matrix] -; (for [[user-id server-id actual] matrix -; :let [user-set (server->user-set server-id) -; expected (contains? user-set server-id)]] -; [actual expected])) -; -;(defn rand-user [db] -; (d/q '[:find (rand ?user-uuid) . -; :in $ -; :where -; [?user :user/account ?account] -; [?user :eacl/id ?user-uuid]] -; db)) -; -;(defn rand-server [db] -; (d/q '[:find (rand ?server-uuid) . -; :in $ -; :where -; [?server :server/account ?account] -; [?server :eacl/id ?server-uuid]] -; db)) -; -;(deftest eacl-benchmarks -; ; todo switch to with-mem-conn. -; ;(def datomic-uri "datomic:dev://localhost:4597/eacl-benchmark") -; -; (comment -; (def datomic-uri "datomic:mem://mem-eacl-benchmark") -; (d/delete-database datomic-uri) -; (d/create-database datomic-uri) -; (def conn (d/connect datomic-uri)) -; (.release conn) -; #_[]) -; -; (comment -; -; (testing "Transact EACL Datomic Schema" -; (tx! conn (concat schema/v5-schema))) -; -; (testing "Transact a realistic EACL Permission Schema" -; (tx! conn fixtures/base-fixtures)) -; -; (testing "some schema to police our data" -; (tx! conn [{:db/ident :server/name -; :db/doc "Just to add some real data into the mix." -; :db/cardinality :db.cardinality/one -; :db/valueType :db.type/string -; :db/index true} -; -; {:db/ident :user/account -; :db/cardinality :db.cardinality/many -; :db/valueType :db.type/ref -; :db/index true} -; -; {:db/ident :server/account -; :db/cardinality :db.cardinality/one -; :db/valueType :db.type/ref -; :db/index true}])) -; -; (def test-account (d/q '[:find (rand ?account-uuid) . -; :where -; [?server :server/account ?account] -; [?account :eacl/id ?account-uuid]] -; (d/db conn))) -; -; test-account -; -; (def test-user (d/q '[:find (rand ?user-uuid) . -; :in $ ?account-uuid -; :where -; [?user :user/account ?account] -; [?user :eacl/id ?user-uuid]] -; (d/db conn) test-account)) -; -; (def client (spiceomic/make-client conn {})) -; -; ;; Add platform relations for all accounts: -; -; (let [platform-id (d/entid (d/db conn) [:eacl/id "platform"])] -; (->> (d/q '[:find [?account ...] -; :where [?server :server/account ?account]] -; (d/db conn)) -; (map (fn [acc] -; (Relationship (->platform platform-id) :platform (->account acc)))) -; (d/transact conn) -; (deref))) -; -; (time (count (:data (eacl/lookup-resources client {:subject (->user "super-user") -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 1000})))) -; -; (time (count (:data (eacl/lookup-resources client {:subject (->user "super-user") -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 100000000})))) -; -; (time (count (:data (eacl/lookup-resources client {:subject (->user "super-user") -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 1000})))) -; -; (let [test-user (rand-user (d/db conn))] -; (let [{:as page1 -; p1-data :data -; p1-cursor :cursor} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 600}) -; page1-count (count p1-data) -; -; {:as page2 -; p2-data :data -; p2-cursor :cursor} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor p1-cursor -; :limit 400}) -; page2-count (count p2-data) -; -; {:as page3 -; p3-data :data -; p3-cursor :cursor} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor p2-cursor -; :limit 100}) -; page3-count (count p3-data) -; -; total-count (+ page1-count page2-count page3-count) -; all-pages-data (concat p1-data p2-data p3-data) -; unique-count (count (distinct all-pages-data))] -; (assert (> total-count 700)) -; {:total/count total-count -; :count [page1-count page2-count page3-count] -; :cursors [p1-cursor p2-cursor p3-cursor] -; :data (concat p1-data p2-data p3-data) -; :unique-count unique-count})) -; -; (let [test-user (rand-user (d/db conn))] -; ; fetch two pages of data (we are not checking correctness here) -; (let [{:as page1 -; data :data -; p1-cursor :cursor} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 600}) -; c1 (count data) -; -; {:as page2 -; data :data -; p2-cursor :cursor} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor p1-cursor -; :limit 400}) -; c2 (count data)] -; [p1-cursor p2-cursor])) -; -; (eacl/count-resources client {:subject (->user "super-user") -; :permission :view -; :resource/type :server -; :cursor nil}) -; -; (crit/quick-bench -; (let [page (eacl/lookup-resources client {:subject (->user "super-user") -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 1000000000})])) -; -; (crit/quick-bench -; (let [test-user (rand-user (d/db conn))] -; ; fetch two pages of data (we are not checking correctness here) -; (let [{:as page1 :keys [data cursor]} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor nil -; :limit 600}) -; c1 (count data) -; -; {:as page2 :keys [data cursor]} -; (eacl/lookup-resources client {:subject (->user test-user) -; :permission :view -; :resource/type :server -; :cursor cursor -; :limit 400}) -; c2 (count data)] -; nil))) -; -; (let [test-server (rand-server (d/db conn))] -; (prn 'test-server test-server) -; (time (count (eacl/lookup-subjects client {:resource (->server test-server) -; :permission :view -; :subject/type :user})))) -; -; (time (count (eacl/read-relationships client {:resource/type :account}))) -; (time (count (eacl/read-relationships client {:resource/type :server}))) -; ()) -; -; ; Transact Test Data -; (let [num-accounts 100 -; num-users 10 -; num-servers 1000 -; account-uuids (repeatedly num-accounts d/squuid) -; account-txes (time (->> account-uuids -; (mapv (partial make-account-txes {:num-users num-users -; :num-servers num-servers})) -; (flatten))) -; tx-count (count account-txes)] -; ;(log/warn "Skipping txe.") -; (log/debug "Transacting " tx-count " things.") -; (tx! conn account-txes) -; -; (let [!counter (atom 0) -; !mistakes (atom 0) -; client (spiceomic/make-client conn {}) -; db (d/db conn) -; {:as setup :keys [accounts users servers]} (time (setup-benchmark db)) -; _ (do (log/debug "N accounts" (count accounts)) -; (log/debug "N users" (count users)) -; (log/debug "N servers" (count servers))) -; server->user-set (time (into {} -; (for [server-id servers] -; [server-id (set (server->user-ids db server-id))]))) -; server->users (time (into {} -; (for [server-id servers] -; [server-id (vec (server->user-ids db server-id))])))] -; ;(prn 'server->users server->users) -; (when false ; true ; false ; do -; (log/debug "Starting benchmark...") -; (crit/quick-bench -; (let [random-server (rand-nth servers) -; expected-userset (server->user-set random-server) -; expected-user-list (server->users random-server) -; ;_ (prn 'expected-user-list expected-user-list) -; random-user (if (and (>= (rand) 0.5) (seq expected-user-list)) ; some empty -; (rand-nth expected-user-list) -; (rand-nth users)) -; expected (contains? expected-userset random-user) -; _ (swap! !counter inc) -; actual (eacl/can? client (->user random-user) :view (->server random-server))] -; ;(log/debug expected random-user random-server) -; (when (not= expected actual) -; (swap! !mistakes inc)) -; [expected actual])) -; (prn 'counter @!counter 'mistakes @!mistakes))))) -; -;;(let [result-matrix (time (run-benchmark !counter client setup)) -;; checks (check-results server->user-set result-matrix) -;; matches (map (fn [[actual expected]] -;; (= actual expected)) checks) -;; disparities (frequencies matches)] -;; (log/debug 'disparities disparities) -;; (log/debug 'counter @!counter) -;; disparities)))) -; -;(comment -; ; ok so we have 100 accounts, 10 users per account, and 1000 servers per account -; ; th ameans we ahve 100 * 1000 = 100,000 servers. -; ; total users = 10 * 100 = 1,000 users -; ; users * servers = 100,000 * 1,000 = 100,000,000 M permission checks. -; -; (let [db (d/db conn) -; ; pull some relations -; users-cos (d/q '[:find ?user ?company -; :where -; [?user :user/username] -; [?rel :eacl/subject ?user] -; [?rel :eacl/relation :company/owner] -; [?rel :eacl/resource ?company]] -; db)] -; (prn (count users-cos) 'users-cos) -; (time -; (doall (frequencies (->> users-cos (pmap (fn [[user company]] -; (eacl/can? db user :company/view company)))))))) -; -; (let [subjects (d/q '[:find [?subject ...] -; :where -; [?subject :eacl/subject]] -; (d/db conn)) -; subject (rand-nth subjects)])) -;;(prn 'can? (eacl/can? (d/db conn) (:db/id subject) :company/view [:eacl/id (first cids)])))) -; -;(comment -; (d/q '[:find (count ?account) . -; :where -; [?account :eacl/type :account]] -; (d/db conn)) -; -; (d/q '[:find (count ?server) . -; :where -; [?server :eacl/type :server]] -; (d/db conn)) -; -; (d/q '[:find (count ?user) . -; :where -; [?user :eacl/type :user]] -; (d/db conn))) diff --git a/test/eacl/datomic/config_test.clj b/test/eacl/datomic/config_test.clj index d19bfab6..7c4ea59e 100644 --- a/test/eacl/datomic/config_test.clj +++ b/test/eacl/datomic/config_test.clj @@ -28,13 +28,13 @@ ; todo: also test read/write-relationships, and count-resources. - (testing "lookup-resources throws for missing subject ident with some detail" - (is (thrown? Throwable (eacl/lookup-resources client - {:subject (->user :missing-ident) - :permission :view - :resource/type :server - :limit 1000 - :cursor nil})))) + (testing "lookup-resources returns an empty page for a missing subject ident (SpiceDB-consistent, audit D9)" + (is (= [] (:data (eacl/lookup-resources client + {:subject (->user :missing-ident) + :permission :view + :resource/type :server + :limit 1000 + :cursor nil}))))) (testing "basic can? works when passing :db/ident" (is (true? (eacl/can? client (->user :test/user1) :view (->server :test/server1)))) diff --git a/test/eacl/datomic/differential_test.clj b/test/eacl/datomic/differential_test.clj new file mode 100644 index 00000000..6839f4d0 --- /dev/null +++ b/test/eacl/datomic/differential_test.clj @@ -0,0 +1,190 @@ +(ns eacl.datomic.differential-test + "Seeded randomized differential tests codifying the audit's cross-engine + invariant: for every (subject, permission, resource-type), + + lookup-resources set == can?-derived ground truth + == paginated union at several page sizes + == count-resources + + and the reverse via lookup-subjects. Hand-rolled seeded RNG — no new deps. + These invariants held for the engines during the 2026-07-06 audit; this + makes them executable against every future change." + (:require [clojure.test :refer [deftest testing is]] + [datomic.api :as d] + [eacl.core :refer [spice-object]] + [eacl.datomic.datomic-helpers :refer [with-mem-conn]] + [eacl.datomic.impl :as impl :refer [Relationship]] + [eacl.datomic.impl.indexed :as idx] + [eacl.datomic.schema :as schema])) + +(def ^:private differential-schema + "Exercises direct relations, arrow->permission, arrow->relation, + self-permission, and overlapping multi-path unions." + "definition user {} + + definition platform { + relation super_admin: user + } + + definition account { + relation owner: user + relation platform: platform + + permission admin = owner + platform->super_admin + } + + definition server { + relation account: account + relation shared: user + + permission admin = account->admin + shared + permission view = admin + shared + }") + +(def ^:private recursive-schema + "definition user {} + + definition folder { + relation parent: folder + relation reader: user + + permission read = reader + parent->read + }") + +(defn- rand-subset + [^java.util.Random rng coll p] + (vec (filter (fn [_] (< (.nextDouble rng) p)) coll))) + +(defn- entity-txes + [ids] + (mapv (fn [id] {:db/id id :eacl/id id}) ids)) + +(defn- eid-of [db id] (d/entid db [:eacl/id id])) + +(defn- collect-paged + "Collects a full paginated enumeration at the given page size." + [db query page-size] + (loop [cursor nil + acc []] + (let [page (idx/lookup-resources db (assoc query :limit page-size :cursor cursor)) + acc' (into acc (map :id (:data page)))] + (if (seq (:data page)) + (recur (:cursor page) acc') + acc')))) + +(defn- check-forward-invariants! + [db label subject permission resource-type all-resource-eids sorted?] + (let [query {:subject subject :permission permission :resource/type resource-type} + full (mapv :id (:data (idx/lookup-resources db (assoc query :limit -1)))) + truth (set (filter #(idx/can? db subject permission (spice-object resource-type %)) + all-resource-eids))] + (is (= truth (set full)) + (str label ": lookup-resources set must equal can? ground truth")) + (is (= (count full) (count (distinct full))) + (str label ": no duplicates")) + (when sorted? + (is (= full (sort full)) + (str label ": non-recursive results are in ascending eid order"))) + (doseq [page-size [1 3 7]] + (is (= full (collect-paged db query page-size)) + (str label ": paginated union at page size " page-size " must equal the full enumeration"))) + (is (= (count full) + (:count (idx/count-resources db (assoc query :limit -1)))) + (str label ": count-resources must agree")) + full)) + +(defn- check-reverse-invariants! + [db label resource permission subject-type all-subject-eids] + (let [subjects (mapv :id (:data (idx/lookup-subjects db {:resource resource + :permission permission + :subject/type subject-type + :limit -1}))) + truth (set (filter #(idx/can? db (spice-object subject-type %) permission resource) + all-subject-eids))] + (is (= truth (set subjects)) + (str label ": lookup-subjects set must equal can? ground truth")) + (is (= (count subjects) (count (distinct subjects))) + (str label ": no duplicate subjects")))) + +(deftest differential-nonrecursive-test + (doseq [seed [7 23 42 1337]] + (with-mem-conn [conn schema/v6-schema] + (let [rng (java.util.Random. (long seed)) + users (mapv #(str "user-" %) (range 3)) + accounts (mapv #(str "acct-" %) (range 3)) + servers (mapv #(str "srv-" %) (range 12)) + platform "platform-1"] + (schema/write-schema! conn differential-schema) + @(d/transact conn (entity-txes (concat users accounts servers [platform]))) + (let [db0 (d/db conn) + rels (concat + ;; platform super admins + (for [u (rand-subset rng users 0.3)] + (Relationship (spice-object :user u) :super_admin (spice-object :platform platform))) + ;; account owners + platform membership + (mapcat (fn [a] + (concat + (for [u (rand-subset rng users 0.5)] + (Relationship (spice-object :user u) :owner (spice-object :account a))) + (when (< (.nextDouble rng) 0.5) + [(Relationship (spice-object :platform platform) :platform (spice-object :account a))]))) + accounts) + ;; servers: account membership + direct shares + (mapcat (fn [s] + (concat + (when (< (.nextDouble rng) 0.8) + [(Relationship (spice-object :account (nth accounts (.nextInt rng (count accounts)))) + :account (spice-object :server s))]) + (for [u (rand-subset rng users 0.15)] + (Relationship (spice-object :user u) :shared (spice-object :server s))))) + servers))] + @(d/transact conn (into [] (mapcat #(impl/tx-relationship db0 %)) rels))) + (let [db (d/db conn) + server-eids (mapv #(eid-of db %) servers) + account-eids (mapv #(eid-of db %) accounts) + user-eids (mapv #(eid-of db %) users)] + (testing (str "seed " seed ": forward invariants for every user × permission") + (doseq [u users + [perm rt eids] [[:view :server server-eids] + [:admin :server server-eids] + [:admin :account account-eids]]] + (check-forward-invariants! db (str "seed " seed " user " u " " perm " " rt) + (spice-object :user (eid-of db u)) + perm rt eids true))) + (testing (str "seed " seed ": reverse invariants for every server") + (doseq [s servers] + (check-reverse-invariants! db (str "seed " seed " server " s) + (spice-object :server (eid-of db s)) + :view :user user-eids)))))))) + +(deftest differential-recursive-test + (doseq [seed [11 99]] + (with-mem-conn [conn schema/v6-schema] + (let [rng (java.util.Random. (long seed)) + folders (mapv #(str "folder-" %) (range 10)) + users ["reader-1" "reader-2"]] + (schema/write-schema! conn recursive-schema) + @(d/transact conn (entity-txes (concat folders users))) + (let [db0 (d/db conn) + rels (concat + ;; random parent edges — cycles are legal and must terminate + (keep (fn [f] + (when (< (.nextDouble rng) 0.6) + (let [parent (nth folders (.nextInt rng (count folders)))] + (when (not= parent f) + (Relationship (spice-object :folder parent) :parent (spice-object :folder f)))))) + folders) + ;; each user reads two random folders + (for [u users + f (take 2 (distinct [(nth folders (.nextInt rng (count folders))) + (nth folders (.nextInt rng (count folders))) + (nth folders (.nextInt rng (count folders)))]))] + (Relationship (spice-object :user u) :reader (spice-object :folder f))))] + @(d/transact conn (into [] (mapcat #(impl/tx-relationship db0 %)) rels))) + (let [db (d/db conn) + folder-eids (mapv #(eid-of db %) folders)] + (testing (str "seed " seed ": recursive forward invariants (stable discovery order, exact dedup)") + (doseq [u users] + (check-forward-invariants! db (str "seed " seed " user " u " :read :folder") + (spice-object :user (eid-of db u)) + :read :folder folder-eids false)))))))) diff --git a/test/eacl/datomic/fixtures.clj b/test/eacl/datomic/fixtures.clj index a3b95526..c3c4a5fb 100644 --- a/test/eacl/datomic/fixtures.clj +++ b/test/eacl/datomic/fixtures.clj @@ -352,7 +352,8 @@ (defn relationship-fixtures [db] - (mapcat #(impl/tx-relationship db %) relationship-fixture-data)) + ; :allow-tempids? because entity fixtures land in the same transaction. + (mapcat #(impl/tx-relationship db % {:allow-tempids? true}) relationship-fixture-data)) (defn base-fixtures [db] @@ -372,7 +373,7 @@ {:db/id "account-3" :db/ident :test/account3 :eacl/id "account-3"}] - (mapcat #(impl/tx-relationship db %) + (mapcat #(impl/tx-relationship db % {:allow-tempids? true}) [(Relationship (->account "account-3") :account (->server "account3-server3.1")) (Relationship (->user :test/user1) :owner (->account "account-3"))]))) diff --git a/test/eacl/datomic/impl/indexed_test.clj b/test/eacl/datomic/impl/indexed_test.clj index cc7214ad..38ffbbb9 100644 --- a/test/eacl/datomic/impl/indexed_test.clj +++ b/test/eacl/datomic/impl/indexed_test.clj @@ -11,7 +11,6 @@ :refer [Relation Relationship Permission can? read-relationships]] - ;[eacl.datomic.impl.datalog :as impl.datalog :refer [lookup-subjects]] [eacl.datomic.impl.indexed :as impl.indexed :refer [count-resources lookup-resources lookup-subjects]])) ; Test grouping & cleanup is in progress. @@ -86,6 +85,18 @@ [db {:as page :keys [data cursor]}] (set (paginated->spice db page))) +(defn collect-paginated-spice + [db lookup-fn query] + (loop [cursor nil + acc []] + (let [page (lookup-fn db (cond-> query cursor (assoc :cursor cursor))) + data (paginated->spice db page) + acc' (into acc data) + next-cur (:cursor page)] + (if (and next-cur (seq data)) + (recur next-cur acc') + acc')))) + (def recursive-parent-schema-string "definition user {} @@ -123,6 +134,80 @@ [eacl-id] (spice-object :account [:eacl/id eacl-id])) +(def duplicate-recursive-parent-schema-string + "definition user {} + + definition account { + relation parent: account + relation reader: user + + permission read = reader + parent->read + }") + +(defn- load-duplicate-parent-db! + [conn] + (schema/write-schema! conn duplicate-recursive-parent-schema-string) + @(d/transact conn [{:db/id "user-1" :eacl/id "user-1"} + {:db/id "a" :eacl/id "a"} + {:db/id "b" :eacl/id "b"} + {:db/id "c" :eacl/id "c"}]) + @(d/transact conn + (into [] + (mapcat #(impl/tx-relationship (d/db conn) %)) + [(Relationship (spice-object :user "user-1") :reader (spice-object :account "a")) + (Relationship (spice-object :user "user-1") :reader (spice-object :account "b")) + (Relationship (spice-object :account "a") :parent (spice-object :account "c")) + (Relationship (spice-object :account "b") :parent (spice-object :account "c"))])) + (d/db conn)) + +(defn- load-direct-plus-recursive-duplicate-db! + [conn] + (schema/write-schema! conn duplicate-recursive-parent-schema-string) + @(d/transact conn [{:db/id "user-1" :eacl/id "user-1"} + {:db/id "a" :eacl/id "a"} + {:db/id "b" :eacl/id "b"}]) + @(d/transact conn + (into [] + (mapcat #(impl/tx-relationship (d/db conn) %)) + [(Relationship (spice-object :user "user-1") :reader (spice-object :account "a")) + (Relationship (spice-object :user "user-1") :reader (spice-object :account "b")) + (Relationship (spice-object :account "a") :parent (spice-object :account "b"))])) + (d/db conn)) + +(defn- load-cycle-parent-db! + [conn] + (schema/write-schema! conn duplicate-recursive-parent-schema-string) + @(d/transact conn [{:db/id "user-1" :eacl/id "user-1"} + {:db/id "a1" :eacl/id "a1"} + {:db/id "a2" :eacl/id "a2"}]) + @(d/transact conn + (into [] + (mapcat #(impl/tx-relationship (d/db conn) %)) + [(Relationship (spice-object :user "user-1") :reader (spice-object :account "a1")) + (Relationship (spice-object :account "a1") :parent (spice-object :account "a2")) + (Relationship (spice-object :account "a2") :parent (spice-object :account "a1"))])) + (d/db conn)) + +(defn- load-deep-recursive-parent-db! + [conn depth] + (schema/write-schema! conn recursive-parent-schema-string) + @(d/transact conn + (into [{:db/id "user-1" :eacl/id "user-1"}] + (map (fn [i] + {:db/id (str "acc-" i) + :eacl/id (str "acc-" i)})) + (range depth))) + @(d/transact conn + (into [] + (mapcat #(impl/tx-relationship (d/db conn) %)) + (concat + [(Relationship (spice-object :user "user-1") :reader (spice-object :account "acc-0"))] + (for [i (range (dec depth))] + (Relationship (spice-object :account (str "acc-" i)) + :parent + (spice-object :account (str "acc-" (inc i)))))))) + (d/db conn)) + (deftest permission-helper-tests (testing "Permission helper with new unified API" (is (= #:eacl.permission{:eacl/id "eacl:permission::server::admin::self::relation::owner" @@ -345,17 +430,17 @@ (paginated->spice db) (set)))) - (testing "...and server { permission view_via_arrow_relation = account->view_via_arrow_relation } works") - (is (= #{(spice-object :server "account1-server1") - (spice-object :server "account1-server2") - (spice-object :server "account2-server1")} - (->> (lookup-resources db {:subject (->user super-user-eid) - :permission :view_server_via_arrow_relation - :resource/type :server - :limit 1000 - :cursor nil}) - (paginated->spice db) - (set))))) + (testing "...and server { permission view_server_via_arrow_relation = account->view_via_arrow_relation } works" + (is (= #{(spice-object :server "account1-server1") + (spice-object :server "account1-server2") + (spice-object :server "account2-server1")} + (->> (lookup-resources db {:subject (->user super-user-eid) + :permission :view_server_via_arrow_relation + :resource/type :server + :limit 1000 + :cursor nil}) + (paginated->spice db) + (set)))))) (testing "We can enumerate resources with lookup-resources" (is (= #{(spice-object :server "account1-server1") @@ -453,7 +538,7 @@ (is (can? db' (->user (d/entid db' [:eacl/id "user-1"])) :view (->server [:eacl/id "account2-server1"]))) (is (can? db' (->user (d/entid db' [:eacl/id "super-user"])) :view (->server [:eacl/id "account2-server1"]))) - (is (not (can? db' (->user (d/entid db' [:eacl/id "user2"])) :view (->server [:eacl/id "account2-server1"])))) + (is (not (can? db' (->user (d/entid db' [:eacl/id "user-2"])) :view (->server [:eacl/id "account2-server1"])))) (testing ":test/user2 cannot access any servers" ; is this correct? (is (= #{} (->> (lookup-resources db' {:resource/type :server @@ -529,11 +614,13 @@ (Relationship (->user :user/super-user) :shared_admin (->server :test/server1)))) (second (impl/tx-relationship (d/db *conn*) (Relationship (->user :user/super-user) :shared_admin (->server :test/server1)))) - ; We can use tempids in Relationship because tuple tx-data keeps the tempid. + ; Tempids in Relationship require explicit opt-in now (audit §12). (first (impl/tx-relationship (d/db *conn*) - (Relationship (->user :user/super-user) :shared_admin (->server "server3")))) + (Relationship (->user :user/super-user) :shared_admin (->server "server3")) + {:allow-tempids? true})) (second (impl/tx-relationship (d/db *conn*) - (Relationship (->user :user/super-user) :shared_admin (->server "server3"))))]))) + (Relationship (->user :user/super-user) :shared_admin (->server "server3")) + {:allow-tempids? true}))]))) (let [db' (d/db *conn*)] (testing "ensure user1 can only see servers from account1, so excludes server-3" @@ -871,6 +958,69 @@ :subject (->user super-user-eid) :cursor page2-cursor})))))))))))))))) +(deftest tx-relationship-strictness-test + ;; Audit §12: silent tempid pass-through minted ghost entities on typo'd ids. + (with-mem-conn [conn schema/v6-schema] + @(d/transact conn [(Relation :account :owner :user)]) + @(d/transact conn [{:eacl/id "alice"} {:eacl/id "acct-1"}]) + (let [db (d/db conn)] + (testing "a typo'd string id throws :eacl/unknown-object instead of minting a ghost entity" + (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Unknown object" + (impl/tx-relationship db + (Relationship (spice-object :user "alice") :owner (spice-object :account "acct-1x")))))) + + (testing "unallocated positive numeric eids are rejected with the typed error, not a raw transactor error" + (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Unknown object" + (impl/tx-relationship db + (Relationship (spice-object :user 17592186999999) :owner (spice-object :account [:eacl/id "acct-1"])))))) + + (testing "{:allow-tempids? true} supports same-transaction entity+relationship creation" + (let [tx (concat [{:db/id "new-user" :eacl/id "new-user"}] + (impl/tx-relationship db + (Relationship (spice-object :user "new-user") :owner (spice-object :account [:eacl/id "acct-1"])) + {:allow-tempids? true})) + {:keys [db-after]} @(d/transact conn tx)] + (is (impl/find-one-relationship-id db-after + {:subject (spice-object :user [:eacl/id "new-user"]) + :relation :owner + :resource (spice-object :account [:eacl/id "acct-1"])}))))))) + +(deftest relation-datoms-keyword-collation-test + ;; Audit §2: the old [:a]..[:z] index-range made relations whose subject-type + ;; keyword collates outside that window invisible to permission evaluation. + (with-mem-conn [conn schema/v6-schema] + @(d/transact conn [(Relation :zone :owner :zebra) ; sorts after :z + (Relation :zone :editor :Admin) ; sorts before :a + (Relation :zone :viewer :my.app/user) ; namespaced + (Relation :zone :ownerx :user) ; prefix-isolation foil for :owner + (Permission :zone :admin {:relation :owner})]) + @(d/transact conn [{:db/id "z1" :eacl/id "zebra-1"} + {:db/id "zone1" :eacl/id "zone-1"}]) + @(d/transact conn (impl/tx-relationship (d/db conn) + (Relationship (spice-object :zebra [:eacl/id "zebra-1"]) + :owner + (spice-object :zone [:eacl/id "zone-1"])))) + (let [db (d/db conn)] + (testing "relations are visible for any legal subject-type keyword" + (is (= [[:zone :editor :Admin]] (mapv :v (impl.indexed/relation-datoms db :zone :editor)))) + (is (= [[:zone :viewer :my.app/user]] (mapv :v (impl.indexed/relation-datoms db :zone :viewer))))) + + (testing "prefix isolation: (:zone :owner) does not match (:zone :ownerx) or later attributes" + (is (= [[:zone :owner :zebra]] (mapv :v (impl.indexed/relation-datoms db :zone :owner))))) + + (testing "end-to-end: permission evaluation works for a :zebra subject" + (let [zebra (spice-object :zebra (d/entid db [:eacl/id "zebra-1"])) + zone (spice-object :zone (d/entid db [:eacl/id "zone-1"]))] + (is (true? (can? db zebra :admin zone))) + (is (= [(spice-object :zone "zone-1")] + (paginated->spice db (lookup-resources db {:subject zebra + :permission :admin + :resource/type :zone})))) + (is (= [(spice-object :zebra "zebra-1")] + (paginated->spice db (lookup-subjects db {:resource zone + :permission :admin + :subject/type :zebra}))))))))) + (deftest permission-schema-helper-tests (let [db (d/db *conn*)] @@ -1164,6 +1314,124 @@ :resource/type :account :limit 100}))))))))) +(deftest recursive-arrow-permission-pagination-stability-test + (with-mem-conn [conn schema/v6-schema] + (let [db (load-recursive-parent-db! conn) + user (recursive-user-ref "user-1") + paged-query {:subject user + :permission :read + :resource/type :account + :limit 2} + full-query (assoc paged-query :limit 100) + page-order (collect-paginated-spice db lookup-resources paged-query) + full-order (paginated->spice db (lookup-resources db full-query))] + (testing "recursive pagination should equal the large-limit query in order and membership" + (is (= full-order page-order))) + (testing "recursive pagination should be stable across repeated runs on the same db basis" + (is (= page-order + (collect-paginated-spice db lookup-resources paged-query)))) + (testing "recursive pagination should not emit duplicates across pages" + (is (= (count page-order) + (count (distinct page-order)))))))) + +(deftest recursive-arrow-permission-deduplication-tests + (testing "direct and recursive duplicates are emitted once" + (with-mem-conn [conn schema/v6-schema] + (let [db (load-direct-plus-recursive-duplicate-db! conn) + user (recursive-user-ref "user-1") + results (collect-paginated-spice db lookup-resources + {:subject user + :permission :read + :resource/type :account + :limit 1})] + (is (= #{(spice-object :account "a") + (spice-object :account "b")} + (set results))) + (is (= (count results) (count (distinct results))))))) + + (testing "two recursive branches yielding the same descendant are emitted once" + (with-mem-conn [conn schema/v6-schema] + (let [db (load-duplicate-parent-db! conn) + user (recursive-user-ref "user-1") + results (collect-paginated-spice db lookup-resources + {:subject user + :permission :read + :resource/type :account + :limit 1})] + (is (= #{(spice-object :account "a") + (spice-object :account "b") + (spice-object :account "c")} + (set results))) + (is (= (count results) (count (distinct results)))))))) + +(deftest recursive-arrow-permission-data-cycle-test + (with-mem-conn [conn schema/v6-schema] + (let [db (load-cycle-parent-db! conn) + user (recursive-user-ref "user-1") + results (collect-paginated-spice db lookup-resources + {:subject user + :permission :read + :resource/type :account + :limit 1})] + (testing "real data loops terminate and return the reachable set once" + (is (= #{(spice-object :account "a1") + (spice-object :account "a2")} + (set results))) + (is (= (count results) (count (distinct results))))) + (testing "count-resources agrees with paginated lookup on a data cycle" + (is (= 2 + (:count (count-resources db {:subject user + :permission :read + :resource/type :account + :limit 10})))))))) + +(deftest recursive-arrow-permission-max-depth-test + (with-mem-conn [conn schema/v6-schema] + (let [db (load-deep-recursive-parent-db! conn 52) + user (recursive-user-ref "user-1") + leaf (recursive-account-ref "acc-51")] + (testing "default max-depth 50 fails on a deeper chain" + (is (thrown-with-msg? + clojure.lang.ExceptionInfo + #"max depth" + (lookup-resources db {:subject user + :permission :read + :resource/type :account + :limit 100})))) + (testing "explicitly larger max-depth allows the same query" + (is (= 52 + (count (paginated->spice db + (lookup-resources db {:subject user + :permission :read + :resource/type :account + :limit 100 + :max-depth 60})))))) + (testing "can? and lookup-subjects use the same max-depth contract" + (is (thrown-with-msg? + clojure.lang.ExceptionInfo + #"max depth" + (can? db {:subject user + :permission :read + :resource leaf}))) + (is (thrown-with-msg? + clojure.lang.ExceptionInfo + #"max depth" + (lookup-subjects db {:resource leaf + :permission :read + :subject/type :user + :limit 100}))) + (is (true? (can? db {:subject user + :permission :read + :resource leaf + :max-depth 60}))) + (is (= #{(spice-object :user "user-1")} + (paginated->spice-set db + (lookup-subjects db {:resource leaf + :permission :read + :subject/type :user + :limit 100 + :max-depth 60})))))))) + ; uncommented because server :owner relation went away. ;(testing "Performance - early termination" ; ;; Add multiple paths that grant the same permission diff --git a/test/eacl/datomic/parser_test.clj b/test/eacl/datomic/parser_test.clj index 9a4829ca..49c75729 100644 --- a/test/eacl/datomic/parser_test.clj +++ b/test/eacl/datomic/parser_test.clj @@ -1,9 +1,15 @@ -(ns eacl.datomic.parser_test +(ns eacl.datomic.parser-test + ;; ns renamed from eacl.datomic.parser_test: the cognitect test-runner default + ;; pattern #".*-test$" did not match the underscore name, so these tests were + ;; silently excluded from `clj -X:test` runs. (:require [clojure.test :as t :refer [deftest testing is]] [instaparse.core :as insta] [eacl.spicedb.parser :as parser] [eacl.datomic.impl :as impl])) +(defn- ex-type [f] + (try (f) nil (catch clojure.lang.ExceptionInfo e (:type (ex-data e))))) + (def example-schema-string "definition user {} @@ -152,4 +158,101 @@ (testing ".all() arrow function is rejected during validation" (let [schema "definition doc { relation group: group permission view = group.all(member) }"] (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Unsupported function: \.all\(\)" - (parser/->eacl-schema (parser/parse-schema schema))))))) \ No newline at end of file + (parser/->eacl-schema (parser/parse-schema schema))))))) + +(deftest parse-failure-safety-tests + (testing "a failed parse throws a typed error and never coerces to an empty schema" + (is (= :eacl.schema/parse-error + (ex-type #(parser/->eacl-schema (parser/parse-schema "definition user {"))))) + (testing "the error carries instaparse failure detail (line/column)" + (try + (parser/->eacl-schema (parser/parse-schema "definition user { relation owner user }")) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl.schema/parse-error (:type (ex-data e)))) + (is (:failure (ex-data e))))))) + + (testing "transform-schema throws on non-schema input instead of returning nil" + (is (= :eacl.schema/parse-error + (ex-type #(parser/transform-schema (parser/parse-schema "definition user {"))))))) + +(deftest comment-support-tests + (testing "// line comments and /* */ block comments are whitespace" + (let [commented "// leading comment + definition user {} + /* block + comment */ + definition account { + relation owner: user // trailing comment + permission admin = owner /* inline */ + owner + }" + plain "definition user {} + definition account { + relation owner: user + permission admin = owner + owner + }" + parse #(parser/->eacl-schema (parser/parse-schema %))] + (is (= (parse plain) (parse commented))))) + + (testing "comment-only input is still a parse error (a schema needs definitions)" + (is (= :eacl.schema/parse-error + (ex-type #(parser/->eacl-schema (parser/parse-schema "// nothing here"))))))) + +(deftest duplicate-declaration-tests + (testing "duplicate definition blocks are rejected, not silently last-won" + (is (= :eacl.schema/duplicate-definition + (ex-type #(parser/->eacl-schema + (parser/parse-schema "definition user {} + definition account { relation owner: user } + definition account { relation viewer: user }")))))) + + (testing "duplicate relation declarations within a definition are rejected" + (is (= :eacl.schema/duplicate-relation + (ex-type #(parser/->eacl-schema + (parser/parse-schema "definition user {} + definition doc { relation owner: user relation owner: user }")))))) + + (testing "a multi-type relation declared once with | is not a duplicate" + (is (= 2 (count (:relations (parser/->eacl-schema + (parser/parse-schema "definition user {} + definition group {} + definition doc { relation owner: user | group }"))))))) + + (testing "a permission sharing a name with a relation on the same definition is rejected" + (is (= :eacl.schema/name-collision + (ex-type #(parser/->eacl-schema + (parser/parse-schema "definition user {} + definition doc { relation x: user permission x = x }"))))))) + +(deftest paren-expression-tests + (testing "parenthesized union operands flatten to their components" + (let [parse #(parser/->eacl-schema (parser/parse-schema %)) + paren (parse "definition user {} + definition d { relation owner: user relation editor: user + permission manage = (owner + editor) }") + plain (parse "definition user {} + definition d { relation owner: user relation editor: user + permission manage = owner + editor }")] + (is (= (set (:permissions plain)) (set (:permissions paren)))) + (testing "nested parens and mixed operands also flatten" + (is (= (set (:permissions plain)) + (set (:permissions (parse "definition user {} + definition d { relation owner: user relation editor: user + permission manage = ((owner)) + (editor) }")))))))) + + (testing "parenthesized arrow bases are rejected with a clear validation error, not an AssertionError" + (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Parenthesized expressions" + (parser/->eacl-schema + (parser/parse-schema "definition user {} + definition d { relation a: user relation b: user + permission p = (a + b)->c }")))))) + +(deftest arrow-target-kind-tests + (testing "arrow target resolving to mixed kinds across subject types is rejected" + ;; mgmt is a RELATION on user but a PERMISSION on group. + (is (= :eacl.schema/mixed-arrow-target + (ex-type #(parser/->eacl-schema + (parser/parse-schema "definition user { relation mgmt: user } + definition group { relation lead: user permission mgmt = lead } + definition account { relation owner: user | group + permission admin = owner->mgmt }"))))))) \ No newline at end of file diff --git a/test/eacl/datomic/performance_test.clj b/test/eacl/datomic/performance_test.clj deleted file mode 100644 index 068e514a..00000000 --- a/test/eacl/datomic/performance_test.clj +++ /dev/null @@ -1,221 +0,0 @@ -;(ns eacl.datomic.performance-test -; "Performance benchmarks for EACL optimizations" -; (:require [clojure.test :as t :refer [deftest testing is]] -; [datomic.api :as d] -; [eacl.datomic.datomic-helpers :refer [with-mem-conn]] -; [eacl.datomic.fixtures :as fixtures :refer [->user ->server ->account ->vpc]] -; [eacl.core :as eacl :refer [spice-object]] -; [eacl.datomic.schema :as schema] -; [eacl.datomic.impl :as impl :refer [Relation Relationship Permission]] -; [eacl.datomic.rules :as original-rules] -; [eacl.datomic.rules.optimized :as rules] -; [clojure.tools.logging :as log])) -; -;(defn measure-time -; "Measure execution time of a function in milliseconds" -; [f] -; (let [start (System/nanoTime) -; result (f) -; end (System/nanoTime) -; duration-ms (/ (- end start) 1000000.0)] -; {:result result -; :time-ms duration-ms})) -; -;(defn generate-test-data -; "Generate test data with specified number of users, accounts, and servers" -; [conn num-users num-accounts num-servers prefix] -; (let [users (for [i (range num-users)] -; {:db/id (str prefix "-user-" i) -; :entity/id (str prefix "-user-" i) -; :resource/type :user}) -; -; accounts (for [i (range num-accounts)] -; {:db/id (str prefix "-account-" i) -; :entity/id (str prefix "-account-" i) -; :resource/type :account}) -; -; servers (for [i (range num-servers)] -; {:db/id (str prefix "-server-" i) -; :entity/id (str prefix "-server-" i) -; :resource/type :server}) -; -; ;; Create relationships -; ;; Each user owns an account -; user-account-rels (for [i (range (min num-users num-accounts))] -; (Relationship (->user (str prefix "-user-" i)) -; :owner -; (->account (str prefix "-account-" i)))) -; -; ;; Each account has servers -; servers-per-account (quot num-servers num-accounts) -; account-server-rels (for [i (range num-accounts) -; j (range servers-per-account)] -; (Relationship (->account (str prefix "-account-" i)) -; :account -; (->server (str prefix "-server-" (+ (* i servers-per-account) j)))))] -; -; @(d/transact conn (concat users accounts servers -; user-account-rels account-server-rels)))) -; -;(defn run-performance-comparison -; "Run performance comparison between original and optimized rules" -; [db test-name query-fn original-rules optimized-rules] -; (log/debug (str "\n" test-name ":")) -; -; ;; Test with original rules -; (let [{:keys [time-ms result]} (measure-time #(query-fn db original-rules))] -; (log/debug (format " Original: %.2f ms (found %d results)" -; time-ms (count result)))) -; -; ;; Test with optimized rules -; (let [{:keys [time-ms result]} (measure-time #(query-fn db optimized-rules))] -; (log/debug (format " Optimized: %.2f ms (found %d results)" -; time-ms (count result))))) -; -;(deftest performance-comparison-test -; (testing "Performance comparison between original and optimized rules" -; (with-mem-conn [conn schema/v5-schema] -; ;; Set up base schema -; @(d/transact conn fixtures/base-fixtures) -; -; ;; Generate larger test dataset -; (log/debug "\nGenerating test data...") -; (generate-test-data conn 100 20 500 "perf1") -; -; (let [db (d/db conn)] -; -; ;; Test can? performance -; (run-performance-comparison -; db "can? check (direct permission)" -; (fn [db rules] -; (let [subject (d/entity db [:entity/id "perf1-user-0"]) -; resource (d/entity db [:entity/id "perf1-server-0"])] -; (log/debug 'subject subject 'resource resource) -; (d/q '[:find ?subject -; :in $ % ?subject ?perm ?resource -; :where -; (has-permission ?subject ?perm ?resource)] -; db rules -; (:db/id subject) -; :view -; (:db/id resource)))) -; original-rules/check-permission-rules -; rules/check-permission-rules) -; -; ;; Test lookup-subjects performance -; (run-performance-comparison -; db "lookup-subjects (find who can view server-0)" -; (fn [db rules] -; (let [resource (d/entity db [:entity/id "perf1-server-0"])] -; (d/q '[:find [?subject ...] -; :in $ % ?subject-type ?permission ?resource-eid -; :where -; (has-permission ?subject-type ?subject ?permission ?resource-eid) -; [(not= ?subject ?resource-eid)]] -; db rules -; :user -; :view -; (:db/id resource)))) -; original-rules/rules-lookup-subjects -; rules/rules-lookup-subjects) -; -; ;; Test lookup-resources performance (highest priority) -; (run-performance-comparison -; db "lookup-resources (find servers user-0 can view)" -; (fn [db rules] -; (let [subject (d/entity db [:entity/id "perf1-user-0"])] -; (d/q '[:find [?resource ...] -; :in $ % ?subject-type ?subject-eid ?permission ?resource-type -; :where -; (has-permission ?subject-type ?subject-eid ?permission ?resource-type ?resource)] -; db rules -; :user -; (:db/id subject) -; :view -; :server))) -; original-rules/rules-lookup-resources -; rules/rules-lookup-resources) -; -; ;; Test with larger dataset -; (log/debug "\n\nGenerating larger test dataset...") -; (generate-test-data conn 500 100 2000 "perf2") -; (let [db-large (d/db conn)] -; -; (log/debug "\nLarger dataset tests:") -; -; ;; Test lookup-resources with larger dataset -; (run-performance-comparison -; db-large "lookup-resources on larger dataset" -; (fn [db rules] -; (let [subject (d/entity db [:entity/id "perf2-user-10"])] -; (d/q '[:find [?resource ...] -; :in $ % ?subject-type ?subject-eid ?permission ?resource-type -; :where -; (has-permission ?subject-eid ?permission ?resource-type ?resource) -; [?resource :resource/type ?resource-type]] -; db rules -; :user -; (:db/id subject) -; :view -; :server))) -; original-rules/rules-lookup-resources -; rules/rules-lookup-resources)))))) -; -;(deftest staged-lookup-resources-test -; (testing "Staged lookup-resources implementation" -; (with-mem-conn [conn schema/v5-schema] -; @(d/transact conn fixtures/base-fixtures) -; (generate-test-data conn 100 20 100000 "staged") -; -; (let [db (d/db conn)] -; (log/debug "\n\nStaged lookup-resources test:") -; -; ;; Test original implementation -; (let [{:keys [time-ms result]} -; (measure-time -; #(impl/lookup-resources db {:resource/type :server -; :permission :view -; :subject (->user "staged-user-5") -; :limit 5000000}))] -; (log/debug (format " Full implementation: %.2f ms (found %d results)" -; time-ms (count result))))))) -; -; (testing "Staged lookup-resources with limit & offset implementation" -; (with-mem-conn [conn schema/v5-schema] -; @(d/transact conn fixtures/base-fixtures) -; (generate-test-data conn 100 20 100000 "staged") -; -; (let [db (d/db conn)] -; (log/debug "\n\nStaged lookup-resources test with limit & offset:") -; -; ;; Test original implementation -; (let [{:keys [time-ms result]} -; (measure-time -; #(impl/lookup-resources db {:offset 2000 -; :limit 1500 -; :resource/type :server -; :permission :view -; :subject (->user "staged-user-5")}))] -; (log/debug (format " Full implementation: %.2f ms (found %d results)" -; time-ms (count result))))))) -; -; (testing "Staged lookup-resources with small limit, large offset" -; (with-mem-conn [conn schema/v5-schema] -; @(d/transact conn fixtures/base-fixtures) -; (generate-test-data conn 100 20 100000 "staged") -; -; (let [db (d/db conn)] -; (log/debug "\n\nStaged lookup-resources test with small limit, large offset:") -; -; ;; Test original implementation -; (let [{:keys [time-ms result]} -; (measure-time -; #(impl/lookup-resources db {:offset 2000 -; :limit 20 -; :resource/type :server -; :permission :view -; :subject (->user "staged-user-5")}))] -; (log/debug (format " Full implementation: %.2f ms (found %d results)" -; time-ms (count result)))))))) -; -;;; Run with: clj -M:test -n eacl.datomic.performance-test \ No newline at end of file diff --git a/test/eacl/datomic/schema_basis_test.clj b/test/eacl/datomic/schema_basis_test.clj new file mode 100644 index 00000000..89281815 --- /dev/null +++ b/test/eacl/datomic/schema_basis_test.clj @@ -0,0 +1,163 @@ +(ns eacl.datomic.schema-basis-test + "Pins the write-schema!-driven cache-invalidation contract (issue #74) and + the Datomic view-classification facts the cache scope relies on, plus the + audit §3 regression (no cache-slot sharing across db bases). + + The contract: ONLY eacl.datomic.schema/write-schema! invalidates the + permission-path caches — it bumps :eacl/schema-version in the same + transaction as any definition change. Unrelated d/transact calls must leave + every cache key untouched. Programmatic edits of relation/permission datoms + bypass the stamp and are explicitly unsupported (issue #74)." + (:require [clojure.test :refer [deftest testing is]] + [datomic.api :as d] + [eacl.core :refer [spice-object]] + [eacl.datomic.datomic-helpers :refer [with-mem-conn]] + [eacl.datomic.impl :as impl :refer [Relation Permission Relationship]] + [eacl.datomic.impl.indexed :as idx] + [eacl.datomic.schema :as schema])) + +(def ^:private schema-v1 + "definition user {} + definition account { relation owner: user + permission admin = owner }") + +(def ^:private schema-v2 + "definition user {} + definition account { relation owner: user + relation viewer: user + permission admin = owner + viewer }") + +(deftest pinned-datomic-behaviors-test + (with-mem-conn [conn schema/v6-schema] + (schema/write-schema! conn schema-v1) + (let [db1 (d/db conn) + _ (schema/write-schema! conn schema-v2) + db2 (d/db conn)] + + (testing ".id returns the same database UUID on plain/as-of/with views (scope key component)" + (is (= (str (.id db2)) + (str (.id (d/as-of db2 (d/basis-t db1)))) + (str (.id (:db-after (d/with db2 []))))))) + + (testing "view classification predicates identify their views; with-dbs read as plain" + (is (false? (d/is-filtered db2))) + (is (false? (d/is-filtered (d/as-of db2 (d/basis-t db1))))) + (is (false? (d/is-filtered (d/since db2 (d/basis-t db1))))) + (is (true? (d/is-filtered (d/filter db2 (fn [_ _] true))))) + (is (true? (d/is-history (d/history db2)))) + (is (false? (d/is-history db2))) + (is (nil? (d/as-of-t (:db-after (d/with db2 []))))) + (is (nil? (d/since-t (:db-after (d/with db2 []))))) + (is (some? (d/since-t (d/since db2 (d/basis-t db1)))))) + + (testing "the version datom is a plain cardinality-one assert: as-of views read their era's value" + (is (some? (idx/schema-version db1))) + (is (not= (idx/schema-version db1) (idx/schema-version db2))) + (is (= (idx/schema-version db1) + (idx/schema-version (d/as-of db2 (d/basis-t db1))))))))) + +(deftest write-schema-invalidation-test + ;; Audit §3 regression, reworked for issue #74: invalidation is signaled by + ;; write-schema!'s version bump (visible to every peer via the db), not + ;; derived from db content and not dependent on the local eviction. + (with-mem-conn [conn schema/v6-schema] + (schema/write-schema! conn schema-v1) + @(d/transact conn [{:db/id "u" :eacl/id "u"} {:db/id "a" :eacl/id "a"}]) + @(d/transact conn (impl/tx-relationship (d/db conn) + (Relationship (spice-object :user [:eacl/id "u"]) + :owner + (spice-object :account [:eacl/id "a"])))) + (idx/evict-permission-paths-cache!) + (let [db1 (d/db conn) + u (spice-object :user [:eacl/id "u"]) + a (spice-object :account [:eacl/id "a"])] + (is (true? (idx/can? db1 u :admin a))) + (is (= 1 (count (idx/get-permission-paths db1 :account :admin))) "cache populated") + + (testing "write-schema! invalidates via the version key alone — no local eviction needed (the cross-peer story)" + (with-redefs [idx/evict-permission-paths-cache! (fn [] nil)] + (schema/write-schema! conn schema-v2)) + (let [db2 (d/db conn)] + (is (= 2 (count (idx/get-permission-paths db2 :account :admin))) + "new version, new cache slot: paths recomputed without eviction") + (is (true? (idx/can? db2 u :admin a))) + + (testing "as-of at the pre-change basis resolves the HISTORICAL paths, even after both were cached" + (is (= 1 (count (idx/get-permission-paths (d/as-of db2 (d/basis-t db1)) :account :admin)))) + (is (= 2 (count (idx/get-permission-paths db2 :account :admin))))))) + + (testing "programmatic schema edits are invisible to the caches — by design (issue #74)" + (let [viewer-perm-eid (d/q '[:find ?e . + :where + [?e :eacl.permission/resource-type :account] + [?e :eacl.permission/permission-name :admin] + [?e :eacl.permission/target-name :viewer]] + (d/db conn))] + @(d/transact conn [[:db.fn/retractEntity viewer-perm-eid]]) + (is (= 2 (count (idx/get-permission-paths (d/db conn) :account :admin))) + "raw d/transact did not bump the version: stale paths served until the next write-schema!") + (idx/evict-permission-paths-cache!) + (is (= 1 (count (idx/get-permission-paths (d/db conn) :account :admin))) + "manual evict-permission-paths-cache! is the recovery hatch")))))) + +(deftest unrelated-transact-keeps-cache-test + ;; Issue #74 itself: relationship/application writes must not bust the path + ;; cache — neither before any write-schema! (nil version) nor after one. + (with-mem-conn [conn schema/v6-schema] + @(d/transact conn [(Relation :account :owner :user) + (Permission :account :admin {:relation :owner})]) + @(d/transact conn [{:eacl/id "u"} {:eacl/id "a"}]) + (idx/evict-permission-paths-cache!) + (let [calls (atom 0) + orig idx/calc-permission-paths] + (with-redefs [idx/calc-permission-paths (fn [& args] + (swap! calls inc) + (apply orig args))] + (testing "pre-version databases (no write-schema! yet) still cache across unrelated writes" + (idx/get-permission-paths (d/db conn) :account :admin) + (is (= 1 @calls)) + @(d/transact conn (impl/tx-relationship (d/db conn) + (Relationship (spice-object :user [:eacl/id "u"]) + :owner + (spice-object :account [:eacl/id "a"])))) + (idx/get-permission-paths (d/db conn) :account :admin) + (is (= 1 @calls) + "unchanged schema across relationship writes must hit the cache")) + + (testing "after write-schema!, the version stays put across unrelated writes: cache stays hot" + (schema/write-schema! conn schema-v1) + (idx/get-permission-paths (d/db conn) :account :admin) + (let [warm @calls] + @(d/transact conn [{:eacl/id "unrelated-entity"}]) + @(d/transact conn (impl/tx-relationship (d/db conn) + (Relationship (spice-object :user [:eacl/id "u"]) + :owner + (spice-object :account [:eacl/id "unrelated-entity"])))) + (idx/get-permission-paths (d/db conn) :account :admin) + (is (= warm @calls) + "d/transact of relationships/entities must not recompute paths (issue #74)"))))))) + +(deftest filtered-views-cannot-poison-cache-test + (with-mem-conn [conn schema/v6-schema] + (schema/write-schema! conn schema-v1) + (let [db (d/db conn) + perm-eid (d/q '[:find ?e . + :where + [?e :eacl.permission/resource-type :account] + [?e :eacl.permission/permission-name :admin]] + db)] + (testing "a d/filter db hiding the permission cannot publish empty paths under the plain db's key" + (idx/evict-permission-paths-cache!) + (let [filtered (d/filter db (fn [_db datom] (not= perm-eid (:e datom)))) + filt-paths (idx/get-permission-paths filtered :account :admin)] + (is (= 0 (count filt-paths)) "the filtered view itself must not see the permission") + (is (= 1 (count (idx/get-permission-paths db :account :admin))) + "the plain db, queried AFTER the filtered view, must still see it") + (is (nil? (idx/schema-version-stamp filtered)) + "filter views are unclassifiable and never share the cache"))) + + (testing "since and history views are likewise unclassifiable" + (is (nil? (idx/schema-version-stamp (d/since db 0)))) + (is (nil? (idx/schema-version-stamp (d/history db)))) + (is (some? (idx/schema-version-stamp db))) + (is (some? (idx/schema-version-stamp (d/as-of db (d/basis-t db))))))))) diff --git a/test/eacl/datomic/schema_test.clj b/test/eacl/datomic/schema_test.clj index 05568f0d..d8a7ce25 100644 --- a/test/eacl/datomic/schema_test.clj +++ b/test/eacl/datomic/schema_test.clj @@ -4,7 +4,8 @@ [eacl.datomic.datomic-helpers :refer [with-mem-conn]] [eacl.datomic.schema :as schema] [eacl.datomic.fixtures :as fixtures] - [eacl.datomic.impl :as impl])) + [eacl.datomic.impl :as impl] + [eacl.spicedb.parser])) (def example-schema-string "definition user {} @@ -170,6 +171,70 @@ (is (= 3 (count (:relations schema)))) (is (= 3 (count (:permissions schema))))))))) +(deftest write-schema-parse-failure-test + (testing "a malformed schema string throws a typed error and leaves the stored schema untouched" + (with-mem-conn [conn schema/v6-schema] + (schema/write-schema! conn example-schema-string) + (let [before (schema/read-schema (d/db conn))] + (try + ;; missing closing brace — pre-fix this silently retracted the ENTIRE schema + (schema/write-schema! conn "definition user {} + definition account { + relation owner: user + permission admin = owner") + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl.schema/parse-error (:type (ex-data e)))))) + (is (= before (schema/read-schema (d/db conn))) + "schema must be unchanged after a failed write")))) + + (testing "a schema containing comments (e.g. pasted from the SpiceDB playground) writes cleanly" + (with-mem-conn [conn schema/v6-schema] + (is (schema/write-schema! conn "// users of the system + definition user {} + /* accounts own things */ + definition account { + relation owner: user // the owner + permission admin = owner + }")) + (is (= 1 (count (:relations (schema/read-schema (d/db conn))))))))) + +(deftest write-schema-empty-guard-test + (testing "zero-definition output cannot wipe a non-empty schema (parser-gap belt-and-braces)" + (with-mem-conn [conn schema/v6-schema] + (schema/write-schema! conn example-schema-string) + (with-redefs [eacl.spicedb.parser/->eacl-schema (fn [_] {:definitions [] :relations [] :permissions []})] + (try + (schema/write-schema! conn "anything") + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl.schema/empty-schema-guard (:type (ex-data e)))))) + (testing "explicit opt-in allows the wipe when nothing would be orphaned" + (is (schema/write-schema! conn "anything" {:allow-empty-schema? true})) + (is (= {:relations [] :permissions []} (schema/read-schema (d/db conn))))))))) + +(deftest arrow-validation-order-independence-test + (let [schema-with-owner-types (fn [types] + (str "definition user { relation boss: user permission mgmt = boss } + definition group {} + definition account { relation owner: " types " + permission admin = owner->mgmt }"))] + (testing "arrow targets are validated against ALL subject types, regardless of declaration order" + ;; mgmt exists on user but not group: both orders must be rejected identically. + (doseq [types ["user | group" "group | user"]] + (with-mem-conn [conn schema/v6-schema] + (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Invalid schema" + (schema/write-schema! conn (schema-with-owner-types types))) + (str "owner: " types " should be rejected — mgmt missing on group"))))) + + (testing "accepted when the target exists on every subject type" + (with-mem-conn [conn schema/v6-schema] + (is (schema/write-schema! conn + "definition user { relation boss: user permission mgmt = boss } + definition group { relation lead: user permission mgmt = lead } + definition account { relation owner: user | group + permission admin = owner->mgmt }")))))) + (deftest fixtures-schema-round-trip-test "Tests that fixtures.schema can be written and read back correctly. ADR 012 requirement: 'Rewrite the fixtures... to a new test/eacl/fixtures.schema file'" diff --git a/test/eacl/spice_test.clj b/test/eacl/spice_test.clj index fc3c4acf..3c3839b7 100644 --- a/test/eacl/spice_test.clj +++ b/test/eacl/spice_test.clj @@ -10,6 +10,13 @@ [clojure.tools.logging :as log] [eacl.spicedb.consistency :as consistency :refer [fully-consistent]])) +(defn- invalid-cursor-reason + [f] + (try (f) nil + (catch clojure.lang.ExceptionInfo e + (when (= :eacl/invalid-cursor (:type (ex-data e))) + (:reason (ex-data e)))))) + (deftest opaque-cursor-token-test (testing "cursor->token round-trip preserves cursor" (let [cursor {:v 2 :e "some-id" :p {0 "intermediate-id"}} @@ -22,14 +29,206 @@ (testing "nil cursor produces nil token" (is (nil? (spiceomic/cursor->token nil)))) - (testing "nil or invalid input returns nil cursor" - (is (nil? (spiceomic/token->cursor nil))) - (is (nil? (spiceomic/token->cursor "garbage"))) - (is (nil? (spiceomic/token->cursor "eacl1_not-valid-base64!!!")))) + (testing "nil means first page" + (is (nil? (spiceomic/token->cursor nil)))) + + (testing "invalid input throws a typed error instead of silently restarting pagination (audit §7)" + (is (= :undecodable (invalid-cursor-reason #(spiceomic/token->cursor "garbage")))) + (is (= :undecodable (invalid-cursor-reason #(spiceomic/token->cursor "eacl1_not-valid-base64!!!"))))) + + (testing "tokens do not expire by default; expiry is enforced only when the client configures :cursor-ttl-seconds" + (let [cursor {:v 2 :e 123 :p {}} + expired-token (spiceomic/cursor->token cursor {:cursor-ttl-seconds -10})] + (testing "a stale token decodes fine under default (no-TTL) config" + (is (= cursor (spiceomic/token->cursor expired-token)))) + (testing "the same token throws {:reason :expired} when a TTL is configured" + (is (= :expired (invalid-cursor-reason #(spiceomic/token->cursor expired-token {:cursor-ttl-seconds 60}))))) + (testing "tokens without :t never expire, even with a TTL configured" + (is (= cursor (spiceomic/token->cursor (spiceomic/cursor->token cursor) {:cursor-ttl-seconds 60})))))) (testing "backward compat: raw cursor map passes through token->cursor" (let [cursor {:v 2 :e 12345 :p {0 67890}}] - (is (= cursor (spiceomic/token->cursor cursor)))))) + (is (= cursor (spiceomic/token->cursor cursor))))) + + (testing "v3 recursive cursor round-trips without coercion" + (let [cursor {:v 3 + :mode :recursive-forward + :max-depth 50 + :stack [{:kind :direct-stream + :node [:account :read] + :relation-eid 42 + :cursor 1001 + :depth 50}] + :best-depth {[:account :read] {1001 50}} + :emitted #{1001} + :last 1001} + token (spiceomic/cursor->token cursor)] + (is (= cursor (spiceomic/token->cursor token))) + (is (= cursor (spiceomic/default-internal-cursor->spice nil {} cursor))) + (is (= cursor (spiceomic/default-spice-cursor->internal nil {} cursor)))))) + +(deftest protocol-completeness-tests + ;; Audit §13: write-relationship!/delete-relationship! were declared on the + ;; protocol but unimplemented -> AbstractMethodError. + (with-mem-conn [conn schema/v6-schema] + (let [client (spiceomic/make-client conn {}) + u1 (spice-object :user "u1") + a1 (spice-object :account "a1")] + (eacl/write-schema! client "definition user {} + definition account { relation owner: user permission admin = owner }") + @(d/transact conn [{:eacl/id "u1"} {:eacl/id "a1"}]) + + (testing "write-relationship! (positional arity) creates and returns a token" + (let [{token :zed/token} (eacl/write-relationship! client :touch u1 :owner a1)] + (is (string? token)) + (is (true? (eacl/can? client u1 :admin a1))))) + + (testing "delete-relationship! (positional arity) removes" + (eacl/delete-relationship! client u1 :owner a1) + (is (false? (eacl/can? client u1 :admin a1)))) + + (testing "map arities work" + (eacl/write-relationship! client {:operation :touch :subject u1 :relation :owner :resource a1}) + (is (true? (eacl/can? client u1 :admin a1))) + (eacl/delete-relationship! client {:subject u1 :relation :owner :resource a1}) + (is (false? (eacl/can? client u1 :admin a1)))) + + (testing "unsupported consistency throws a typed error (not an assert)" + (try + (eacl/can? client u1 :admin a1 (consistency/fresh "tok")) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/unsupported-consistency (:type (ex-data e))))))) + + (testing "expand-permission-tree throws a typed not-implemented error" + (try + (eacl/expand-permission-tree client {:resource a1 :permission :admin}) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/not-implemented (:type (ex-data e)))))))))) + +(deftest cursor-schema-fingerprint-tests + (with-mem-conn [conn schema/v6-schema] + (let [client (spiceomic/make-client conn {}) + schema-v1 "definition user {} + definition account { relation owner: user relation viewer: user + permission admin = owner } + definition widget { relation owner: user relation editor: user + permission view = owner }" + ;; changes widget/view only — account/admin paths are untouched + schema-v2 "definition user {} + definition account { relation owner: user relation viewer: user + permission admin = owner } + definition widget { relation owner: user relation editor: user + permission view = owner + editor }" + ;; changes account/admin's own paths + schema-v3 "definition user {} + definition account { relation owner: user relation viewer: user + permission admin = owner + viewer } + definition widget { relation owner: user relation editor: user + permission view = owner + editor }"] + (eacl/write-schema! client schema-v1) + @(d/transact conn [{:eacl/id "u1"} {:eacl/id "a1"} {:eacl/id "a2"} {:eacl/id "a3"}]) + (eacl/create-relationships! client + [(->Relationship (->user "u1") :owner (->account "a1")) + (->Relationship (->user "u1") :owner (->account "a2")) + (->Relationship (->user "u1") :owner (->account "a3"))]) + (let [q {:subject (->user "u1") :permission :admin :resource/type :account :limit 2} + page1 (eacl/lookup-resources client q)] + (is (= ["a1" "a2"] (mapv :id (:data page1)))) + + (testing "unchanged schema resumes exactly, no duplicates or gaps" + (is (= ["a3"] (mapv :id (:data (eacl/lookup-resources client (assoc q :cursor (:cursor page1)))))))) + + (testing "an UNRELATED schema change does not invalidate the cursor" + (eacl/write-schema! client schema-v2) + (is (= ["a3"] (mapv :id (:data (eacl/lookup-resources client (assoc q :cursor (:cursor page1)))))))) + + (testing "a schema change to THIS query's paths throws :eacl/stale-cursor" + (eacl/write-schema! client schema-v3) + (try + (eacl/lookup-resources client (assoc q :cursor (:cursor page1))) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/stale-cursor (:type (ex-data e))))))))))) + +(deftest strict-object-id-resolution-tests + (with-mem-conn [conn schema/v6-schema] + (let [client (spiceomic/make-client conn {})] + (eacl/write-schema! client "definition user {} + definition account { relation owner: user permission admin = owner }") + @(d/transact conn [{:eacl/id "alice"} {:eacl/id "bob"} {:eacl/id "acct-1"} {:eacl/id "acct-2"}]) + (eacl/create-relationships! client + [(->Relationship (spice-object :user "alice") :owner (spice-object :account "acct-1")) + (->Relationship (spice-object :user "bob") :owner (spice-object :account "acct-2"))]) + + (testing "read-relationships with a nonexistent subject returns [], not ALL relationships (audit §4)" + (is (= [] (vec (eacl/read-relationships client {:resource/type :account + :subject/id "i-do-not-exist"})))) + (is (= 1 (count (eacl/read-relationships client {:resource/type :account + :subject/id "alice"}))))) + + (testing "lookups and counts return empty results for unknown objects (SpiceDB-consistent, D9)" + (is (= [] (:data (eacl/lookup-resources client {:subject (spice-object :user "ghost") + :permission :admin + :resource/type :account})))) + (is (= 0 (:count (eacl/count-resources client {:subject (spice-object :user "ghost") + :permission :admin + :resource/type :account})))) + (is (= [] (:data (eacl/lookup-subjects client {:resource (spice-object :account "no-such-acct") + :permission :admin + :subject/type :user})))) + (is (false? (eacl/can? client (spice-object :user "ghost") :admin (spice-object :account "acct-1"))))) + + (testing "writes to unknown objects throw :eacl/unknown-object naming the object (audit §11)" + (try + (eacl/create-relationships! client + [(->Relationship (spice-object :user "ghost-user") :owner (spice-object :account "acct-1"))]) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/unknown-object (:type (ex-data e)))) + (is (= {:type :user :id "ghost-user"} (:object (ex-data e))))))))) + + (testing "make-client config validation (audit §5)" + (with-mem-conn [conn schema/v6-schema] + (let [setup (spiceomic/make-client conn {})] + (eacl/write-schema! setup "definition user {} + definition account { relation owner: user permission admin = owner }") + @(d/transact conn [{:eacl/id "u1"} {:eacl/id "a1"}]) + (eacl/create-relationships! setup + [(->Relationship (spice-object :user "u1") :owner (spice-object :account "a1"))]) + + (testing "the README-documented :entid->object-id key is honored" + (let [ext-client (spiceomic/make-client conn + {:entid->object-id (fn [db eid] (str "EXT-" (:eacl/id (d/entity db eid))))})] + (is (= ["EXT-a1"] + (mapv :id (:data (eacl/lookup-resources ext-client {:subject (spice-object :user "u1") + :permission :admin + :resource/type :account}))))))) + + (testing "the deprecated :entity->object-id alias still works" + (let [alias-client (spiceomic/make-client conn + {:entity->object-id (fn [ent] (str "ALIAS-" (:eacl/id ent)))})] + (is (= ["ALIAS-a1"] + (mapv :id (:data (eacl/lookup-resources alias-client {:subject (spice-object :user "u1") + :permission :admin + :resource/type :account}))))))) + + (testing "unknown option keys fail fast instead of silently falling back to defaults" + (try + (spiceomic/make-client conn {:entid->objectid (fn [_db eid] eid)}) ; misspelled + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/invalid-config (:type (ex-data e)))) + (is (= [:entid->objectid] (:unknown-keys (ex-data e))))))) + + (testing "supplying both the canonical key and the alias is rejected" + (try + (spiceomic/make-client conn {:entid->object-id (fn [_db eid] eid) + :entity->object-id (fn [ent] (:eacl/id ent))}) + (is false "should have thrown") + (catch clojure.lang.ExceptionInfo e + (is (= :eacl/invalid-config (:type (ex-data e))))))))))) (deftest spicedb-helper-tests (testing "spice-object takes [type id ?relation] and yields a SpiceObject with support for subject_relation" diff --git a/test_cursor_pagination.clj b/test_cursor_pagination.clj deleted file mode 100644 index 93d8eb2b..00000000 --- a/test_cursor_pagination.clj +++ /dev/null @@ -1,85 +0,0 @@ -(ns test-cursor-pagination - (:require [clojure.test :as t :refer [deftest testing is]] - [datomic.api :as d] - [eacl.datomic.datomic-helpers :refer [with-mem-conn]] - [eacl.datomic.fixtures :as fixtures :refer [->user ->server ->account]] - [eacl.core :as eacl :refer [spice-object]] - [eacl.datomic.schema :as schema] - [eacl.datomic.impl :as impl :refer [Relation Relationship Permission]] - [eacl.datomic.impl.indexed :as impl.indexed :refer [lookup-resources]])) - -(deftest cursor-pagination-bug-reproduction - "Reproduces the cursor pagination bug where intermediate relationships get filtered incorrectly" - (with-mem-conn [conn schema/v6-schema] - ;; Set up schema and entities to create the bug scenario - @(d/transact conn - [;; Schema - (Relation :account :owner :user) - (Relation :server :account :account) - (Permission :server :view {:arrow :account :permission :admin}) - (Permission :account :admin {:relation :owner}) - - ;; Entities - order matters for entity IDs - ;; Create servers first (lower eids) - {:db/id "server-1" :eacl/id "server-1"} - {:db/id "server-2" :eacl/id "server-2"} - {:db/id "server-3" :eacl/id "server-3"} - {:db/id "server-4" :eacl/id "server-4"} - - ;; Create accounts after servers (higher eids) - {:db/id "account-low" :eacl/id "account-low"} - {:db/id "account-high" :eacl/id "account-high"} - - ;; Create user - {:db/id "user-1" :eacl/id "user-1"} - - ;; Relationships - (Relationship (->user "user-1") :owner (->account "account-low")) - (Relationship (->user "user-1") :owner (->account "account-high")) - (Relationship (->account "account-low") :account (->server "server-1")) - (Relationship (->account "account-low") :account (->server "server-2")) - (Relationship (->account "account-high") :account (->server "server-3")) - (Relationship (->account "account-high") :account (->server "server-4"))]) - - (let [db (d/db conn) - user1-eid (d/entid db [:eacl/id "user-1"]) - server2-eid (d/entid db [:eacl/id "server-2"]) - account-low-eid (d/entid db [:eacl/id "account-low"]) - account-high-eid (d/entid db [:eacl/id "account-high"])] - - (println "Entity IDs:") - (println " server-1:" (d/entid db [:eacl/id "server-1"])) - (println " server-2:" (d/entid db [:eacl/id "server-2"])) - (println " server-3:" (d/entid db [:eacl/id "server-3"])) - (println " server-4:" (d/entid db [:eacl/id "server-4"])) - (println " account-low:" account-low-eid) - (println " account-high:" account-high-eid) - - (testing "Page 1 with limit 2 should return first 2 servers" - (let [page1 (lookup-resources db {:subject (->user user1-eid) - :permission :view - :resource/type :server - :limit 2 - :cursor nil})] - (println "Page 1 results:" (count (:data page1))) - (println "Page 1 cursor:" (:cursor page1)) - (is (= 2 (count (:data page1)))))) - - (testing "Page 2 with cursor from page 1 should return remaining servers" - (let [page1 (lookup-resources db {:subject (->user user1-eid) - :permission :view - :resource/type :server - :limit 2 - :cursor nil}) - page2 (lookup-resources db {:subject (->user user1-eid) - :permission :view - :resource/type :server - :limit 2 - :cursor (:cursor page1)})] - (println "Page 2 results:" (count (:data page2))) - (println "Page 2 cursor:" (:cursor page2)) - ;; This should NOT be empty - this is the bug we're fixing - (is (= 2 (count (:data page2))) "Page 2 should have 2 results, not be empty")))))) - -;; Run the test -(cursor-pagination-bug-reproduction) \ No newline at end of file