Skip to content

feat: expose job operations over the C ABI #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
0xgleb merged 8 commits into from
Aug 12, 2026
Merged

feat: expose job operations over the C ABI #31

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
bbd11c7
expose C ABI job operations on the landed stream foundation
0xgleb Aug 11, 2026
ccc8914
pin concurrent close joiners inside the wait before the destroyer runs
0xgleb Aug 11, 2026
e6f57f3
give the haskell CI job room for its cold-cache build
0xgleb Aug 12, 2026
c1f1cd8
package the Haskell binding by feature
0xgleb Aug 12, 2026
951c895
Merge pull request #32 from dataclique/refactor/0.4-haskell-features
0xgleb Aug 12, 2026
29d5d07
carry exact C struct types through the capi stubs and satisfy fourmolu
0xgleb Aug 12, 2026
f66d042
page oversized stream loads instead of refusing them
0xgleb Aug 12, 2026
9b69daf
format the paged stream load test
0xgleb Aug 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,43 @@ jobs:
- name: examples
run: nix run .#check-examples

haskell:
runs-on: ubuntu-latest
# The first uncached build compiles the GHC dependency closure, which
# takes far longer than the steady state; cachix carries the closure
# across runs, so the bound only has to cover the cold start once.
timeout-minutes: 45

steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

- uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
with:
nix_version: "2.31.2"
nix_conf: |
accept-flake-config = true
fallback = true
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
keep-env-derivations = true
keep-outputs = true

- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
continue-on-error: true
with:
name: data-cartel-public
authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}

- uses: nix-community/cache-nix-action@7df957e333c1e5da7721f60227dbba6d06080569 # v7
with:
primary-key: nix-${{ runner.os }}-haskell-${{ hashFiles('**/*.nix', '**/flake.lock') }}
restore-prefixes-first-match: nix-${{ runner.os }}-haskell-
gc-max-store-size-linux: 5G

- name: haskell
run: nix build .#checks.x86_64-linux.haskell -L

hooks:
runs-on: ubuntu-latest
timeout-minutes: 10
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ cbindgen = "=0.29.2"
ciborium = "0.2.2"
cqrs-es = "0.5.0"
event-sorcery = { path = "crates/event-sorcery" }
getrandom = "0.3.4"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
sqlite-es = { path = "crates/sqlite-es" }
Expand Down
25 changes: 16 additions & 9 deletions SPEC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,15 +264,22 @@ boundary:

ABI major version 0 has these normative resource limits:

| Resource | Limit |
| ----------------------------------------------- | -----: |
| Encoded request buffer | 16 MiB |
| Encoded response buffer, including error detail | 64 MiB |
| CBOR container nesting depth | 32 |
| One opaque domain payload | 1 MiB |
| Events in one commit | 1,024 |
| Items in one page or list response | 4,096 |
| UTF-8 error-detail text | 4 KiB |
| Resource | Limit |
| ----------------------------------------------- | ---------------: |
| Encoded request buffer | 16 MiB |
| Encoded response buffer, including error detail | 64 MiB |
| CBOR container nesting depth | 32 |
| One opaque domain payload | 1 MiB |
| Events in one commit | 1,024 |
| Items in one page or list response | 4,096 |
| UTF-8 error-detail text | 4 KiB |
| UTF-8 identifier text | 4 KiB |
| Store busy timeout | 2,147,483,647 ms |

An identifier is any caller-supplied name the store persists or matches on: an
aggregate type or id, a job id or kind, and a worker name. The busy-timeout
bound is what the SQLite driver accepts; a larger value is refused at open
rather than failing later as an infrastructure error.
Comment on lines +267 to +282

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

State the error code for a rejected busy timeout and clarify whether the identifier limit is per identifier.

Three gaps block a binding author from implementing these two new rows.

The busy-timeout row does not behave like the other rows in this table. Line 286 states that exceeding any limit returns RESOURCE_LIMIT with no partial write. The new paragraph instead says the value "is refused at open" but names no code. State which code open returns, and say whether it is RESOURCE_LIMIT or MALFORMED_INPUT.

The identifier row does not say whether 4 KiB applies to each identifier or to all identifiers in one request. Compare "One opaque domain payload", which is explicit. Use the same wording.

The RESOURCE_LIMIT detail at line 310 is [resource, observed, limit], but no section defines the stable resource names. Two bindings will emit different strings for the same limit. Add the identifier for each row in this table.

📝 Proposed wording 
-| UTF-8 identifier text                           |            4 KiB |
+| One UTF-8 identifier text value                 |            4 KiB |
 | Store busy timeout                              | 2,147,483,647 ms |
 
 An identifier is any caller-supplied name the store persists or matches on: an
 aggregate type or id, a job id or kind, and a worker name. The busy-timeout
 bound is what the SQLite driver accepts; a larger value is refused at open
-rather than failing later as an infrastructure error.
+with `MALFORMED_INPUT` rather than failing later as an infrastructure error.
+
+Each row above has a stable `resource` name used in the `RESOURCE_LIMIT`
+detail. Those names are part of ABI major version 0.

As per path instructions: "For topic references, suggest improvements that would make non-obvious framework behavior or pitfalls easier to discover."

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@SPEC.md` around lines 267 - 282, Update the resource-limit table and
surrounding specification text to state that an out-of-range busy timeout is
rejected at open with MALFORMED_INPUT, not RESOURCE_LIMIT. Clarify that the 4
KiB UTF-8 identifier limit applies independently to each identifier. Define and
use stable resource names for every limit row, including busy timeout and
identifier text, so RESOURCE_LIMIT details are consistent across bindings.

Source: Path instructions


Lengths and counts MUST be checked before proportional allocation or durable
work whenever the encoded shape makes that possible. Exceeding any limit MUST
Expand Down
11 changes: 6 additions & 5 deletions bindings/haskell/event-sorcery.cabal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,12 @@ library
hs-source-dirs: src
exposed-modules:
EventSorcery.Engine
EventSorcery.Engine.Codec
EventSorcery.Engine.Protocol
EventSorcery.Engine.Internal.FFI
EventSorcery.Stream

other-modules:
EventSorcery.Engine.Acquisition
EventSorcery.Engine.Internal.FFI
EventSorcery.Engine.Internal

extra-libraries: event_sorcery_ffi
build-depends:
Expand All @@ -66,11 +66,11 @@ library
, event-sorcery:acquisition
, text >=2.0 && <2.2

test-suite codec-spec
test-suite wire-spec
import: strict
type: exitcode-stdio-1.0
hs-source-dirs: test
main-is: CodecSpec.hs
main-is: WireSpec.hs
build-depends:
, base >=4.18 && <5
, bytestring >=0.11 && <0.13
Expand All @@ -85,6 +85,7 @@ test-suite engine-spec
hs-source-dirs: test
main-is: EngineSpec.hs
other-modules: EventSorcery.Engine.AcquisitionSpec
ghc-options: -threaded -with-rtsopts=-N
build-depends:
, base >=4.18 && <5
, bytestring >=0.11 && <0.13
Expand Down
Loading