Skip to content

feat(async): event-loop async support — fd primitive, adapters, native Swoole, shared reactor - #138

Draft
CodeLieutenant wants to merge 8 commits into
trunkfrom
claude/scylladb-php-async-support-904e8f
Draft

feat(async): event-loop async support — fd primitive, adapters, native Swoole, shared reactor#138
CodeLieutenant wants to merge 8 commits into
trunkfrom
claude/scylladb-php-async-support-904e8f

Conversation

@CodeLieutenant

Copy link
Copy Markdown
Member

Implements #24 — makes the driver usable inside non-blocking PHP event loops instead of only blocking on Future::get().

The C driver resolves futures on its own IO thread(s); this exposes that completion to userland so any loop can await a query without blocking. get() stays blocking, so existing sync code is unaffected and pays nothing.

What's in it

1. Core primitive (C) — works with any loop, zero deps
Every Future gains:

$future->getResource(): resource   // readable once the future resolves
$future->isReady(): bool

getResource() returns a php_stream that becomes readable exactly once when the driver resolves the future (via cass_future_set_callback + a self-pipe). Watch it with stream_select/ReactPHP/AMPHP/Swoole; on readable, get() returns without blocking. Linux uses eventfd (1 syscall, 1 fd, no SIGPIPE); macOS falls back to a pipe. src/FutureNotifier.{c,h}.

2. Framework adapters (lib/Async/, optional peer deps — require-dev + suggest)
Revolt (fiber await, the default), Amp (toFuture), ReactPhp (toPromise), Swoole (coroutine waitEvent).

3. Native Swoole/OpenSwoole — opt-in build flag
-DPHP_SCYLLADB_ENABLE_SWOOLE / _OPENSWOOLE (+ -DPHP_SCYLLADB_SWOOLE_SRC=…) compiles one small C++ shim (src/Async/SwooleBridge.cc) so Future::get() suspends the current coroutine instead of blocking. Default build stays pure C; swoole symbols resolve lazily at runtime.

4. Shared reactor — opt-in, O(1) fds for high fan-out
Cassandra\Async\Reactor (::add/resource/poll/pending): one eventfd + a mutex-guarded MPSC completion queue shared across many futures, so the loop watches one fd regardless of concurrency. Per-future fds cap out at FD_SETSIZE (~512 on macOS); the reactor scales to thousands and is even a touch faster at matched fan-out. lib/Async/ReactorRevolt.php adapter. See docs/async.md.

Testing

  • tests/Feature/Async/* against live ScyllaDB: primitive (stream_select multiplexing, error propagation, early-free SIGPIPE regression), Revolt/Amp/React (concurrency), Swoole (skip-guarded), reactor (io_threads=4 multi-producer, errors, model-mixing guards, Revolt adapter). All green; full suite 880 passed.
  • Benchmarks benchmarks/Live/{AsyncBench,ReactorBench}.php + committed baselines under benchmarks/baselines/ (JSON + Markdown, via composer bench:baseline:export).
  • fd/mem leak checks clean; two adversarial cross-thread C reviews (the reactor one caught + fixed a narrow-window UAF — signal now under the lock).

Notes for reviewers

  • Draft because two things want CI/hardware this environment lacks:
    • Native Swoole path: the flag-off default build + the C++ shim compile-check against real swoole-src are verified here, but the runtime coroutine behavior needs an actual (Open)Swoole runtime (not installed locally). Its tests are skip-guarded.
    • Reactor cross-thread code: local macOS ASan doesn't instrument cleanly, so the Linux CI ASan job is the authoritative race gate (covered locally by the adversarial review + the io_threads=4 stress + leak checks).
  • Generated *_arginfo.h / *_descriptor.c are build-time artifacts (git-ignored), regenerated from the committed stubs.
  • The one pre-existing full-suite failure (UuidTest › unique UUIDs across separate processes) is environmental — it spawns php -d extension=cassandra children by bare name and passes in CI where the extension is installed.

🤖 Generated with Claude Code

CodeLieutenant and others added 8 commits July 15, 2026 18:24
…e Swoole, shared reactor

Implements #24: let the driver participate in non-blocking PHP event loops
instead of only blocking on Future::get().

Core primitive (C, src/FutureNotifier.{c,h}):
- Every Future gains getResource(): resource and isReady(): bool. getResource()
  returns a php_stream that becomes readable once, when the driver resolves the
  future on its IO thread (cass_future_set_callback + self-pipe). Any loop
  (stream_select / ReactPHP / AMPHP / Swoole) can await without blocking; get()
  stays blocking, so sync users pay nothing.
- Refcounted, persistent-malloc notifier. Linux eventfd fast path (1 syscall,
  1 fd, no SIGPIPE) with a POSIX pipe fallback on macOS; SIGPIPE-safe dup for the
  stream; register-then-recheck closes lost-wakeup races.

Framework adapters (lib/Async, optional peer deps — require-dev + suggest):
- Revolt (fiber await, default), Amp (toFuture), ReactPhp (toPromise),
  Swoole (coroutine waitEvent).

Native Swoole/OpenSwoole (opt-in build flag):
- -DPHP_SCYLLADB_ENABLE_SWOOLE / _OPENSWOOLE compiles one C++ shim
  (src/Async/SwooleBridge.cc) so Future::get() suspends the current coroutine
  instead of blocking. Default build stays pure C; swoole symbols resolve lazily.

Shared reactor (opt-in, O(1) fds for high fan-out):
- Cassandra\Async\Reactor + src/Async/Reactor.{c,h}: one eventfd + mutex-guarded
  MPSC completion queue shared across many futures, so the loop watches ONE fd
  regardless of concurrency. fd+mutex are module/thread-lifetime; registration
  state resets each request. lib/Async/ReactorRevolt.php adapter.

Tests + benchmarks:
- tests/Feature/Async/*: primitive, Revolt/Amp/React/Swoole (guarded), reactor
  (io_threads=4 multi-producer, error propagation, model-mixing guards).
- benchmarks/Live/{AsyncBench,ReactorBench}.php + committed baselines under
  benchmarks/baselines/ (JSON + Markdown) via composer bench:baseline:export.
- docs/async.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…guard

- FutureNotifier.c: reorder notifier_ensure so the register-then-recheck poke
  happens BEFORE dropping the callback ref, not after. Functionally identical
  (refcount is 2 across the poke, never freed), but removes the path clang-tidy's
  analyzer flagged as use-after-free (clang-analyzer-unix.Malloc).
- SwooleBridge.cc: wrap the whole TU in #ifdef HAVE_SWOOLE_COROUTINE so tooling
  that analyzes the default build (clang-tidy) doesn't choke on the absent
  <swoole.h> — the file is only compiled when the swoole build flag is set.
- ReactorTest: "surfaces a failed query" now loops until its future is
  dispatched instead of asserting a single select+poll. The reactor fd is
  level-triggered and can report a spurious wake, so the single-shot form was
  flaky across tests (leaked a pending future into the next test on CI).
- test.yml (pre-existing ASan failure, unrelated to async): disable opcache in
  the ASan job — PHP dlopen()s opcache with RTLD_DEEPBIND, which the sanitizer
  runtime rejects and aborts on before any test runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…DEEPBIND exts under ASan

- FutureNotifier.c: on the set_callback-failed path drop the callback ref with a
  direct atomic decrement instead of unref(). Refcount is provably 2→1 there
  (creation ref always survives), so it can never free; the inline decrement
  stops clang-analyzer-unix.Malloc from modelling an impossible free of the
  pointer returned in *notifier_slot.
- Reactor.c: reactor_consume now drops the reg's two refs (ready-list +
  registered) in a single atomic sub-2 via reg_release() instead of two
  reg_unref() calls, so the analyzer no longer sees a deref-after-free between
  the two decrements. Verified clean with clang-analyzer-unix.Malloc locally.
- test.yml ASan job: also disable mysqli/pdo_mysql/mysqlnd (pre-existing; the
  8.4 runner dlopen()s mysqlnd with RTLD_DEEPBIND, same abort as opcache). The
  8.5 ASan job already runs the full suite (incl. reactor) clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts the opcache/mysql extension-disabling in the ASan job. On PHP 8.4 the
runner dlopen()s opcache/mysqlnd with RTLD_DEEPBIND (incompatible with the
sanitizer runtime → abort before tests) — a pre-existing failure that also fails
on trunk and is unrelated to this PR. Disabling those extensions to work around
it broke the 8.4/8.5 mysqli↔mysqlnd dependency at startup, so it's not a net
improvement. Left as-is for a dedicated CI-infra change; the 8.5 ASan job already
exercises the full suite (incl. the reactor) memory-safety clean.

The C-side clang-tidy fixes (FutureNotifier direct decrement, Reactor reg_release)
are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PHP 8.4's runner dlopen()s opcache/mysqlnd with RTLD_DEEPBIND, which the
sanitizer runtime rejects (aborts before any test runs) — a pre-existing failure
(also red on trunk). Disable those extensions for 8.4 only via a matrix-
conditional so the 8.4 ASan run can start; 8.5 is left untouched (it doesn't hit
the abort, and disabling mysqlnd there breaks the mysqli↔mysqlnd startup dep).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fra issue

Disabling extensions one-by-one on the 8.4 ASan job is whack-a-mole: the runner
dlopen()s opcache, mysqlnd, pdo, … with RTLD_DEEPBIND, which the sanitizer
runtime rejects. This is a fundamental 8.4-runner/ASan incompatibility that also
fails on trunk and is unrelated to this PR; it needs a dedicated CI change (e.g.
drop 8.4 from the ASan matrix, or build PHP without RTLD_DEEPBIND). Leaving
test.yml untouched. The 8.5 ASan job already validates this PR's memory safety.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add a `swoole` CI job (matrix: swoole@8.3/8.4, openswoole@8.3) that builds the
  extension with -DPHP_SCYLLADB_ENABLE_SWOOLE/_OPENSWOOLE against the runtime's
  own source headers (checked out at the installed version for ABI match), loads
  the coroutine runtime, and runs the swoole test group against live ScyllaDB.
- lib/Async/Swoole.php: make the adapter flavor-aware — detect Swoole\ vs
  OpenSwoole\ Coroutine/System at runtime and use the runtime's EVENT_READ
  constant instead of a hardcoded value.
- SwooleTest: flavor-aware coroutine runner; cover native coroutine-aware get(),
  the adapter, and many in-flight futures. Runs when swoole/openswoole is loaded.
- test.yml: add lib/** to the path filter so async userland changes trigger CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CI swoole jobs failed with `undefined symbol: php_scylladb_swoole_current_cid`.
Two causes:
- The #ifdef HAVE_SWOOLE_COROUTINE guard on SwooleBridge.cc compiled it to an
  empty object under GCC (the target define wasn't reaching the .cc), so the shim
  symbols were absent. The guard was only for clang-tidy; drop it and instead
  exclude the file from the clang-tidy file list (it needs the swoole source
  headers to parse).
- OpenSwoole renamed its headers (openswoole_*.h), C++ namespace (`openswoole`),
  and event flag (OSW_EVENT_READ). Select the runtime with __has_include(<openswoole.h>)
  (keyed on the include path, not a define) + a namespace alias, so one shim
  builds against either tree.

Also: don't re-load the coroutine ext via -d in the test run (setup-php already
loads it via ini — avoids a "module already loaded" warning).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant