feat(async): event-loop async support — fd primitive, adapters, native Swoole, shared reactor - #138
Draft
CodeLieutenant wants to merge 8 commits into
Draft
feat(async): event-loop async support — fd primitive, adapters, native Swoole, shared reactor#138CodeLieutenant wants to merge 8 commits into
CodeLieutenant wants to merge 8 commits into
Conversation
…e Swoole, shared reactor Implements #24: let the driver participate in non-blocking PHP event loops instead of only blocking on Future::get(). Core primitive (C, src/FutureNotifier.{c,h}): - Every Future gains getResource(): resource and isReady(): bool. getResource() returns a php_stream that becomes readable once, when the driver resolves the future on its IO thread (cass_future_set_callback + self-pipe). Any loop (stream_select / ReactPHP / AMPHP / Swoole) can await without blocking; get() stays blocking, so sync users pay nothing. - Refcounted, persistent-malloc notifier. Linux eventfd fast path (1 syscall, 1 fd, no SIGPIPE) with a POSIX pipe fallback on macOS; SIGPIPE-safe dup for the stream; register-then-recheck closes lost-wakeup races. Framework adapters (lib/Async, optional peer deps — require-dev + suggest): - Revolt (fiber await, default), Amp (toFuture), ReactPhp (toPromise), Swoole (coroutine waitEvent). Native Swoole/OpenSwoole (opt-in build flag): - -DPHP_SCYLLADB_ENABLE_SWOOLE / _OPENSWOOLE compiles one C++ shim (src/Async/SwooleBridge.cc) so Future::get() suspends the current coroutine instead of blocking. Default build stays pure C; swoole symbols resolve lazily. Shared reactor (opt-in, O(1) fds for high fan-out): - Cassandra\Async\Reactor + src/Async/Reactor.{c,h}: one eventfd + mutex-guarded MPSC completion queue shared across many futures, so the loop watches ONE fd regardless of concurrency. fd+mutex are module/thread-lifetime; registration state resets each request. lib/Async/ReactorRevolt.php adapter. Tests + benchmarks: - tests/Feature/Async/*: primitive, Revolt/Amp/React/Swoole (guarded), reactor (io_threads=4 multi-producer, error propagation, model-mixing guards). - benchmarks/Live/{AsyncBench,ReactorBench}.php + committed baselines under benchmarks/baselines/ (JSON + Markdown) via composer bench:baseline:export. - docs/async.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…guard - FutureNotifier.c: reorder notifier_ensure so the register-then-recheck poke happens BEFORE dropping the callback ref, not after. Functionally identical (refcount is 2 across the poke, never freed), but removes the path clang-tidy's analyzer flagged as use-after-free (clang-analyzer-unix.Malloc). - SwooleBridge.cc: wrap the whole TU in #ifdef HAVE_SWOOLE_COROUTINE so tooling that analyzes the default build (clang-tidy) doesn't choke on the absent <swoole.h> — the file is only compiled when the swoole build flag is set. - ReactorTest: "surfaces a failed query" now loops until its future is dispatched instead of asserting a single select+poll. The reactor fd is level-triggered and can report a spurious wake, so the single-shot form was flaky across tests (leaked a pending future into the next test on CI). - test.yml (pre-existing ASan failure, unrelated to async): disable opcache in the ASan job — PHP dlopen()s opcache with RTLD_DEEPBIND, which the sanitizer runtime rejects and aborts on before any test runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…DEEPBIND exts under ASan - FutureNotifier.c: on the set_callback-failed path drop the callback ref with a direct atomic decrement instead of unref(). Refcount is provably 2→1 there (creation ref always survives), so it can never free; the inline decrement stops clang-analyzer-unix.Malloc from modelling an impossible free of the pointer returned in *notifier_slot. - Reactor.c: reactor_consume now drops the reg's two refs (ready-list + registered) in a single atomic sub-2 via reg_release() instead of two reg_unref() calls, so the analyzer no longer sees a deref-after-free between the two decrements. Verified clean with clang-analyzer-unix.Malloc locally. - test.yml ASan job: also disable mysqli/pdo_mysql/mysqlnd (pre-existing; the 8.4 runner dlopen()s mysqlnd with RTLD_DEEPBIND, same abort as opcache). The 8.5 ASan job already runs the full suite (incl. reactor) clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts the opcache/mysql extension-disabling in the ASan job. On PHP 8.4 the runner dlopen()s opcache/mysqlnd with RTLD_DEEPBIND (incompatible with the sanitizer runtime → abort before tests) — a pre-existing failure that also fails on trunk and is unrelated to this PR. Disabling those extensions to work around it broke the 8.4/8.5 mysqli↔mysqlnd dependency at startup, so it's not a net improvement. Left as-is for a dedicated CI-infra change; the 8.5 ASan job already exercises the full suite (incl. the reactor) memory-safety clean. The C-side clang-tidy fixes (FutureNotifier direct decrement, Reactor reg_release) are retained. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PHP 8.4's runner dlopen()s opcache/mysqlnd with RTLD_DEEPBIND, which the sanitizer runtime rejects (aborts before any test runs) — a pre-existing failure (also red on trunk). Disable those extensions for 8.4 only via a matrix- conditional so the 8.4 ASan run can start; 8.5 is left untouched (it doesn't hit the abort, and disabling mysqlnd there breaks the mysqli↔mysqlnd startup dep). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fra issue Disabling extensions one-by-one on the 8.4 ASan job is whack-a-mole: the runner dlopen()s opcache, mysqlnd, pdo, … with RTLD_DEEPBIND, which the sanitizer runtime rejects. This is a fundamental 8.4-runner/ASan incompatibility that also fails on trunk and is unrelated to this PR; it needs a dedicated CI change (e.g. drop 8.4 from the ASan matrix, or build PHP without RTLD_DEEPBIND). Leaving test.yml untouched. The 8.5 ASan job already validates this PR's memory safety. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add a `swoole` CI job (matrix: swoole@8.3/8.4, openswoole@8.3) that builds the extension with -DPHP_SCYLLADB_ENABLE_SWOOLE/_OPENSWOOLE against the runtime's own source headers (checked out at the installed version for ABI match), loads the coroutine runtime, and runs the swoole test group against live ScyllaDB. - lib/Async/Swoole.php: make the adapter flavor-aware — detect Swoole\ vs OpenSwoole\ Coroutine/System at runtime and use the runtime's EVENT_READ constant instead of a hardcoded value. - SwooleTest: flavor-aware coroutine runner; cover native coroutine-aware get(), the adapter, and many in-flight futures. Runs when swoole/openswoole is loaded. - test.yml: add lib/** to the path filter so async userland changes trigger CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CI swoole jobs failed with `undefined symbol: php_scylladb_swoole_current_cid`. Two causes: - The #ifdef HAVE_SWOOLE_COROUTINE guard on SwooleBridge.cc compiled it to an empty object under GCC (the target define wasn't reaching the .cc), so the shim symbols were absent. The guard was only for clang-tidy; drop it and instead exclude the file from the clang-tidy file list (it needs the swoole source headers to parse). - OpenSwoole renamed its headers (openswoole_*.h), C++ namespace (`openswoole`), and event flag (OSW_EVENT_READ). Select the runtime with __has_include(<openswoole.h>) (keyed on the include path, not a define) + a namespace alias, so one shim builds against either tree. Also: don't re-load the coroutine ext via -d in the test run (setup-php already loads it via ini — avoids a "module already loaded" warning). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #24 — makes the driver usable inside non-blocking PHP event loops instead of only blocking on
Future::get().The C driver resolves futures on its own IO thread(s); this exposes that completion to userland so any loop can await a query without blocking.
get()stays blocking, so existing sync code is unaffected and pays nothing.What's in it
1. Core primitive (C) — works with any loop, zero deps
Every
Futuregains:getResource()returns aphp_streamthat becomes readable exactly once when the driver resolves the future (viacass_future_set_callback+ a self-pipe). Watch it withstream_select/ReactPHP/AMPHP/Swoole; on readable,get()returns without blocking. Linux useseventfd(1 syscall, 1 fd, no SIGPIPE); macOS falls back to a pipe.src/FutureNotifier.{c,h}.2. Framework adapters (
lib/Async/, optional peer deps —require-dev+suggest)Revolt(fiber await, the default),Amp(toFuture),ReactPhp(toPromise),Swoole(coroutinewaitEvent).3. Native Swoole/OpenSwoole — opt-in build flag
-DPHP_SCYLLADB_ENABLE_SWOOLE/_OPENSWOOLE(+-DPHP_SCYLLADB_SWOOLE_SRC=…) compiles one small C++ shim (src/Async/SwooleBridge.cc) soFuture::get()suspends the current coroutine instead of blocking. Default build stays pure C; swoole symbols resolve lazily at runtime.4. Shared reactor — opt-in, O(1) fds for high fan-out
Cassandra\Async\Reactor(::add/resource/poll/pending): one eventfd + a mutex-guarded MPSC completion queue shared across many futures, so the loop watches one fd regardless of concurrency. Per-future fds cap out atFD_SETSIZE(~512 on macOS); the reactor scales to thousands and is even a touch faster at matched fan-out.lib/Async/ReactorRevolt.phpadapter. Seedocs/async.md.Testing
tests/Feature/Async/*against live ScyllaDB: primitive (stream_selectmultiplexing, error propagation, early-free SIGPIPE regression), Revolt/Amp/React (concurrency), Swoole (skip-guarded), reactor (io_threads=4multi-producer, errors, model-mixing guards, Revolt adapter). All green; full suite 880 passed.benchmarks/Live/{AsyncBench,ReactorBench}.php+ committed baselines underbenchmarks/baselines/(JSON + Markdown, viacomposer bench:baseline:export).Notes for reviewers
io_threads=4stress + leak checks).*_arginfo.h/*_descriptor.care build-time artifacts (git-ignored), regenerated from the committed stubs.UuidTest › unique UUIDs across separate processes) is environmental — it spawnsphp -d extension=cassandrachildren by bare name and passes in CI where the extension is installed.🤖 Generated with Claude Code