Skip to content

Latest commit

 

History

History
1526 lines (1321 loc) · 90.3 KB

streaming.md

File metadata and controls

1526 lines (1321 loc) · 90.3 KB

satd Streaming Consumption API — Protocol Spec

A push-based consumption surface for downstream indexers, wallets, Lightning nodes, exchanges, watchtowers, and explorers: a real-time event firehose plus live, cursor-resumable watch subscriptions, served over gRPC, WebSocket/SSE, and a satd-native ZMQ socket. One schema — the protobuf definition in satd.events.v1 is the source of truth — mapped to JSON for the WebSocket transport. Bitcoin Core's own ZMQ topics (-zmqpub*) are a separate compatibility surface outside this spec (§3).

The surface is strictly publish/read-only and decoupled from consensus: it consumes the node's existing chain/mempool broadcasts and re-reads blocks the node already holds, never touching the block-connection or mempool-acceptance path (§10). All wire additions are additive — the schema stays v1.

1. Purpose & scope

Existing node-consumption APIs (Core JSON-RPC + ZMQ, Electrum, Esplora) each leave the same three gaps: descriptor lifecycle, outpoint-level subscriptions, and cursor-based event replay. Every serious consumer reinvents them on top. This API serves all three natively.

The key generalization: outpoint subscription is the base primitive. Channel-close detection, watchtower triggers, deposit confirmation, and theft monitoring all reduce to "tell me when this outpoint is spent." Address-watching is outpoint-watching with a derivation rule layered on top. We build down to outpoints and layer scripts, descriptors, and transaction-id watches on as conveniences over the same matcher.

Out of scope: mining operations (getblocktemplate / submitblock — Stratum is the venue), wallet key management and signing (the node stays keyless), and any consensus or block-production knobs.

First-party clients. This document specifies the wire contract; it is implementable directly from the .proto. Two SDKs implement it in full and are maintained in this repository — satd-events-client (Rust) and satdevents (Go, at clients/go/). They are held at parity by a differential harness that drives both against one node through an identical watch spec and diffs their rendered events, so a behaviour described below as "the SDK does X" is true of both. Where this spec names one SDK's method, the other's counterpart carries the same name in its own idiom.

2. Relationship to existing surfaces

This is a consolidation of existing substrate, not a greenfield build. satd already provides:

Substrate Crate / module Role here
NodeEvent envelope (schema v1, edge-stamped, monotonic seq) node::events The wire event type, extended additively below
Broadcast firehose (broadcast::Sender<NodeEvent>, cap 4096) node::events::publisher Live event source
gRPC NodeEventStream (satd.events.v1) events::grpc The native transport (§3, §7.3)
satd-native ZMQ PUB (-eventszmqbind) events::zmq Envelope sink, carried unchanged
Bitcoin Core ZMQ (-zmqpub*) events::core_zmq, events::zmtp A compatibility projection of the firehose, outside this spec (§3)
Electrum per-scripthash registry, status-hash electrum-proto Pattern reused; not the new surface
Esplora REST + SSE (PermitStream / WatchLease RAII) esplora-handlers SSE firehose pattern reused (§3)
SpendIndex (outpoint → spending input, persistent) node-index::spend_* The query side; the live notifier (§7) is the other half
Unified auth (stream:subscribe / stream:watch + per-token quota) satd-auth Reused wholesale; no new auth surface (§9)

The delta this spec defines is the live notifier: durable replay cursors, an outpoint/script/txid matcher with a bidirectional control channel, the JSON/ WebSocket transport, transaction-lifecycle and confirmation-depth watches, and a descriptor convenience layer.

3. Transports

One schema, three transports:

  1. gRPC native (satd.events.v1, tonic) — the primary transport for programmatic consumers. A server-streaming Subscribe (firehose) and a bidirectional Watch (firehose + managed watch-set, §7.3).
  2. JSON-over-WebSocket (GET /ws) — a hand-mapped JSON rendering of the same proto tagged-unions, with a client→server control channel mirroring Watch.
  3. Server-Sent Events (GET /sse) — a read-only JSON firehose (no control channel) for browser / curl consumers, reusing the Esplora SSE pattern.

The JSON transports deliberately do not use grpc-gateway/REST transcoding: it would drag a Go toolchain into a build that already contends with bindgen / libclang / musl-static. Hand-mapping a stable, narrow oneof surface is cheaper than owning that toolchain.

A satd-native ZMQ PUB sink (-eventszmqbind) carries the firehose bodies only — not the per-subscriber watch matches — and uses per-topic sequence numbers; gaps are detected the Core way (sequence jumps), so it does not carry the in-band lag signal (§6.4).

Bitcoin Core's -zmqpub* topics (hashblock, hashtx, rawblock, rawtx, sequence) are also published, byte for byte as Core sends them, for software written against Core. They are a compatibility projection of the same firehose and not part of this protocol: no envelope, no cursor, no watch-set, and no order promised between block- and mempool-derived messages. The Operator Manual's streaming chapter documents them, and CORE_DIFFERENCES.md lists how they differ from Core's.

3.1 Listener placement

WS/SSE bind a dedicated --streamws port, not an upgrade on the Core-compat JSON-RPC port. This keeps the differentiated stream a distinct service on a distinct port — avoiding the compatibility trap where integrators reach for the Core-shaped surfaces and never discover the stream — and isolates its blast radius from the RPC listener.

Every streaming listener — --streamws and the gRPC NodeEventStream — runs on the isolated API tokio runtime (--api-threads), never the core block-connecting runtime. This is a hard placement rule that composes with the API-runtime split: a flood of streaming clients cannot contend with the threads that connect blocks and accept mempool transactions.

4. Schema versioning

NodeEvent.schema_version is 1. Per the node::events::schema evolution policy, adding oneof variants and fields is not a major bump — every addition here is additive, so schema_version stays 1. A rename or removal would force a major bump; those are avoided pre-freeze.

Unknown oneof arms and fields MUST be ignored by older readers (forward-compat for rolling upgrades), exactly as the existing categories bitfield already tolerates unknown bits.

Every Subscribe and Watch response carries two gRPC response headers, sent as soon as the stream is set up (before any event, so a quiet Watch still delivers them):

Header Value
satd-version The node's version string, e.g. 0.6.0 or 0.6.0-pre.
satd-events-schema schema_version as a decimal string (1).

Nodes before 0.6.0 send neither header. The first-party SDKs compare the node's major/minor version against their own and warn when the node is one minor version behind, or refuse when it is two or more behind. A missing header counts as 0.5. A schema mismatch is always refused. The full rule, including the obligation that new request fields degrade to a superset when an older node ignores them, is in STABILITY_POLICY.md → Streaming API & SDK compatibility. The WebSocket/SSE carrier does not send these headers; its JSON envelope carries schema_version on every event.

5. Event envelope

message NodeEvent {
  uint32    schema_version = 1;
  EdgeStamp stamp          = 2;
  Cursor    cursor         = 3;   // set on confirmed-side bodies (§6)
  oneof body {
    MempoolEvent  mempool        = 10;  // accept / confirm / evict / replace
    ChainEvent    chain          = 11;  // connect / disconnect / reorg marker (§8)
    Heartbeat     heartbeat      = 12;
    OutpointSpent outpoint_spent = 13;  // watch match (§7.2)
    ScriptMatched script_matched = 14;  // watch match (§7.2)
    DescriptorNeedsAddresses descriptor_needs_addresses = 15;  // reserved, never emitted (§11)
    Lagged        lagged         = 16;  // in-band lag notice (§6.4)
    TxidMatched   txid_matched   = 17;  // txid lifecycle: seen / confirmed (§7.4)
    TxidReplaced  txid_replaced  = 18;  // txid lifecycle: RBF-replaced (§7.4)
    TxidEvicted   txid_evicted   = 19;  // txid lifecycle: policy eviction (§7.4)
    TxidUnconfirmed txid_unconfirmed = 20;  // txid lifecycle: reorg rollback (§7.4)
    TxidDepthReached txid_depth_reached = 21;  // depth alarm, single-shot (§7.4)
    TxidFinalized txid_finalized = 22;  // lifecycle auto-close, terminal (§7.4)
    PrefixMatched prefix_matched = 23;  // privacy-preserving prefix watch (§7.5)
    SetCursorResult set_cursor_result = 24;  // deterministic re-anchor ack/reject (§7.3.1)
    WatchSetResult set_watch_set_result = 25;  // deterministic atomic-replace ack/reject (§11.2)
    RescanResult   rescan_result       = 26;  // deterministic historical-rescan ack/reject (§7.6)
    RescanComplete rescan_complete     = 27;  // terminal marker: rescan range drained (§7.6)
    BlockTweaks    block_tweaks        = 28;  // BIP 352 per-block tweak data (§7.7)
    SilentPaymentMatched silent_payment_matched = 29;  // BIP 352 scan-key watch match (§7.7)
    MempoolTweak   mempool_tweak       = 30;  // BIP 352 mempool-time tweak, Tier 1.5 (§7.7)
    StatusEvent    status              = 31;  // node-health condition (§7.8)
    WatchAddRejected watch_add_rejected = 32;  // an incremental watch add the node did not register (§7.3.2)
  }
}

EdgeStamp carries node_id, region, edge_seen_at_ns, edge_wall_ns, and a per-publisher seq. seq resets on process restart: it is the mempool-side replay watermark only, never a durable confirmed-side cursor. Restart is detected through Cursor.instance_id (§6.1).

Reorgs are not a separate top-level body: they are carried on ChainEvent (chain = 11) as a first-class Reorg marker followed by the per-block disconnect/connect sequence (§8). Watch matches (13/14, 17–22) flow on the per-subscriber Watch channel, not the firehose, and are never bridged to ZMQ.

6. Cursors & replay

Reconnect-with-cursor is the single highest-value primitive: it is the one replay mechanism for every subscription type, subsuming Electrum's subscribe-then- get-history dance and Esplora's per-address pagination. A consumer chooses satd over Core RPC + ZMQ for exactly this.

6.1 Cursor type

message Cursor {
  uint32 height      = 1;  // confirmed: block height of the last delivered item
  uint32 tx_index    = 2;  // confirmed: index within that block (reserved; see below)
  uint64 mempool_seq = 3;  // best-effort mempool high-water (advisory; resets on restart)
  uint64 instance_id = 4;  // per-process restart epoch; JSON-encoded as a decimal string
}

A client persists the cursor from the last NodeEvent it durably processed and presents it on reconnect to resume.

Granularity. (height, tx_index) is designed for per-transaction resume so a client could resume mid-block after a disconnect. For chain events resume is block-granular: BlockConnected uses tx_index = 0, the only confirmed chain event per block. The field is a deliberate forward-compatible reservation for per-tx confirmed events — not an oversight.

One position is already assigned: a block's BlockTweaks aggregate carries tx_index = 0xFFFFFFFF (u32::MAX). It is emitted after that block's BlockConnected, so the maximal tx_index keeps it strictly ordered after the chain event at the same height. This matters for a mixed (chain | tweaks) subscription: a client that persisted a chain cursor (h, 0) and reconnects still gets block h's pending tweaks re-emitted (the server detects the cursor sits before 0xFFFFFFFF at that height), instead of the replay skipping to h + 1 and silently dropping the tweak. A tweaks-only subscriber only ever sees (h, 0xFFFFFFFF) cursors and resumes cleanly.

Restart epoch. instance_id is a per-process random u64 minted once at startup — deliberately not the persisted-stable node_id, which cannot distinguish a same-node restart. When a resume cursor's instance_id differs from the live publisher's, the daemon has restarted and the mempool seq space has reset; the server resets the mempool watermark (replaying the full ring) while confirmed/height replay stays durable. On the JSON transports it is serialized as a decimal string so a 64-bit value survives a JavaScript Number (53-bit mantissa); the structured fields (height, tx_index) are small and stay numeric.

6.2 Replay semantics

On resume the server:

  1. Confirmed replay — resolves the active-chain hash at each height in (cursor.height, tip] and re-reads each block, emitting the confirmed events. The block store is the durable event source; no extra log or index is needed. Crucially, the hash at each height is found by walking prev_blockhash back from the tip (active_chain_range, O(span)), not by the best-known-at-height index (get_block_hash_by_height). That index is last-writer-wins and a header-first or side-chain store_block can clobber it — using it would risk replaying a side-chain block as confirmed. The span is capped (MAX_REPLAY_BLOCKS, 10 000) on every carrier; a client behind the cap backfills the older tail with a second from_cursor request.
  2. Snapshot → live handoff — before replay the server captures the current tip and seq watermark, replays confirmed history up to it, then drains the live broadcast from the captured point. The live receiver is subscribed before the snapshot, so nothing in the gap is lost.
  3. Mempool replay — best-effort only. The mempool is not durable; the server replays from its bounded in-memory ring for entries with seq > mempool_seq, then joins live. Clients treat mempool replay as lossy by contract.

6.3 Reorg at the replay boundary

The handoff de-duplicates confirmed events by (height, hash) identity and mempool events by seq high-water. Keying confirmed dedup on the hash, not the height alone, is what makes a reorg at the seam correct: a replacement block has a different hash at the same height, so it is forwarded rather than suppressed as a duplicate. A client that holds the prior tip hash (and/or receives a Reorg, §8) detects a stale cursor and rolls its own state back; the server emits the Reorg and replays forward from the new common ancestor. Clients MUST be prepared to receive BlockDisconnected / Reorg during replay.

6.4 In-band lag signal

A subscriber that falls behind the bounded broadcast buffer would otherwise silently lose events. Instead, every carrier (gRPC Subscribe + Watch, WS, SSE) emits an in-band Lagged:

message Lagged { uint64 dropped_count = 1; Cursor resume_cursor = 2; }

The notice bypasses the category filter (its category_bit is u32::MAX), so a SetCategories mask cannot accidentally suppress the one message a lagging client most needs. resume_cursor is the last-delivered position (seeded from the replay tail, or the current chain tip on the replay-less Watch firehose), so the client re-issues Subscribe(from_cursor) to backfill the gap.

This preserves the core safety property — the server stays drop-on-lag and never backpressures the publisher (§10) — while removing its sharp edge: the client is told, in-band, exactly where to resume. The cost is that recovery is the client's responsibility, which is the right place for it (only the client knows its own durability needs). ZMQ, native or Core's, does not carry Lagged: a lag shows as a sequence-number gap, as on Core.

7. Watch-set subscriptions

satd has SpendIndex (a query index) and the Electrum scripthash registry (a push for scripthashes), but no live outpoint subscription. This section adds an outpoint-, script-, and txid-keyed matcher plus a bidirectional control channel to manage a watch-set on a live connection.

7.1 The matcher

A WatchRegistry holds per-subscriber watch-sets behind O(1) inverted indexes (by_outpoint, by_scripthash, by_txid). It is decoupled from the consensus path, not inline in it: a dedicated run_watch_matcher task subscribes to the existing chain/mempool broadcasts and re-reads each connected block (ChainState::get_block) and each accepted tx (Mempool::get) once the node already holds them, then scans the watch-set and delivers matches.

This decoupling is the central design tradeoff. The alternative — matching inside connect_block / accept_tx — would be marginally cheaper but would add code to the consensus hot path. Re-reading off the hot path costs one extra block read per connected block (and gates it behind a lock-free has_watchers() check, so a node with no subscribers pays nothing), in exchange for zero edits to the accept path and a hard guarantee that no consumer can ever block consensus. The one cost it introduces is that a matcher whose broadcast receiver lags would skip blocks; the matcher closes that hole by rescanning the missed window on Lagged (reading blocks the node already holds), bounded by streammaxresyncblocks (default 10 000) so catch-up can never monopolize the API runtime. Delivery to each subscriber is non-blocking try_send: a slow client's channel fills and matches are dropped-with-notice, never stalling the matcher.

7.2 Match events

message OutpointSpent {
  bytes  outpoint_txid = 1;
  uint32 outpoint_vout = 2;
  bytes  spending_txid = 3;
  uint32 spending_vin  = 4;
  bool   confirmed     = 5;   // false = observed in mempool, not yet in a block
}

message ScriptMatched {
  bytes  scripthash = 1;
  bytes  txid       = 2;
  bool   is_output  = 3;      // true = funding (an output pays the script); false = spending
  uint32 index      = 4;      // vout if is_output else vin
  bool   confirmed  = 5;
  repeated DescriptorMatch descriptor_matches = 6;  // descriptor attribution (§11.1); empty for a direct watch
  uint64 amount     = 7;      // matched value; funded output or spent-prevout, per streamprevoutmeta tier
  bool   has_amount = 8;      // distinguishes a genuine 0-value match from "not retained at this tier"
  bytes  raw_tx     = 9;      // full matching tx, inline; only when this stream opted in (SetWatchOptions)
}

// Per-stream delivery options (SubscribeControl.set_watch_options). Off by
// default — raw_tx inlining is bandwidth-heavy.
message SetWatchOptions { bool include_raw_tx = 1; }

message DescriptorMatch {
  string descriptor       = 1;   // the descriptor string the client registered
  uint32 branch           = 2;   // 0-based BIP-389 multipath branch (external=0, change=1; 0 if single-path)
  uint32 derivation_index = 3;   // absolute derivation index of the matched script
}

// AddScripts carries an optional per-scripthash min_value floor (§7.2.1).
message AddScripts { repeated bytes scripthashes = 1; repeated uint64 min_values = 2; }

OutpointSpent covers spend detection in both the mempool and connected blocks. ScriptMatched covers both sides of a script: funding (is_output = true) and spending (is_output = false). Input-side script matching is the subtle case — the spending transaction does not carry the prevout's scriptPubKey — so for a connected block it is recovered from the block's undo data (Store::get_undo → spent_coins[i].script_pubkey, the i-th non-coinbase input's prevout). This is a single cached point-get per block, paid only when a script is actually watched. The mempool input side is its unconfirmed twin: the spending tx still carries no prevout scriptPubKey, so it is recovered from the prevout scripthashes the mempool entry retains at admission (where the prevouts were already resolved for validation), yielding ScriptMatched{is_output = false, confirmed = false} without re-resolving anything off the hot path. Watching the funding outpoint also surfaces the spend (via OutpointSpent); the script path means a script watcher need not separately enumerate its outpoints to see unconfirmed spends.

ScriptMatched.amount/has_amount carry the matched value in-band at wire parity with SpentPrevout (§7.6.1) — always present on the funding side and for confirmed spends, present for mempool spends under streamprevoutmeta >= amount (the default) — so an exact-script consumer can skip the per-match getrawtransaction enrichment call and the cross-node "not found" mempool race it can hit. raw_tx goes further: the full consensus-serialized matching transaction, inlined only for a connection that has sent SetWatchOptions{include_raw_tx = true} (off by default; the matcher only pays the serialization cost when at least one connection has opted in, and the bytes are cached and shared across matches on the same transaction).

7.2.1 Per-script min_value floor

AddScripts accepts an optional min_values list, parallel to scripthashes: a per-script floor in satoshis below which matches are suppressed server-side, so a watcher that only cares about economically significant activity is not woken by dust. An empty min_values means no floors; a non-empty list MUST have the same length as scripthashes (a mismatch rejects the whole add — never silently unfiltered). Re-asserting a watched scripthash updates its floor in place (it is not a new quota item).

The floor is symmetric with the two sides of ScriptMatched, gating on the value of the coin the script controls in that match:

  • Funding (is_output = true): the matched output's value.
  • Spending (is_output = false): the spent prevout's value — the input carries no value, so it is recovered the same way the prevout scriptPubKey is. For a confirmed block that is the undo data (spent_coins[i].value, free alongside the script); for the mempool it is the prevout value retained at admission (streamprevoutmeta ≥ amount, §12.1) — the default.

On a node configured streamprevoutmeta = hash the mempool spend side has no value to test, so a floored script's unconfirmed-spend matches fail closed (suppressed) rather than being delivered unfiltered — a min_value watcher is never handed a possibly-dust unconfirmed spend it asked to be spared. The floor is still enforced normally on that script's funding and confirmed-block-input sides; only the mempool spend preview is withheld until the operator retains amounts. (Non-floored watches on the same node are unaffected.)

The value itself never enters the wire event — the floor is purely a delivery filter. The floor on the output and confirmed-input sides costs nothing (both values are already in hand); only the mempool-input side depends on the retention tier.

7.3 Bidirectional control channel

The gRPC service offers the unchanged server-streaming Subscribe (the firehose) plus a bidirectional Watch whose client→server messages are a tagged union:

service NodeEventStream {
  rpc Subscribe(SubscribeRequest)        returns (stream NodeEvent);  // firehose; from_cursor replay
  rpc Watch(stream SubscribeControl)     returns (stream NodeEvent);  // firehose + managed watch-set
}

message SubscribeControl {
  oneof msg {
    SetCursor          set_cursor          = 1;  // resume anchor (see below)
    SetCategories      set_categories      = 2;  // firehose category filter (mempool/chain/heartbeat)
    AddScripts         add_scripts         = 3;
    RemoveScripts      remove_scripts      = 4;
    AddOutpoints       add_outpoints       = 5;
    RemoveOutpoints    remove_outpoints    = 6;
    AddDescriptor      add_descriptor      = 7;  // §11
    AddTransactions      add_transactions      = 8;  // txid lifecycle + depth (§7.4)
    RemoveTransactions   remove_transactions   = 9;
    AddScriptPrefixes    add_script_prefixes   = 10; // privacy-preserving prefix watch (§7.5)
    RemoveScriptPrefixes remove_script_prefixes = 11;
    RemoveDescriptor     remove_descriptor     = 12; // drop a descriptor window (§11)
    SetWatchSet          set_watch_set         = 13; // atomic whole-set replace (see below)
    RescanBlocks         rescan_blocks         = 14; // bounded historical rescan (§7.6)
    SetWatchOptions      set_watch_options     = 15; // per-stream delivery opt-ins, e.g. raw_tx (§7.2)
  }
}

A tagged union — not a BIP37-style bloom filter — is what lets new watch kinds slot in without protocol breakage, the design choice that avoids btcd's BIP37 dead-end. The WS control channel is the JSON mirror of this union.

Resume semantics. Durable replay (§6.2) is offered at stream establishment: Subscribe(from_cursor) on gRPC, and ?from_cursor= at WS/SSE connect. A client that sends only SetCategories and no watch-add gets exactly the legacy firehose behavior.

Mid-stream SetCursor on the bidirectional gRPC Watch re-anchors the firehose (§7.3.1); WS/SSE clients, which have no mid-stream control channel, re-anchor by reconnecting with ?from_cursor=.

7.3.1 Mid-stream re-anchor

SetCursor(cursor) on a live Watch replays the confirmed history the client asks to revisit, then resumes the live tail — without tearing down the watch-set. It is an ordered drain-replay-resume, not an interleave:

  1. The server runs the same build_cursor_replay snapshot→live handoff used at establishment (§6.2) for (cursor.height, current_tip].
  2. The replayed confirmed events are emitted in height order, ahead of any further live events — the re-anchor batch is drained to completion before the live tail resumes. Live events that accumulate during the drain follow and may duplicate the tail of the replay; the client de-duplicates by (height, hash) exactly as at the establishment seam (§6.3), so the stream is at-least-once with idempotent confirmed application.
  3. The watch-set and its quota leases are preserved across the re-anchor — the reason to re-anchor in place rather than reconnect with from_cursor, which would force a full re-add (+ re-auth + quota rebuild) of a possibly-large watch-set. A long-lived Watch with thousands of registered scripts/outpoints re-anchors its replay position without rebuilding its watch-set.

Not replayed: historical watch matches. Watch matches are per-subscriber and not durable (they are never part of cursor replay, nor bridged to ZMQ). A re-anchor replays the firehose/confirmed history only; the client reconstructs any historical matches it needs from the replayed blocks against its own watch-set, and receives live matches going forward. Replay work is bounded two ways: concurrent re-anchors are capped at one in flight per stream, and each actionable re-anchor is charged against the connection's per-principal rate limit — the same token bucket as watch-set adds — so back-to-back SetCursors in a tight loop are throttled rather than allowed to re-walk the block index unboundedly. (Operator/loopback principals bypass the rate limit, as elsewhere.)

Deterministic outcome (SetCursorResult)

Every actionable SetCursor produces exactly one in-band SetCursorResult NodeEvent on the stream, so a client can distinguish "accepted, replaying from X" from "ignored, still live from the old position" without inferring it from the event flow:

message SetCursorResult {
  oneof outcome {
    CursorAccepted accepted = 1;
    CursorRejected rejected = 2;
  }
}
message CursorAccepted { Cursor from = 1; bool clamped = 2; uint32 earliest_replayed = 3; }
message CursorRejected {
  enum Reason { REASON_UNSPECIFIED = 0; RATE_LIMITED = 1; CONCURRENT_REANCHOR = 2; EMPTY_CURSOR = 3; NO_SOURCE = 4; }
  Reason reason = 1; Cursor current_head = 2;
}
  • CursorAccepted is emitted ahead of the replay batch: replay is now running. When the requested cursor predates the replay window (MAX_REPLAY_BLOCKS = 10 000 confirmed blocks) the lower end is clamped — clamped = true and earliest_replayed is the first height that will actually be replayed; the client must full-resync the skipped range below it from another source (e.g. RPC getblock).
  • CursorRejected means the re-anchor did not run and the live stream is unchanged: RATE_LIMITED (per-principal limit), CONCURRENT_REANCHOR (a drain already in flight), EMPTY_CURSOR (no cursor in the request), or NO_SOURCE (server has no block_source). current_head is the server's present resume position, so the client can retry, back off, or escalate to a full resnapshot.

A consumer with an at-least-once delivery contract should drive its catch-up state machine off these results rather than treating the SetCursor send as success. WS/SSE clients have no mid-stream control channel and so never see a SetCursorResult — they re-anchor by reconnecting with ?from_cursor=.

7.3.2 Refused watch adds (WatchAddRejected)

An incremental Add* that the server does not register is answered in-band with one WatchAddRejected event, and the stream stays up. An add that registers gets no event, so a client that sees none can rely on its adds having landed. The add is all-or-nothing over its net-new items: none of the items the event names is watched. Items the message re-asserted (already watched) are not named and stay watched; their metadata (a script's min_value floor, a silent-payment target's labels) still updates, and a message that only re-asserts is never refused.

message WatchAddRejected {
  enum Kind   { KIND_UNSPECIFIED = 0; SCRIPTS = 1; OUTPOINTS = 2; TRANSACTIONS = 3;
                DEPTH_ALARMS = 4; DESCRIPTOR = 5; SCRIPT_PREFIXES = 6; SILENT_PAYMENTS = 7; }
  enum Reason { REASON_UNSPECIFIED = 0; QUOTA_EXCEEDED = 1; RATE_LIMITED = 2;
                CAP_EXCEEDED = 3; PERMISSION_DENIED = 4; MALFORMED = 5; }
  Kind   kind = 1;  Reason reason = 2;
  uint64 required = 3;  uint64 held = 4;  uint64 quota = 5;  uint32 retry_after_secs = 6;
  // The refused items, as the client named them; only the field for `kind` is set.
  repeated bytes scripthashes = 7;  repeated Outpoint outpoints = 8;  repeated bytes txids = 9;
  repeated WatchDepthAlarm depth_alarms = 10;
  string descriptor = 11;  uint32 gap_limit = 12;  uint32 start = 13;  bool descriptor_kept = 14;
  repeated ScriptPrefix prefixes = 15;  // masked to `bits`
  repeated bytes scan_pubkeys = 16;     // b_scan·G; the scan secret is never echoed
}
reason Cause Numbers
QUOTA_EXCEEDED the net-new items' units do not fit the token's watch quota (§9) required units, held units already held, quota ceiling
RATE_LIMITED the per-principal add rate limit is spent retry_after_secs
CAP_EXCEEDED a per-connection cap: 16 silent-payment targets, 256 descriptors, or the WS entry cap (streamwsmaxsubscriptions) required = the count the add would reach, quota = the cap
PERMISSION_DENIED the token lacks stream:watch none
MALFORMED the message cannot be applied as a whole: min_values not parallel to its scripthashes, an invalid descriptor, or a txid × depth product over the per-message cap none

MALFORMED echoes the items that parsed, except for an over-cap depth product, which echoes none. Individually unparseable items inside an otherwise valid add (a scripthash that is not 32 bytes, a prefix outside the allowed bit range, an invalid silent-payment key) are skipped without a rejection.

A refused AddDescriptor names the descriptor and the window it asked for. descriptor_kept is true when an earlier window of that descriptor stays watched (a refused slide) and false when the descriptor is not watched at all.

On WS the event is the JSON object {"category": "watch_add_rejected", "kind": "scripts", "reason": "quota_exceeded", "required", "held", "quota", "retry_after_secs", ...} with the items under the same field names. Items come back the way a WS add names them: scripthashes and scan pubkeys as plain hex, txids in display (reversed) hex.

7.4 Transaction lifecycle & confirmation-depth watches

AddTransactions registers, over the same by_txid index, two decoupled primitives selected by min_depths:

message AddTransactions    { repeated bytes txids = 1; repeated uint32 min_depths = 2; uint32 auto_close_depth = 3; }
message RemoveTransactions { repeated bytes txids = 1; repeated uint32 min_depths = 2; }

message TxidMatched     { bytes txid = 1; bool confirmed = 2; uint32 height = 3; }
message TxidReplaced    { bytes txid = 1; bytes replacing_txid = 2; }        // RBF
message TxidEvicted     { bytes txid = 1; string reason = 2; }               // full_pool | expiry | block_conflict | policy | reorg
message TxidUnconfirmed { bytes txid = 1; uint32 prev_height = 2; }          // reorg rollback
message TxidDepthReached{ bytes txid = 1; uint32 depth = 2; uint32 height = 3; }  // single-shot alarm
message TxidFinalized   { bytes txid = 1; uint32 depth = 2; uint32 height = 3; }  // lifecycle auto-close
  • Lifecycle watch (min_depths empty) — one persistent watch per txid that narrates the transaction's full lifecycle: seen (mempool) → confirmed (block) via TxidMatched, then TxidReplaced (RBF, carrying the replacing txid), TxidEvicted (mempool policy), and TxidUnconfirmed (a reorg rolled back its confirming block). An optional auto_close_depth ≥ 1 makes the watch emit a terminal TxidFinalized and self-evict once the tx is that many confirmations deep.
  • Depth alarm (min_depths non-empty) — single-shot triggers keyed (txid, depth): TxidDepthReached fires the moment the tx reaches depth confirmations, then self-evicts. min_depths is a list, so one message arms several thresholds (e.g. [1, 6]); each (txid, depth) pair is an independent watch item.

The two are orthogonal: a depth alarm needs no lifecycle watch, and a lifecycle watch implies no alarm. auto_close_depth exists because no lifecycle state is truly terminal — a reorg un-confirms a confirmed tx, a re-broadcast revives an evicted one, a replacing tx can itself be reorged out — so a watch cannot safely self-clean on "confirmed" or "evicted." The only sound server-side self-clean is a depth the client nominates as its finality horizon, past which it accepts that reorgs are no longer interesting; below that depth, un-confirmations still fire.

Reorg safety. Depth tracking anchors on the confirming (height, hash) and re-checks it against the active chain on every block (the same tip-walk as replay, §6.2, never the pollutable height index). A confirming block reorged off the active chain reverts the entry, which re-arms if the tx reappears. A tx already buried when the watch is registered is resolved best-effort via the txindex (and otherwise arms on its next observed confirmation). TxidUnconfirmed is best-effort: it requires re-reading the disconnected block, so it is skipped if that block has been pruned — the depth anchor's revert remains reliable regardless.

Single-shot eviction is registry-authoritative: a fired alarm or auto-close is torn down server-side immediately, so it cannot re-fire even if its txid reappears in a later block, and its accounting is freed regardless of whether the terminal match reached a lagging client. The carrier then releases the quota lease on forwarding the match. The residual tradeoff: a terminal match dropped to a full channel holds that one quota unit until the connection closes — bounded by the connection lifetime and the per-token quota.

Because AddTransactions carries two repeated lists, the server bounds the txids × min_depths cross-product before allocating it: a malformed or oversized control message is rejected, never amplified into a large allocation.

7.5 Privacy-preserving prefix subscriptions

The §7.2–7.4 watches are precise: the server learns the exact scripts, outpoints, and txids a client cares about. The firehose (§5) is the privacy ceiling — the server learns nothing — but a client that wants only its own activity pays full chain+mempool bandwidth for it. AddScriptPrefixes is the tunable middle. A client registers a k-bit prefix of sha256(scriptPubKey) instead of a full 32-byte scripthash, and the server delivers every transaction whose script falls in that 2⁻ᵏ bucket; the client filters the bucket against its real scripts locally. The server's knowledge is bounded at "this client watches bucket p" — it never learns which script within the bucket is the real target, because it delivers the whole bucket and the discrimination happens client-side.

This is the push dual of BIP 158 (which satd already computes and serves as a block-filter index): the same idea — coarse, deterministic membership the client tests itself — pushed rather than pulled. The push form is more naturally private here, for two reasons. First, there is no fetch step: a BIP 158 client that matches a filter must fetch the full block, and that fetch leaks which block it cared about (mitigated only by fetching blocks from a different server); a prefix subscriber receives the matching transactions inline, so there is no second request to correlate. Second, it covers the mempool, which block filters structurally cannot.

Membership is BIP 158-parity — output scripts and spent-prevout scripts — so a single bucket catches both funding and spending of a script, exactly as §7.2's ScriptMatched does for exact scripts. It reuses the same resolution machinery: the output side is the scripthash scan_tx already computes; the confirmed input side is the prevout script scan_block_spent_scripts already recovers from a block's undo data; the mempool input side is resolved from the prevout scripthashes the mempool entry retains at admission (the prevouts are already resolved there for validation). That retained-hash mempool-spend path is shared with the exact-script watch (§7.2) — the prefix watch is what motivated it, because the alternative unconfirmed-spend signal, an outpoint watch, is a fallback a privacy client cannot use without naming the very outpoints it is hiding. Each resolved scripthash is truncated to the registered prefix lengths and looked up in a parallel by_prefix index (keyed (bits, prefix), gated by a lock-free has_prefix_watchers() like every other watch kind). The per-output cost is O(distinct prefix lengths in use) — a BTreeSet of active k values, typically one or two — independent of subscriber count.

message ScriptPrefix         { bytes prefix = 1; uint32 bits = 2; }  // k bits, left-aligned in ceil(k/8) bytes
message AddScriptPrefixes    { repeated ScriptPrefix prefixes = 1; }  // control field 10
message RemoveScriptPrefixes { repeated ScriptPrefix prefixes = 1; }  // control field 11

message PrefixMatched {                       // body field 23
  ScriptPrefix prefix           = 1;
  bytes        raw_tx           = 2;          // the full matching tx, inline — no precise follow-up fetch
  bool         confirmed        = 3;
  uint32       height           = 4;
  repeated SpentPrevout matched_prevouts = 5; // spend side (see SpentPrevout); empty on a pure funding match
}
// script_pubkey/amount carriage depends on the retention tier (§12.1): confirmed
// spends always carry both (from undo); mempool spends carry the script only
// under `full` and the value only under `>= amount`. `has_amount` disambiguates a
// genuine 0-sat prevout from "value not retained".
message SpentPrevout {
  bytes  outpoint_txid = 1; uint32 outpoint_vout = 2;
  bytes  script_pubkey = 3;  // empty when not retained
  uint64 amount = 4; bool has_amount = 5;
}

On the WS/SSE JSON surface each matched_prevouts entry mirrors this exactly: script_pubkey is a (possibly empty) hex string, amount is a number or null when the value was not retained, and has_amount is the explicit boolean — so a JSON client need not infer retention from the null-vs-0 encoding.

Delivery is self-contained, deliberately. PrefixMatched carries the full raw_tx, not a txid — a txid would force the client to fetch the transaction precisely, re-leaking the exact interest the bucket was hiding. For a confirmed spend it also carries the matched prevout scripts (and values) in matched_prevouts (recovered from undo data), so the client can confirm "this is a spend of one of my outputs" without resolving any outpoint itself. For a mempool spend, how much it can confirm from the event alone depends on the operator's streamprevoutmeta (§12.1): under full the real prevout scriptPubKey is carried (the chainstate-less / privacy client's case — confirm locally, no outpoint resolution, no re-leak); under the default amount the value is carried but the script is empty; under hash neither is, and the client confirms the match against its own UTXO set. Because a bucket is a 2⁻ᵏ slice of uniform scripthash space, inline full-tx delivery is cheap: even a coarse k=8 bucket (anonymity set ≈ 256×) is a low-single-digit transactions per block plus a trickle from the mempool.

Granularity is operator-bounded by streamprefixminbits / streamprefixmaxbits. The maximum (most precise) bound matters most: without it a client could register a near-full-length prefix and rebuild the leaking exact watch with extra steps, so bits is capped well short of a single script. The minimum bounds the bandwidth and quota a single bucket can pull. A client's choice of k is itself metadata, so a future refinement is to advertise a small fixed menu of allowed lengths rather than a free range, making buckets uniform across clients (privacy by uniformity, the property that makes BIP 158 filters identical for everyone) — see §13.

Quota is priced by coarseness. A coarse prefix is both more private and more expensive to serve (a bigger bucket is more delivered traffic), so a prefix item costs units scaling inversely with k rather than a flat one unit (§9) — the bandwidth cost of privacy is surfaced directly in the quota rather than hidden, and a client cannot cheaply pin a near-half-chain bucket under a single "watch."

Privacy properties, stated honestly. The server learns the set of buckets a client subscribes to, and nothing finer; within a bucket the real target is indistinguishable from its 2ᵏ⁻ᵇⁱᵗ cover, and there is no fetch step to narrow it. Two residuals are the client's to manage, not the server's: a wallet that registers many precise prefixes leaks a coarse silhouette of itself (prefer few coarse buckets over many fine ones), and a stable prefix per target is intersection-resistant — repeated matches only re-assert "watches bucket p" — whereas a prefix that varies per query can be intersected back down. The firehose remains the only zero-disclosure option; this is the knob between it and the exact watches, not a replacement for either.

7.6 Bounded historical rescan (RescanBlocks)

SetCursor replay (§6.2) is forward-from-cursor and synthesizes BlockConnected events, but it does not replay watch matches — the client reconstructs them from the replayed blocks against its own watch-set, and the window is capped at MAX_REPLAY_BLOCKS. RescanBlocks is the pull dual: the server runs the matcher over a caller-specified historical range and returns the matches directly.

message RescanBlocks {          // SubscribeControl.rescan_blocks = 14
  uint32 from_height = 1;       // inclusive
  uint32 to_height   = 2;       // inclusive, >= from_height
}

Semantics:

  1. Scans this connection's watch-set only. The rescan runs against a snapshot of the requesting subscription's watch-set in an isolated, single-subscriber matcher — it can never deliver to another connection. It reproduces the four confirmed match bodies a live block scan would (ScriptMatched output- and input-side, OutpointSpent, TxidMatched, PrefixMatched, all confirmed=true), in height order. Depth alarms and txid-lifecycle transitions are not reproduced — those are forward-looking/stateful and have no meaning over a closed historical range.

  2. Deterministic, in-band ack (mirrors SetCursorResult). Exactly one RescanResult precedes any matches:

    message RescanResult { oneof outcome {
        RescanAccepted accepted = 1;   // from/to (post-clamp) + clamped flag
        RescanRejected rejected = 2;   // reason + tip_height
    } }

    The range is clamped to the current tip (clamped set when the requested to_height exceeded it); an inverted range, a range wholly above the tip, or a span exceeding MAX_RESCAN_BLOCKS (= MAX_REPLAY_BLOCKS, 10 000) is rejected whole (INVALID_RANGE / RANGE_TOO_LARGE). Other reasons: NO_SOURCE (no local block bodies/undo), EMPTY_WATCH_SET, RATE_LIMITED, CONCURRENT_RESCAN.

  3. Terminal marker. After the last match the server emits RescanComplete{from_height, to_height, matches}, so the client knows the range is fully drained and how many matches it produced. The stream then resumes its prior live position.

  4. A side query. A rescan does not advance the durable forward cursor, and its match events carry no resume cursor. It is rate-limited per principal, admits one rescan in flight at a time (a second → CONCURRENT_RESCAN), and runs independently of any in-flight SetCursor re-anchor.

  5. Off the consensus hot path. Like replay, the rescan reads blocks (and undo data, when a script/prefix is watched or a silent-payment scan recomputes) the node already holds; it never blocks or backpressures block connection.

  6. Index-accelerated for silent payments (D4). A scan-key rescan normally recomputes each block's tweaks from undo data. When silentpaymentindex is enabled and complete, it instead reads the tweaks from the sp_tweaks index — skipping the undo read and per-tx tweak math — gated per block on the stored row's embedded block_hash matching the block being scanned; a row that is absent, hash-mismatched, or from an incomplete index falls back to recompute for that block. Both paths run the identical kernel, so the fast path changes only speed, never results (an on-vs-off rescan over a reorg-heavy history produces identical matches). The same self-authenticating property that makes the tweaks deep-replay exemption safe (§10) underwrites this fast path.

This closes the streaming-watch-set cold-sync / beyond-MAX_REPLAY_BLOCKS recovery gap without the client walking blocks itself. (BIP157 P2P still covers bulk trustless filter-based sync; this is the server-side push-model equivalent for the watch-set path.) SDK: ResilientWatch::rescan(from, to) / Go ResilientWatch.Rescan(ctx, from, to) — the ack, matches, and RescanComplete arrive in-band on next().

7.7 Silent-payment (BIP 352) subscriptions

Two consumption modes for BIP 352 silent payments ride on the streaming surface. Tier 1 (client-side scan) is live: the node emits BlockTweaks per connected block and replays them by index on from_cursor resume, and requires the node's tweak index (silentpaymentindex=1). Tier 2 (server-side scan-key matching) is live for both confirmed and mempool payments and works with the index off (recompute); a complete index only accelerates its rescan cold-sync. Both first-party SDKs (satd-events-client, satdevents) expose both modes with typed events and helpers (see below). The messages are additive — existing subscribers are unaffected, and the schema version does not bump (§4).

Tier 1 — client-side scan (the recommended, zero-custody mode). A new firehose category streams every block's public tweak data; the client runs the one-ECDH-per-transaction scan locally, so the scan key never leaves the device. The tweaks category is bit 8, and unlike the other category bits it is not part of the categories = 0 ("all") default — a pre-existing 0-subscriber never begins receiving tweak volume after a node upgrade; a client must set bit 8 explicitly. SubscribeRequest gains five tweak-only knobs:

// SubscribeRequest additions (apply only when the tweaks bit is set):
//   uint64 tweak_dust_limit   = 4;  // drop entries whose max_value is below this floor; 0 = unfiltered
//   bool   tweaks_only        = 5;  // compact entries: 33-byte tweak alone, no txid/max_value (BlockTweaks only)
//   bool   mempool_tweaks     = 6;  // also stream MempoolTweak at admission (Tier 1.5); default off
//   bool   tweak_outputs      = 7;  // include taproot_outputs on BlockTweaks entries; default off (MempoolTweak always has them)
//   bool   tweak_unspent_only = 8;  // drop entries whose taproot outputs are all spent; needs a block source

message BlockTweaks {              // NodeEvent body 28 — one per connected block ≥ taproot activation
  bytes  block_hash           = 1;
  uint32 height               = 2;
  repeated TweakEntry entries = 3; // empty = indexed block with no eligible txs
  bool   filtered             = 4; // a per-subscription filter dropped entries
                                  // (tweak_dust_limit floor, or tweak_unspent_only cut-through)
}
message TweakEntry {
  bytes  tweak     = 1;  // 33-byte compressed public tweak T = input_hash · A (always present)
  bytes  txid      = 2;  // omitted under tweaks_only
  uint64 max_value = 3;  // largest eligible output value in sats; omitted under tweaks_only
  repeated TaprootOutput taproot_outputs = 4; // scan candidates; always on MempoolTweak, opt-in on BlockTweaks
}
message TaprootOutput {            // one of the transaction's taproot outputs — a scan candidate
  uint32 vout          = 1;
  bytes  output_pubkey = 2;  // 32-byte x-only key, INTERNAL byte order (do not reverse)
  uint64 value         = 3;  // sats
}
message MempoolTweak {             // NodeEvent body 30 — one per SP-eligible admission (Tier 1.5)
  TweakEntry entry = 1;            // always full: txid + taproot_outputs required for confirm-free matching
}

Tweak data is public chain-derived data (same posture as getblockfilter), so it serves under the existing stream:subscribe capability.

  • Live. On each connected block ≥ taproot activation, the node emits one BlockTweaks (after that block's BlockConnected, at the same durable cursor) built from the just-committed sp_tweaks row — but only while at least one tweak subscriber is attached, so a node with the index on but no tweak consumers pays nothing on the bus. The event carries the block's own hash, so a client that scanned a stale block detects the reorg from the (block_hash, height) it already holds and re-anchors like any consumer.

  • Replay. from_cursor replay for a tweaks-only subscription is index-backed point lookups and — because each row embeds the hash of the block it describes — is exempt from the MAX_REPLAY_BLOCKS clamp (a phone cold-syncs from taproot activation in one subscription). The unclamped span is paged lazily: the server reads it in bounded chunks off the async runtime and streams them under channel backpressure, so peak memory is O(chunk) rather than the whole taproot era, and a slow client throttles the read instead of forcing the server to buffer. A storage/decode error mid-cold-sync is surfaced in-band (the stream ends with an error) rather than skipped as an empty height, so the gap can never pass unnoticed. A mixed-category subscription keeps the clamp for every category; interleaving an unclamped tweak replay with clamped chain events would break cursor ordering (§10).

  • Filters. tweak_dust_limit drops entries whose max_value is below the floor and sets filtered = true; tweaks_only strips txid/max_value, leaving the 33-byte tweak alone. Both apply per subscription, on live and replayed events alike.

  • Cut-through (tweak_unspent_only). Drops entries whose taproot outputs are all already spent on the confirmed chain. Spentness decides only whether an entry survives, never which outputs it carries: an entry with one spent and one live output survives carrying both, because BIP 352 scanning walks k = 0, 1, 2, … and stops at the first k that does not match a carried output — trimming the spent one would truncate the walk and hide a live coin at a higher k. Cut-through never decides that a coin is not yours, only that a coin is gone. Dropped entries set filtered = true, like the dust floor. It is a modifier on the tweaks category (INVALID_ARGUMENT without bit 8) and, because the outputs whose spentness is tested are re-derived from the block, it needs a block source (FAILED_PRECONDITION without one).

    This is a balance scan, not a restore. Spentness is a live UTXO-set test at serve time, not "was it spent as of this height": a coin received at height H and spent at H+100 is absent from a scan of H that runs after H+100. A wallet reconstructing transaction history must leave the flag off, or its history will be missing every coin it has already spent. What the flag buys is the cold sync a phone actually wants — the taproot era minus the coins that no longer exist — at the cost of one block read per event, the same read tweak_outputs already pays for.

    Two edges are deliberate. A block that cannot be read (pruned, or missing) serves the entries unfiltered rather than failing: a superset costs the client redundant ECDH, while an error would end the scan. Note that those entries also arrive with an empty taproot_outputs even under tweak_outputs, since the outputs are re-derived from the block that could not be read — an empty list means "the server could not read the block", never "no candidates", and a client must fall back to the block itself rather than treat the entry as having nothing to scan. And a reorg that un-spends an output does not retract anything already served — the client re-anchors on the (block_hash, height) it holds and rescans the affected range, as it does for any reorg. MempoolTweak is never cut through: its outputs are unconfirmed, so no confirmed UTXO set contains them, and filtering on that would drop every mempool tweak.

  • Outputs (tweak_outputs). Off by default the confirmed firehose is lean — a BlockTweaks TweakEntry carries no taproot_outputs, and a client that wants to confirm a candidate fetches the block (which it holds or can request). Set tweak_outputs = true and the server re-derives each entry's taproot outputs from the block and includes them, so the match is confirmed in-band. It is a modifier on the tweaks category — refused with INVALID_ARGUMENT without bit 8, and with FAILED_PRECONDITION on a node with no block source. Outputs are re-derived at serve time (the on-disk index stores none, so there is no size increase and no reindex), which means a from_cursor cold-sync with tweak_outputs reads each block — heavier than a lean cold-sync, and the reason it is opt-in. The read is paced by the client's consumption (demand-driven streams with backpressure), so it cannot be used to make the node read faster than the client consumes. tweaks_only and tweak_outputs compose: the former strips txid/max_value, the latter still adds the outputs the subscriber asked for. Unaffected on MempoolTweak, which carries its outputs regardless.

  • In-band rejects. A tweaks subscription against a node with silentpaymentindex=0 is refused with FAILED_PRECONDITION; a from_cursor (replay) request while the index is still backfilling is likewise refused, because serving a clamped window would let a light client silently miss payments below the backfill frontier.

Tier 1.5 — mempool-time tweaks (mempool_tweaks). Setting mempool_tweaks = true alongside the tweaks bit additionally streams a MempoolTweak (body 30) at each SP-eligible transaction's admission, so a zero-custody client detects an incoming payment at mempool latency without uploading a scan key — the same latency Tier 2 gets, without the key disclosure. The tweak T = input_hash · A is public data (no custody reason to confirmation-gate it), computed once at admission with the same kernel the index uses, so a mempool tweak and its later BlockTweaks entry are byte-identical.

  • Modifier, not a category. mempool_tweaks requires the tweaks bit; setting it alone is refused with INVALID_ARGUMENT. Default false, so a cold-syncing wallet that wants only block tweaks never receives mempool volume.
  • Ephemeral, best-effort — like the mempool itself. Emitted at admission and on re-admission after a block disconnect. No retraction on RBF/eviction: a replacement emits its own tweak (different inputs → different tweak), and BlockTweaks at connect remains the authoritative confirmed record. Clients dedup by txid (entry.txid, always present — tweaks_only does not apply).
  • Not replayable. MempoolTweak carries no durable cursor and is excluded from the replay window, so a from_cursor resume never yields one. A client that reconnects and missed an admission catches that payment at confirmation via BlockTweaks — no gap, just confirmation-latency for the missed window.
  • Dust floor. tweak_dust_limit applies: a MempoolTweak whose max_value is below the floor is dropped (a single-entry event has no filtered flag).
  • Self-sufficient — carries the outputs. A MempoolTweak entry always includes the transaction's taproot_outputs (each vout, 32-byte x-only key, and value). A client scanning T derives its candidate P_k and confirms the match against these keys in the event itself — no getrawtransaction, which for an unconfirmed tx races eviction/replacement (you could get the signal but find the tx already gone). The outputs are public the moment the tx hits the mempool, so this discloses nothing new, and the node already enumerated them to compute max_value. Keys are internal byte order — compare directly against a derived key with no reversal.
  • Zero cost when unused. The node computes a mempool tweak only while a mempool_tweaks subscriber is attached; otherwise admission does no EC work.

The WS/SSE firehose does not serve the tweaks category in this release (neither BlockTweaks nor MempoolTweak); use the gRPC Subscribe stream (or the JSON-RPC fallback below).

JSON-RPC fallback. getsilentpaymentblockdata "blockhash" ( verbosity dust_limit ) serves the same per-block tweak data for scripts, the reference-implementation differential, and integrators not yet on an SDK. verbosity 0 (default) returns { block_hash, height, tweaks: ["<33B hex>", …] }; verbosity 1 returns each entry as { txid, tweak, max_value }. dust_limit (sats) filters on stored max-values. Errors: -5 unknown/non-active block, -8 disabled index, -1 not yet indexed at that height (row absent). It is a read-only method (served by a read-only listener).

Tier 2 — scan-key watch (the convenience mode). A constrained client may instead register a scan credential and let the node match server-side, receiving push-only wakeups. The scan secret is disclosed to the node so it can run ECDH, but it is not a spending key — spend authority stays with B_spend's private key, which is never sent. Scan secrets are held in-memory per-connection only, zeroized on drop, and never persisted or logged. Registered on the Watch stream:

message SilentPaymentTarget {
  bytes scan_secret      = 1;  // 32-byte b_scan (a watch credential, not a spend key)
  bytes spend_pubkey     = 2;  // 33-byte compressed B_spend
  repeated uint32 labels = 3;  // label ints; the server derives label points. Include 0 to catch change.
}
message AddSilentPayments    { repeated SilentPaymentTarget targets = 1; }  // SubscribeControl 16
message RemoveSilentPayments { repeated bytes scan_pubkeys = 1; }           // SubscribeControl 17; id = b_scan·G (33B)
// SetWatchSet gains: repeated SilentPaymentTarget silent_payments = 9;  (full desired membership)

message SilentPaymentMatched {  // NodeEvent body 29 — one per matched output, on the Watch stream
  bytes  scan_pubkey   = 1;  // registered target identity (b_scan·G) — never echoes the secret
  bytes  txid          = 2;
  uint32 vout          = 3;
  bytes  output_pubkey = 4;  // 32-byte x-only taproot output key
  uint64 amount        = 5;  // sats
  bytes  tweak         = 6;  // 33-byte shared-secret tweak; with k, derives the output key offline
  uint32 k             = 7;  // BIP 352 output counter for this tx
  bool   has_label     = 8;
  uint32 label         = 9;  // valid only when has_label
  bool   confirmed     = 10; // false on mempool acceptance; re-emitted true on block connect
  uint32 height        = 11; // block height when confirmed; 0 in the mempool
  bytes  raw_tx        = 12; // only under SetWatchOptions.include_raw_tx
}

Each target costs one ECDH per eligible transaction, so it is priced accordingly: the per-connection cap is MAX_SP_TARGETS_PER_CONNECTION = 16, each target charges one watch-quota unit, and registration needs the stream:watch capability. A target is identified for removal by its scan pubkey, which the client derives locally. Mempool matches (confirmed = false) are re-emitted confirmed = true on block connect, the same best-effort contract as every other mempool watch kind.

SDK (satd-events-client). Both modes are typed. Tier 1: subscribe with Categories::TWEAKS (plus SubscribeOptions::tweak_dust_limit / tweaks_only) and consume Event::BlockTweaks; the runnable examples/sp_light_scan.rs does the client-side ECDH with bitcoin::secp256k1 and never sends the scan key. Tier 2: WatchHandle::add_silent_payments / remove_silent_payments (and the ResilientWatch / WatchSetBuilder equivalents, which re-register scan keys on every reconnect — the D3 custody model) register SilentPaymentTargets and deliver Event::SilentPaymentMatched; examples/sp_wallet.rs re-derives each match's spending key offline from tweak + k. The scan-key helpers need the crate's default bitcoin feature (to derive the removal identity b_scan·G).

SDK (satdevents, Go). The same two modes, same names: SubscribeOptions{Categories: CategoryTweaks, TweakDustLimit, TweaksOnly, MempoolTweaks, TweakOutputs} with *BlockTweaks / *MempoolTweak for Tier 1, and WatchHandle.AddSilentPayments / RemoveSilentPayments (plus the ResilientWatch and WatchSet equivalents) with *SilentPaymentMatched for Tier 2. examples/sp_light_scan and examples/sp_wallet are the counterparts of the two Rust examples. The removal identity b_scan·G is derived without a curve dependency — SilentPaymentTarget.Validate is an in-tree on-curve check, so the published SDK graph stays gRPC + protobuf and consuming it never forces a secp256k1 implementation on an application that already has one.

7.8 Node-health status events

Every other body on this surface describes the chain or the mempool. A StatusEvent describes the node itself — a stalled tip, a filling disk, a peer-starved daemon — so an operator can drive alerting off the same stream they already consume, instead of polling getblockchaininfo on a timer.

The status category is bit 16. Like tweaks, it is not part of the categories = 0 ("all") default: a subscriber written against an older node never begins receiving a body it has no parser for. Unlike tweaks it has no index prerequisite.

Authorization: status requires rpc:read in addition to stream:subscribe. Every other category describes public chain data, which is what stream:subscribe exists to hand out. StatusEvent describes the host — free bytes on the node's volume, connected peers split inbound/outbound, the mempool byte cap with its occupancy and mempoolminfee, tip height, IBD state, reorg depth and fork heights. That is the content of getblockchaininfo, getpeerinfo, getmempoolinfo and getwarnings, and satd-auth separates the two capabilities precisely so an operator can issue a streaming token to a wallet backend or tenant without granting node read RPC. A token holding only stream:subscribe is refused with PERMISSION_DENIED (gRPC) or 403 (WS/SSE) when it requests bit 16, and a mid-stream SetCategories / SetWatchSet from such a token has the bit stripped. This applies only where -authfile is configured: with no token store the node behaves exactly as before, and the operator principal (cookie / rpcuser / rpcauth) holds every capability.

It is served on the carriers that have a per-subscriber category mask to opt in with: gRPC Subscribe, gRPC Watch, WS, and SSE. It is not emitted on the ZMQ nodeevent topic. ZMQ has no per-subscriber mask — only an all-or-nothing topic toggle — so honoring "explicit-only" there is impossible: emitting status would hand a new body type to every already-deployed nodeevent consumer on upgrade, and their only escape would be eventszmqnodeevent=0, which silences every other envelope too. Operators consuming health over ZMQ should use the alertfile webhook dispatcher or -alertnotify instead.

message StatusEvent {
  StatusKind     kind     = 1;  // ibd_complete | tip_stall | disk_low |
                                // mempool_congested | peer_floor | deep_reorg |
                                // template_invalid
                                // (0 = unspecified; never emitted by satd)
  StatusState    state    = 2;  // raised | cleared | edge
                                // (0 = unspecified; never emitted by satd)
  StatusSeverity severity = 3;  // info | warning | critical
                                // (0 = unspecified; ranks lowest — see below)
  string message = 4;                // human-readable; log it, do not parse it
  map<string, string> details = 5;   // kind-specific fields (see below)
}

Level-triggered. A standing condition emits exactly one raised on entry and exactly one cleared on recovery, with hysteresis between the two thresholds so a value hovering at the line does not flap. Observations with no recovered state (IBD finishing, a deep reorg landing) are edge and never produce a cleared.

Not replayable. A status event carries no cursor and is excluded from the replay ring, so a from_cursor resume never yields one. Durability comes from re-evaluation instead: the detectors re-examine every condition at startup and re-raise the ones still standing, which is what makes health alerting at-least-once across a restart. template_invalid is the exception: it is evaluated when a template is built, so it stands again after a restart only once a template fails again. A condition that both raised and fully cleared while a consumer was away is stale by definition and is not reconstructed.

details keys by kind. Values are strings. Most are decimal numbers, but not all — parse per key rather than assuming the whole map is numeric.

Kind Keys
ibd_complete height
tip_stall raise: seconds_since_block, threshold_seconds, tip_height. Cleared by a block: height. Cleared by the poll: seconds_since_block, threshold_seconds
disk_low free_bytes, threshold_bytes; clear_threshold_bytes when cleared by recovered space
mempool_congested bytes_used, bytes_cap, threshold_pct, mempoolminfee_sat_per_kvb (raise only)
peer_floor peers, peers_outbound, peers_inbound, threshold (raise only)
deep_reorg depth, from_height, to_height, fork_height
template_invalid raise: check (structural or full), origin (getblocktemplate, generate or stratum), reason (Core's reject reason), height, and txid when the offending transaction was found

deep_reorg figures are exact. Depth, fork height and the reconnected chain are read from the reorg log record that perform_reorg writes and fsyncs, not reconstructed by counting disconnect events off the event bus — so a reorg deep enough to overrun the bus ring is reported with the same precision as a shallow one. to_height is the new tip, not the first reconnected block.

Any kind may additionally carry reason — a non-numeric token on a cleared event emitted because the detector can no longer evaluate the condition, rather than because the condition recovered:

Token Meaning
detector_disabled The operator set this detector's threshold to 0.
mempool_cap_zero maxmempool is 0, so there is no occupancy ratio to evaluate. alertmempoolfullpct is still armed — this is not the operator turning the detector off, and a consumer that suppresses detector_disabled should not suppress this.

Additive by construction. details is a string map and StatusKind is an open enum: new kinds and new detail keys ship without a schema bump (§4). A consumer must tolerate an unrecognized kind — message and severity remain meaningful — and must not require any particular details key to be present.

Severity ordering, and the zero value. severity is meant to be filtered by a threshold ("page at warning and above"), so the ranking is normative:

UNSPECIFIED  <  INFO  <  WARNING  <  CRITICAL  <  any unrecognized value

An unrecognized severity ranks above CRITICAL deliberately: a level the consumer cannot name is not one to filter out silently, so an additive change to the taxonomy fails loud rather than dropping alerts. Implementations MUST NOT rank by numeric proto value, which would order a future level by its tag rather than its meaning.

STATUS_SEVERITY_UNSPECIFIED (0) is the exception and ranks lowest. It is proto3's zero value, so it is what an absent field decodes to — an absence of information, not a loud condition. A consumer that ranks it with the unrecognized values pages on every producer that forgets to set the field.

satd never emits 0 for kind, state, or severity: the node's internal enums are closed and the encoder writes an explicit value for each. The zero value is therefore a contract for third-party producers, re-encoding relays, and future schema versions — a consumer must handle it, but it will not arrive from a stock node. For the same reason a consumer should distinguish "unset" from "unrecognized" rather than folding both into one case: they call for different responses (fix the producer vs. upgrade the client). Note that severity is a fixed function of kind in v1, so a consumer that receives UNSPECIFIED severity can recover the intended level from the kind.

The same detections also drive the node's warnings registry, so getwarnings, the Core-compatible -alertnotify hook, and this stream can never disagree about node state. Operator-facing configuration (detector thresholds, the alertfile webhook dispatcher) is documented in the Operator Manual, not here.

Health events are also deliverable over outbound webhooks, whose bodies are byte-identical to the JSON on this surface — one schema, two transports. The delivery contract (headers, signature, retry and drop semantics) is specified in webhooks.md.

8. Reorg events

Reorgs are first-class, emitted in-process as consensus ground truth — a bitcoind sidecar structurally cannot do this (ZMQ has no reorg semantics; a sidecar can only infer a reorg by diffing headers).

message Reorg {
  uint32 from_height = 1;  bytes old_tip = 2;  // tip being abandoned
  uint32 to_height   = 3;  bytes new_tip = 4;  // new active tip
}

The Reorg marker is emitted around the existing BlockDisconnected → BlockConnected sequence the connect path already produces during a reorg, giving clients an explicit fork-point marker rather than forcing them to reconstruct one from a stream of disconnects.

9. Auth, capabilities & quotas

Reused wholesale from the unified auth layer (satd-auth); this API adds no new auth surface. With no token store configured the transports are open (loopback-trust, matching today's events-gRPC behavior); a remote bind requires a token store.

Action Capability Quota
Open a stream; receive the firehose (categories) stream:subscribe —
AddScripts / AddOutpoints / AddTransactions / AddDescriptor stream:watch per-token watch quota + per-add rate limit
AddScriptPrefixes (§7.5) stream:watch same quota, but priced by coarseness (see below)
AddSilentPayments (§7.7 Tier 2) stream:watch one unit per target, capped at 16/connection (MAX_SP_TARGETS_PER_CONNECTION)
Remove* — releases each item's unit immediately

Quota unit (N items = N units). A unit is one watched item: an AddOutpoints of N outpoints consumes N units, an AddScripts of N scripthashes consumes N, and a depth alarm consumes one unit per (txid, depth) pair. A lifecycle watch's auto_close_depth rides free (it is not a separate item). The one exception is a prefix watch (§7.5), which is priced by coarseness — units scale inversely with bits (a coarser bucket delivers more traffic) rather than a flat one unit — so the quota reflects served bandwidth, not item count, for the one watch kind whose cost is not one-item-one-match. This bounds the work a tenant pins on the node, not the message count. An over-quota add is refused in-band with a WatchAddRejected (QUOTA_EXCEEDED, §7.3.2) naming the refused items, without tearing down the subscription; so is an add the per-add rate limit throttles (RATE_LIMITED). RESOURCE_EXHAUSTED (gRPC) and 429 (WS) are returned only when a stream is opened. A SetWatchSet replace reports its own outcome as WatchSetResult (§11.2).

Lease lifecycle. Each item holds a WatchLease (RAII) tracked per item in a subscription-scoped WatchSet. So Remove* returns that item's unit immediately, which is what makes a long-lived client that rotates a watch-set (e.g. a sliding descriptor window, §11) viable without monotonically exhausting its quota. Dedup is cross-message: re-asserting an item the subscription already holds is charged once. The WatchSet is scoped to the subscription, not the control stream, so an HTTP/2 half-close (client closing its send side while keeping the response open) cannot release quota while watches stay live. A per-add rate limit bounds the rate of effective (net-new) adds via the per-principal token bucket, shedding over-budget adds without dropping the connection — a connected client must not be able to spam adds bounded only by steady-state quota.

10. Consensus-safety invariants (non-negotiable)

satd's first value is being a correct Core-compatible node; the streaming API must not compromise that. These are structural guarantees, not policies:

  • The event bus is publish-only out of connect_block / accept_tx. The matcher (§7.1) only reads data the node already holds; it adds no code to, and takes no lock on, the consensus path.
  • A slow client never backpressures the publisher. Degradation is drop-with- notice (§6.4): broadcast send is non-blocking and lossy, and per-subscriber delivery is non-blocking try_send. This is a safety property, not just UX.
  • Watch/quota state is lock-light (inverted-index maps gated by lock-free watcher counts; atomics), off the consensus hot path.
  • Listeners run on the API runtime only (§3.1), never the core block-connecting runtime.
  • The tweak deep-replay exemption is bounded by two structural facts, not a policy knob. The tweaks-only replay (§7.7) waives the MAX_REPLAY_BLOCKS clamp only because (1) each served sp_tweaks row embeds the hash of the block it describes, committed in the same atomic batch as the chainstate — so a served row is self-authenticating and a mid-read reorg surfaces as a hash mismatch, never as silently stale data — and (2) the exemption is gated on sp_index.complete, so a partial index is never served as authoritative (a replay request against an incomplete index is refused in-band). A mixed-category subscription keeps the clamp for all categories: interleaving an unclamped tweak replay with a clamped chain replay would emit non-monotonic cursors, breaking the resume contract. These are the only conditions under which the clamp is waived.

11. Descriptor convenience layer

Pure library work on top of the §7 script primitive — no consensus path, lowest risk.

message AddDescriptor {
  string descriptor = 1;   // rust-miniscript parseable, public-key-only
  uint32 gap_limit  = 2;   // window size; capped at MAX_DESCRIPTOR_WINDOW (1000)
  uint32 start      = 3;   // window start index (default 0)
}

message RemoveDescriptor {
  string descriptor = 1;   // byte-matches the AddDescriptor string
}

The server expands the descriptor via rust-miniscript over the window [start, start + gap_limit), derives the watch scripts, registers them with the §7 matcher, and retains the descriptor → derived-scripthashes membership for the connection — the address-watching-as-outpoint-watching convenience the base primitive was designed for. A BIP-389 multipath descriptor (.../<0;1>/*) is split into its branches and each branch expanded over the same window, so it yields up to branches × gap_limit scripts and costs that many watch units; the branch count is capped at 2 (MAX_DESCRIPTOR_BRANCHES) — more is rejected before the split. Expansion is bounded per branch (MAX_DESCRIPTOR_WINDOW, a DoS limit, so ≤ 2000 scripts for a 2-branch descriptor) and rejects hardened-wildcard and any secret-bearing descriptor at the type level (Descriptor<DescriptorPublicKey>), so no signing material can be submitted.

Each derived scripthash carries an owner count — the number of sources (a direct AddScripts and/or one or more descriptors) that currently watch it. A script's quota unit and matcher registration are released only when its last owner goes, so a scripthash shared by two descriptors, or by a descriptor and a direct add, is never dropped while any source still wants it.

The number of distinct descriptors a single connection may retain is capped (MAX_DESCRIPTORS_PER_CONNECTION = 256). The cap exists because a descriptor whose window expands entirely to already-watched scripts charges no quota unit (nothing is net-new) yet still costs a retained membership entry, so without it a client could grow that map without bound, invisibly to the watch quota and the per-connection watch-set cap. A connection at the cap must RemoveDescriptor before adding a new descriptor; re-asserting (sliding) an already-retained descriptor never counts against it.

RemoveDescriptor drops a descriptor's whole window: every scripthash it contributed whose last owner this removes is released (membership is retained precisely so the server knows which those are — it does not re-derive). Removing an unknown descriptor is a no-op. Re-sending AddDescriptor for the same descriptor string with an advanced start reconciles the slid window server-side: scripts that left the window are released (subject to the same last-owner rule), scripts that entered are added, all-or-nothing on quota — the client no longer has to Remove* the trailing scripts by hand.

Gap-limit tracking is still a client concern, by design. Retaining membership lets the server manage a window (slide, remove); it does not make the server track derivation progress. An earlier approach had the server follow how far the client had derived and push a DescriptorNeedsAddresses side-channel telling it to extend the window. We rejected that and it stays rejected: the server never tracks how far the client has advanced, never decides when to slide, and emits no unsolicited nudge. The client owns address-generation policy and drives the window by advancing start (or dropping it with RemoveDescriptor); the server only expands, watches, reconciles, and releases. The DescriptorNeedsAddresses body (field 15) is reserved for wire-compat but is never emitted.

11.1 Match attribution

A ScriptMatched for a descriptor-derived script carries descriptor_matches — the descriptor(s) whose window currently contains that scripthash, each with the exact (branch, derivation_index) the server derived it at. This saves a descriptor consumer from maintaining its own reverse scripthash → descriptor index (the work it offloaded to the server by sending a descriptor in the first place): a deposit hit can be routed to the right account directly. It is empty for a directly-watched (AddScripts) script, and carries more than one entry when the script falls in more than one descriptor's window (overlap). The server keeps a per-connection reverse index over the retained membership (§11) to attach it.

branch is the 0-based BIP-389 multipath branch (<0;1> → external = 0, change = 1; always 0 for a single-path descriptor) and derivation_index is the absolute index — together the coordinate the server actually derived the script at, ready to use with no gap_limit arithmetic. This is correct for every descriptor shape, including fixed (non-wildcard) and multipath descriptors where a positional offset could not be reversed. Attribution stays consistent with the gap-limit-is-a-client-concern principle (§11): it is reactive (emitted only on a match, never an unsolicited "derive more" nudge) and the server still tracks no derivation progress — it reports only where a matched script sits, never advancing a gap limit on the client's behalf.

11.2 Atomic whole-set replace (SetWatchSet)

SetWatchSet carries the complete desired watch-set — scripts (+floors), outpoints, txid lifecycles, depth alarms, descriptors (+windows), prefixes, and the category filter — in one message, and asks the server to become it:

message SetWatchSet {
  uint32 categories                   = 1;  // 0 = all
  repeated bytes scripthashes         = 2;
  repeated uint64 min_values          = 3;  // parallel to scripthashes (empty = no floors)
  repeated Outpoint outpoints         = 4;
  repeated AddDescriptor descriptors  = 5;  // expanded over each window, as AddDescriptor
  repeated ScriptPrefix prefixes      = 6;
  repeated WatchLifecycle lifecycles  = 7;  // { txid, auto_close_depth }
  repeated WatchDepthAlarm depth_alarms = 8; // { txid, depth }
}

The server reconciles it under the watch-set lock, by effective scripthash coverage (descriptors expanded): items in both the old and new set keep their registration and quota lease untouched — including a scripthash whose mechanism changes across the replace (a direct add becoming descriptor-covered, or vice versa), which a client-side Add*/Remove* diff cannot see and would briefly unwatch. Departed items are released; net-new items are charged. The replace is all-or-nothing on the whole target for three independent reasons:

  • Quota — if the target's total unit cost exceeds the principal's quota the watch-set is left unchanged (reason = QUOTA_EXCEEDED, required = units needed, quota = the unit ceiling).
  • Entry cap — the replace is bounded by the same per-connection watch-set entry cap as the incremental adds on that carrier (WebSocket streamwsmaxsubscriptions; a prefix counts as one entry). A target with more entries than the cap is refused whole (reason = CAP_EXCEEDED, required = entry count, quota = the cap). This bound applies even to a loopback/no-auth connection, which has no quota — so one SetWatchSet frame can never install more than the cap. (gRPC entry-caps neither its incremental adds nor a replace; quota is its bound.)
  • Malformed input — because a SetWatchSet is a full snapshot, any element the server cannot parse or expand (a bad scripthash, outpoint, txid, descriptor, or prefix — or a min_values length that does not match scripthashes) refuses the whole snapshot (reason = MALFORMED, required/quota = 0) and leaves the live set unchanged. It is not silently dropped — dropping one item would shrink the snapshot and unwatch still-wanted scripts while reporting success. (This is the one place the incremental Add* paths differ: those are best-effort and skip an unparseable item, since they only grow the set.)

Unlike the incremental adds, a replace does not charge the per-add rate limiter: it is the watch-set re-establishment primitive (reconnect / SDK reload), and rate-limiting it would let a reconnect storm block clients from restoring their watches. Steady-state size stays bounded by quota and the entry cap above.

The outcome is deterministic and in-band (mirrors SetCursorResult):

message WatchSetResult {
  oneof outcome { WatchSetAccepted accepted = 1; WatchSetRejected rejected = 2; }
}
message WatchSetAccepted { uint32 added = 1; uint32 removed = 2; uint32 unchanged = 3; }
message WatchSetRejected {
  enum Reason { REASON_UNSPECIFIED = 0; QUOTA_EXCEEDED = 1; MALFORMED = 2; CAP_EXCEEDED = 3; }
  Reason reason = 1; uint64 required = 2; uint64 quota = 3;
}

This is the primitive an SDK reload/realign uses: one round-trip, gap-free, quota-correct even for a same-size swap at exactly the quota ceiling — never a client-computed delta whose ordering can strand coverage (send Remove* first and an at-quota client is briefly unwatched; send Add* first and the over-quota adds are silently dropped). On WebSocket the same operation is {"type":"set_watch_set", ...} and the result a {"category":"watch_set_result", "outcome":"accepted"|"rejected", ...} event.

12. Operator limits

Every remote-facing surface is bounded so it cannot be driven to fd / memory / task exhaustion, mirroring the rest of the node's admission controls. All are restart!-classified config; 0 means unlimited:

Key Default Bounds
streamwsmaxconns 256 concurrent /ws + /sse connections
streamwsmaxsubscriptions 256 watch-set size per WS connection; an add past it is refused with watch_add_rejected (cap_exceeded)
streamwsmaxmessagebytes 262144 a single inbound WS control frame
eventsgrpcmaxconns 64 concurrent gRPC connections
eventsgrpcmaxsubscriptions 256 concurrent Subscribe + Watch streams, across all gRPC connections

Admission shedding runs ahead of authentication and request-body buffering, so a connection flood — authenticated or not — is bounded before it does work.

12.1 Mempool spend-side prevout retention

The matcher's mempool spend-side path (§7.2, §7.5) needs prevout data the spending input does not carry. Whatever it needs must be captured at admission — prev_outputs is resolved for validation and then dropped, so there is no off-hot-path way to recover it later. streamprevoutmeta tunes how much is kept per mempool input, trading memory for matcher capability:

Value Per-input cost Enables
hash scripthash only (32 B) exact-script + prefix bucket matching; client resolves outpoints itself
amount (default) + prevout value (8 B) a mempool-input min_value floor on spend-side script matches
full + full prevout scriptPubKey (variable, one heap allocation) a chainstate-less client confirming a mempool prefix spend without resolving any outpoint

Retention is paid for every mempool entry regardless of subscribers (it happens at admission), so the default is the cheapest tier that makes min_value work out of the box; full is opt-in because it both costs the most and widens the data the node holds in memory. Unlike the §12 caps this is a live-reloadable mempool-policy key (SIGHUP): a change governs subsequent admissions, while entries already pooled keep whatever they were admitted with (mempool matching is best-effort by contract, §6.2). The confirmed-block spend side is unaffected — it always recovers full script and value from undo data (§7.2).

Memory accounting. Retained prevout metadata is held alongside each mempool entry and is not counted against maxmempool (which bounds only serialized transaction weight). Budget for it separately. The worst case is bounded by the mempool's input count: roughly entries × inputs_per_entry × per_input_cost, where per_input_cost is 32 B (hash), 40 B (amount), or 32 B + the prevout script length (full, typically ~34 B for witness outputs, but attacker- influenceable up to the standardness script-size limit). For a full default-size mempool this is sub-1% of the mempool's own footprint under hash/amount; under full size it as maxmempool plus that per-input script overhead, and prefer a lower tier on memory-constrained nodes that do not need chainstate-less spend confirmation.

13. Open questions

These are genuine open design points, deferred until a real consumer's feedback can settle them ahead of any v1 freeze:

  • Anchor consumer. Which downstream integrator co-designs the surface before a v1 freeze. The wire shapes here are a working contract, not yet a commitment.
  • Stability commitment. When to freeze the wire surface as a committed v1 for satd consumers. This API is a deliberate satd differentiator — a native, in-process consensus stream (first-class reorgs, cursor-resumable matches, mempool coverage) that an out-of-process sidecar over Core's ZMQ structurally cannot match (§8). The goal is a stable satd contract that pulls integrators onto the node itself, not a lowest-common-denominator cross-node specification that would commoditize the advantage away. Any broader standardization is a downstream governance question, explicitly not a design goal for this surface.
  • Cursor opacity. Whether to keep (height, tx_index, mempool_seq, instance_id) as structured, debuggable fields (current design) or move to an opaque token that is future-proof against cursor-format changes.
  • Descriptor expansion bounds. The right max gap-limit / max derived watch-set per token, and how it composes with the §9 per-item quota.
  • Prefix-watch granularity (§7.5). Whether to expose bits as a free range within [streamprefixminbits, streamprefixmaxbits] (current design) or a small fixed menu of allowed lengths — the menu makes buckets uniform across clients (so the choice of k is not itself a fingerprint) at the cost of client flexibility. Also: whether mempool spend-side matching (which costs one retained scripthash per mempool-entry input) is worth its memory on nodes with no prefix subscribers, or should be gated behind a "prefix subscriptions enabled" flag.

Cross-reference: ROADMAP.md (strategic framing), Esplora REST API (the surface this consolidates beyond), CHANGELOG.md (user-facing summary).