Skip to content

rpc: isolate public HTTP capacity from management RPC - #1685

Merged
gosunuts merged 2 commits into
mainfrom
feat/1586-public-query-listener
Oct 27, 2026
Merged

gosunuts merged 2 commits into
mainfrom
feat/1586-public-query-listener

Conversation

@gosunuts

@gosunuts gosunuts commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Public Esplora/REST connections currently consume the same pre-authentication connection budget as management JSON-RPC. Add opt-in --public-bind so public HTTP traffic has a separate bounded listener and cannot consume management connection permits.

When enabled, /api, /esplora and enabled REST routes move to the public port; --rpc-bind serves authenticated JSON-RPC only. The public listener closes JSON-RPC requests with 404 before authentication or body upload. Without this option, the combined listener, routes and 128-connection default remain unchanged.

  • Add independent positive --rpc-max-connections and --public-max-connections limits (128 each), with CLI/environment/TOML resolution through the existing option table.
  • Reuse the existing connection permits, per-source bounds, timeouts and dispatcher quotas. Both bindings share one Context and its authoritative chain, mempool, indexes and broadcast gateway; each binding retains its own Handler cancellation latch and blocking quota.
  • Bind both sockets before starting listeners, record every spawned listener in NodeServices, and join both through existing startup rollback and shutdown. Preserve existing RpcServer construction and serving APIs.
  • Expose bounded-label occupancy, rejection and request-duration metrics, and update API-34, WF-02, embedding and operator documentation. Public mode retains both existing Esplora namespaces, including backend helpers and transaction broadcast.

Validation on Windows/MSVC, native validation with fjall:

  • RPC server tests: 35 passed, including public keep-alive plus incomplete-head saturation, successful authenticated management RPC, public rejection/recovery, finite management admission, independent cancellation and closed listener routes.
  • Existing HTTP authentication/CORS integration tests: 21 passed.
  • Layered configuration tests: 25 passed, including wildcard bind conflicts, distinct concrete addresses and simultaneous ephemeral binds; CLI precedence regression passed.
  • Node startup/shutdown and public-bind rollback regressions passed, including socket release and reopening node state after failed startup.
  • Standalone template-provider lockfile includes the RPC metrics dependency; its cargo metadata --locked --offline --format-version 1 check passed.
  • Existing chain_feed::tests::unsettled_or_failed_chain_fences_never_publish_a_stable_hint passed.
  • cargo fmt --all -- --check and Clippy for rpc/node/binary with fjall, all targets and -D warnings passed.

Sync impact: this changes transport admission and service composition, not authoritative synchronization, validation or commit paths. SYNC-03 is relevant to bounded optional RPC resources; existing Context capability, stable-read, gateway and lock-order boundaries are retained. The existing chain-fence proof above exercises refusal during unsettled/failed transitions. These tests establish connection-capacity isolation only: CPU, process descriptor limits, storage and lock contention remain shared, and foreground progress under resource exhaustion is not claimed. Existing process/reference/performance campaigns remain in their owning CI lanes.

Fixes #1586.

RetriggerView in GreptileConfidence Score: 5/5

The changes since the previous review appear safe to merge.

Summary

The PR adds an optional public HTTP listener with a separate connection limit.

  • Public HTTP connections get their own listener and capacity.

No new actionable issues were found in the changes since the previous review. No previous review threads were supplied.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Config[Resolved configuration] --> Bind[Bind both sockets before starting listeners]
  Bind --> Management[Management listener: authenticated JSON-RPC]
  Bind --> Public[Public listener: Esplora and enabled REST]
  Management --> Context[Shared Context and node services]
  Public --> Context
  Services[NodeServices] --> Management
  Services --> Public
  Services --> Shutdown[Stop and join both listeners]
Loading

Reviews (2) · Last reviewed commit: "fix(rpc): address split-listener review ..." · Reviewed by Greptile

@amazon-q-developer amazon-q-developer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

This PR correctly implements the --public-bind feature to isolate public HTTP traffic (Esplora/REST) from the management JSON-RPC listener with independent connection budgets. The core architecture — RpcSurface, RpcBindings, the surface-aware accept loop, route filtering before authentication/body upload, independent ConnectionPermit per surface, and the RequestTimer metrics — is well-designed and coherent.

The startup rollback path (binding management first, then public; StartupGuard teardown releasing both) is correctly implemented and verified by the new lifecycle tests.

Findings

Potentially blocking:

  • config.rs: The public-bind same-address validation exempts port 0, but does so even when both addresses use port 0. Two TcpListener::bind("0.0.0.0:0") calls could receive the same OS-assigned port, causing a runtime bind failure instead of a useful validation error. See inline comment.

Non-blocking observations:

  • lifecycle.rs: The public listener is wired with the management rpc_auth object. Since the public surface filter blocks all JsonRpc routes before authentication, this is inert — but it creates an implicit coupling that could silently authenticate management credentials if the surface boundary ever changes. Worth documenting or structurally separating.
  • server.rs: The set_write_timeout failure before writing the 503-busy response propagates as an error, which means the busy response is silently dropped and the error is logged as a generic "connection setup failed". This is minor but worth being aware of.

All 35 RPC server tests, the lifecycle tests (split_rpc_listeners_start_and_join_through_node_services, public_bind_failure_rolls_back_management_binding_and_node_state), and the existing integration tests pass per the PR description. The implementation correctly preserves the existing combined-listener behavior when --public-bind is not set.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Comment thread crates/node/src/lifecycle.rs
Comment thread crates/node/src/config.rs
Comment thread crates/rpc/src/server.rs

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 18 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread docs/observability.md
Comment thread crates/node/tests/unit/lifecycle/tests.rs Outdated
Comment thread docs/rest-interface.md
Comment thread docs/contracts/wallet-facing.md Outdated
Comment thread docs/contracts/external-api.md Outdated
Comment thread crates/node/src/lifecycle.rs
Comment thread crates/node/src/config.rs Outdated
Comment thread crates/node/src/lifecycle.rs

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 10 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/node/src/config.rs">

<violation number="1" location="crates/node/src/config.rs:391">
P2: This rejects distinct IPv6 link-local listeners that use the same address on different interfaces because `ip()` drops the scope ID. Account for distinct nonzero scope IDs when checking link-local addresses.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread crates/node/src/config.rs
Comment on lines +391 to +394
|| (address.ip() != management.ip()
&& (address.is_ipv4() != management.is_ipv4()
|| (!address.ip().is_unspecified()
&& !management.ip().is_unspecified())))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This rejects distinct IPv6 link-local listeners that use the same address on different interfaces because ip() drops the scope ID. Account for distinct nonzero scope IDs when checking link-local addresses.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At crates/node/src/config.rs, line 391:

<comment>This rejects distinct IPv6 link-local listeners that use the same address on different interfaces because `ip()` drops the scope ID. Account for distinct nonzero scope IDs when checking link-local addresses.</comment>

<file context>
@@ -380,10 +380,20 @@ impl NodeConfig {
+                // Cross-family dual-stack conflicts remain OS-checked at bind.
+                address.port() == 0
+                    || address.port() != management.port()
+                    || (address.ip() != management.ip()
+                        && (address.is_ipv4() != management.is_ipv4()
+                            || (!address.ip().is_unspecified()
</file context>
Suggested change
|| (address.ip() != management.ip()
&& (address.is_ipv4() != management.is_ipv4()
|| (!address.ip().is_unspecified()
&& !management.ip().is_unspecified())))
|| ((address.ip() != management.ip()
|| matches!(
(address, management),
(SocketAddr::V6(public), SocketAddr::V6(management))
if public.ip().is_unicast_link_local()
&& public.scope_id() != 0
&& management.scope_id() != 0
&& public.scope_id() != management.scope_id()
))
&& (address.is_ipv4() != management.is_ipv4()
|| (!address.ip().is_unspecified()
&& !management.ip().is_unspecified())))

@gosunuts
gosunuts enabled auto-merge October 27, 2026 02:57
@gosunuts
gosunuts disabled auto-merge October 27, 2026 02:57
@gosunuts
gosunuts merged commit f024fc3 into main Oct 27, 2026
17 checks passed
@gosunuts
gosunuts deleted the feat/1586-public-query-listener branch October 27, 2026 02:57
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Main brings #1684 opt-in Tor/I2P overlay, #1685 public-listener
PeerRemote design, plus #1676 descriptor scan and #1677 compact-scan
bound. Branch brings state-payload plumbing (BlockPayload wire,
BodyInterest admission, index bundle validation).

Resolutions:
- crates/node/src/config.rs: keep both validations; branch
  validate_index_bundle plus main p2p.overlay.validate.
- crates/p2p/src/listener.rs send_block: main PeerRemote
  parameter on branch BlockPayload body; single-conversion entry.
- crates/p2p/src/listener.rs handle_compact_outcome: keep
  branch Option follow-up return (single enqueue_owned_body_fetch
  publisher per branch 5dee503) with main PeerRemote parameter.
  Main header-carried body_fetch_owned mark not restored: branch
  sync/headers.rs and inbound.rs intentionally supersede it with
  one atomic enqueue; restoring would reintroduce the racy
  double-mark 5dee503 deleted.
- aggregate_budget_tests.rs: main PeerEndpoint address with
  branch is_none assertion matching the Option return.
- crates/rpc/tests/core_compat.rs: main ipv4/ipv6/onion/i2p
  networks assertion (branch had dropped the stale len-3 check).
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merges origin/main at 46de452 (Tor/I2P #1684 over #1685 public-listener split) into stack/11-tools-docs (shared observer/retention ports, ZMQ body interest, corpus and validation-tooling docs).

Listener conflicts (main design wins; no ListenerRole or rpc-public-bind restored): crates/p2p/src/listener.rs keeps the branch sync-owner handle_compact_outcome returning Option<(Hash256, Message)> via enqueue_owned_body_fetch, because the merged tree unanimously uses that API (enqueue/defer/resolve_owned_body_fetch, 3-arg send_headers; record_owned_body_fetch and body_fetch_owned exist nowhere else in the merge). Adopts main's #1684 additive change on the same function: peer_addr is impl Into<PeerRemote> with let peer_addr = peer_addr.into(). crates/p2p/src/listener/aggregate_budget_tests.rs keeps the branch .is_none() assertion for the Option return while adopting main's PeerEndpoint::from address construction. All other paths auto-merged (mempool admission/gateway/mutation/orphan, node config/options, rpc core_compat, sync, docs); Cargo.lock merged without conflict so no metadata regeneration was needed.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merges origin/main (46de452: #1684 opt-in Tor/I2P connectivity, #1685
RPC public/management listener split) into the mining-pool stack.

Intent preserved from main: PeerRemote endpoint identity across the
listener (no ListenerRole or rpc-public-bind restoration), overlay
config validation, and the ipv4/ipv6/onion/i2p getnetworkinfo compat
pins.

Intent preserved from the branch: Stratum mining-pool config and
validation (validate_index_bundle, --pool gating), the exact block
payload owner (Message::Block carries BlockPayload; InboundBlock stores
the wire payload via InboundBlock::new), and the unified compact-block
fetch-ownership contract (handle_compact_outcome returns the follow-up
for the single enqueue_owned_body_fetch site).

AddrMan (auto-merged, no conflicts): branch canonical v1 shape holds --
FORMAT="bitcoin-rs-addrman", SCHEMA_VERSION=1, Stored.format
preserved, old unmarked pre-release files never decoded, no migration
decoders; main's #1684 functional changes apply onto that shape.

Mempool (auto-merged, no conflicts): branch BodyInterest, admission,
gateway, mutation, and orphan intent kept alongside main's newer
behavior; no side silently dropped.

Discarded listener intent: main's split (Block, serialized) listener
plumbing and bool compact-outcome contract give way to the branch's
exact-payload/ownership shapes the merged wire and inbound types
already carry; branch's SocketAddr addr types give way to main's
PeerRemote/PeerEndpoint identity.

Resolutions:
- crates/node/src/config.rs: keep both validators,
  validate_index_bundle plus p2p.overlay.validate.
- crates/p2p/src/listener.rs send_block: main's
  peer_addr: impl Into<PeerRemote> with the branch's
  payload: BlockPayload parameter.
- crates/p2p/src/listener.rs handle_compact_outcome: branch's
  Option<(Hash256, Message)> return (matches the merged
  enqueue_owned_body_fetch caller and BlockPayload body) with main's
  let peer_addr = peer_addr.into() prologue.
- aggregate_budget_tests.rs: main's PeerEndpoint address with the
  branch's .is_none() assertion for the Option contract.
- crates/rpc/tests/core_compat.rs: main's expanded networks assertion
  (branch had removed the length check without replacement).
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merge origin/main (Tor/I2P overlay #1684, public listener #1685) into the
index-snapshot lane. Branch intent preserved: BlockPayload unification,
index bundle snapshot config/tests, and layered config coverage.

Resolutions:
- crates/node/src/config.rs: union; keep validate_index_bundle (branch)
  and p2p.overlay.validate (main), one call each.
- crates/p2p/src/listener.rs send_block: main PeerRemote param
  (impl Into<PeerRemote> + into()) with branch BlockPayload body; all nine
  merged call sites pass payload, so main-only (block, serialized) would
  break every caller and the already-merged forward_block payload form.
- crates/p2p/src/listener.rs handle_compact_outcome: main PeerRemote
  param with branch Option<(Hash256, Message)> flow; the merged caller in
  process_compact_wire_message destructures Some((hash, follow_up)), so
  main-only bool would break it. No ListenerRole restored.
- crates/p2p/src/listener/aggregate_budget_tests.rs: main PeerEndpoint
  addr with branch .is_none() assertion.
- crates/rpc/tests/core_compat.rs: main ipv4/ipv6/onion/i2p networks
  pin (replaces stale len==3 the branch had dropped); branch BlockPayload
  test changes kept via automerge.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Main brings #1684 AddrMan overlay support and #1685 public-listener
addressing (PeerRemote/PeerEndpoint, overlay validate, expanded
getnetworkinfo compat). Branch preserves AssumeUTXO index-bundle
config plus the BlockPayload/linearizable-compact-outcome listener
lineage from its parent stacks.

Resolutions:
- crates/node/src/config.rs: union; keep validate_index_bundle()
  (branch) and add p2p.overlay.validate() (main).
- crates/p2p/src/listener.rs send_block: main's
  impl Into<PeerRemote> peer with branch's BlockPayload body; every
  merged caller (message loop, compact outcome, unit tests) passes a
  payload against wire Message::Block(BlockPayload), so the
  (Block, serialized) signature has no live caller in this tree.
- crates/p2p/src/listener.rs handle_compact_outcome: keep the
  Option-follow-up/enqueue_owned_body_fetch design all merged callers
  use, with main's PeerRemote addressing at the boundary.
- aggregate_budget_tests.rs: main's PeerEndpoint addr with the
  Option-style .is_none() assertion matching the kept signature.
- crates/rpc/tests/core_compat.rs: take main's expanded networks
  assertion (branch had dropped the stale len==3 check); all other
  auto-merged cases from both sides kept.
- No ListenerRole or rpc-public-bind restored; Cargo.lock regenerated
  via cargo metadata.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merge origin/main (46de452, #1684 Tor/I2P overlay + #1676/#1677 scans) into the Lightning lane.

Listener (crates/p2p/src/listener.rs, aggregate_budget_tests.rs): main's #1685 addressing wins — PeerRemote/PeerEndpoint throughout, no ListenerRole or rpc-public-bind restored. The compact-outcome path keeps the branch's enqueue-owned-body-fetch design (Option return, single linearizable enqueue site) because the merged tree carries the branch's 3-arg send_headers/InboundHeaders and enqueue-only BlockSync API; main's bool/record variant has no matching API left post-merge. Tests updated to PeerEndpoint with the is_none assertion.

All other auto-merged regions keep both intents: branch Lightning/ZMQ/admission work plus main's overlay, scan-bound, and RPC compat changes.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merges origin/main at 46de452 (Tor/IVNP overlay connectivity, #1684, atop the #1685 public-listener split) into the performance-evidence lane.

Listener (crates/p2p/src/listener.rs, aggregate_budget_tests.rs): main's #1685 addressing design wins - peer_addr stays impl Into<PeerRemote>, test addrs stay PeerEndpoint, no ListenerRole, no rpc-public-bind. The compact-outcome ownership path keeps the branch's enqueue design (Option return, single linearizable enqueue_owned_body_fetch call site, 3-arg send_headers without body_fetch_owned) because the merged sync/inbound tree carries only that API: record_owned_body_fetch and body_fetch_owned exist nowhere in the merged tree, so main's bool/record caller could not resolve. Branch block_not_found/NotFound release path and block_not_found_tests kept; their sync.block_not_found dependency merged cleanly.

All other auto-merged hunks (mempool admission/gateway/mutation/orphan, node config/options, sync, tx_relay, rpc compat, docs, Cargo.lock) staged as merged. Discarded: main's bool handle_compact_outcome body and record_owned_body_fetch caller, incompatible with the merged sync ownership API.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Merges origin/main (46de452: #1684 Tor/I2P overlay connectivity plus #1685 public-listener work) into the 03-rpc lane (registration-time observer interest, ZMQ body interest on rawtx, retained RPC allocations).

Listener conflicts (crates/p2p/src/listener.rs, aggregate_budget_tests.rs): resolved by composing main's PeerEndpoint/PeerRemote address generalization (impl Into<PeerRemote> params, into() conversion, PeerEndpoint test addr) with the branch Option/enqueue_owned_body_fetch ownership protocol that the merged tree implements (3-arg send_headers, 3-field InboundHeaders, sync enqueue API). Neither bare side compiles: main's bool signature against the branch body (Some/None returns, if-let-Some caller) is a type error, and the branch signature without main's into() line misuses the generalized generic param. No ListenerRole or rpc-public-bind content exists anywhere in the tree, so nothing of that design was restored. Discarded alternative (documented): restoring main's bool/fetch_issued/record_owned_body_fetch callee end-to-end would require rewriting auto-merged files outside the conflicted set (sync.rs method, send_headers arity, InboundHeaders constructors, staged sync tests that call the enqueue API) and would drop the lane's body-interest plumbing; left for Main to revisit if desired.
metaphoric-dev pushed a commit that referenced this pull request Oct 28, 2026
Main intent: public listener (#1685 lineage) and AddrMan v1 plus Tor/I2P
overlay transport (#1684) land in origin/main. Branch intent: BIP331
package relay, BIP332 stale-tip relay, BIP434 negotiation, mempool
admission/gateway/mutation/orphan, and relay policy stay.

Resolutions:
- p2p listener.rs + aggregate_budget_tests.rs: main design wins;
  RelayOffers + relay_offers/dial_routes plumbing kept for branch config.
- AddrMan: single v1 shape (FORMAT=bitcoin-rs-addrman, SCHEMA_VERSION=1,
  Stored.format); old unmarked pre-release files preserved, never decoded;
  no migration decoders or historical fixtures. Main #1684 functional
  changes (validation, scoping, ASMap, anchors, atomic publish, addrv2
  gossip enablement) adapted to that shape. Branch addrv2 rotation kept.
- Endpoint identity: branch PeerEndpoint enum kept (pervasive common
  use); main PeerAddress/PeerNetwork/PeerRemote kept; enum gains
  main-compatible API (address/new/socket_addr/to_canonical/is_ipv4+6,
  From<(T,u16)>, FromStr, Serialize/Deserialize, NetworkLabel mappings).
  Single crate::OverlayConfig (main overlay::OverlayConfig); branch
  DialRoutes/RouteTable/prefilter kept and wired via overlay.dial_routes.
- PeerSession/PeerInfo/PeerToken keep branch field names plus main
  addr (PeerRemote); peer_table keyed by PeerRemote with anonymous
  handling preserved.
- Discouragement/BannedReader, eviction keys, sync staller keys:
  main PeerAddress/PeerRemote routing wins; branch tests ported.
- Node config/options: main settings kept; one overlay field; main
  onlynet (PeerNetwork) with Core-conf keys restored (proxy, onion,
  torcontrol, i2psam, i2pacceptincoming, onlynet); torpassword dropped
  (no main equivalent).
- Discarded/superseded: ListenerRole never restored; rpc-public-bind
  untouched; branch overlay startup/TorControlOwner/establish fns and
  their tests (superseded by overlay.start); branch tor_control module
  (superseded by overlay/tor); legacy migration decoders and fixtures;
  duplicate struct fields/exports/imports from auto-merge.

Verification: zero unmerged paths; git diff --check clean; strict
conflict-marker scan clean; cargo check -p bitcoin-rs-p2p --lib clean.
Test targets have residual type-mismatch errors from the identity
unification (documented for follow-up); parent gates own test proof.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rpc/esplora: preserve management RPC capacity when public query connections are saturated

1 participant